Software Quality.com

Regression vs. risk-based testing: Managing complexity

By Vasudeva Naidu

Software regression testing ensures that changes made to applications will help and not hurt their functionality, but shortcuts are often taken, which can lead to failure. Based on my experience, I have noticed that more than 80% of organizations do not have a well-defined regression test strategy in-place even for the testing of their most critical business applications. If we all know its importance, why do organizations fail to establish a well-defined regression testing strategy?

When I started analyzing this practice across 10 different organizations, for which I had consulted for over a span of three years, I realized that the devil was in the details. Recent increases in IT system complexities have introduced several new variables affecting the successful deployment of regression testing. In addition, IT organizations are not adopting advanced regression test strategies to keep pace with the changing IT dimensions. Risk-based testing (RBT) is one such strategy which is barely used by organizations today. But in my view, this form of regression testing will replace conventional regression testing methods in the near future.

Why do organizations struggle with regression testing?

The majority of testing organizations build regression test beds by accumulating test cases from several releases. Execution of these test beds is based on the availability of time, test environment and data. While the testers executing these regression test sets acquire skills  by practice, they do not undergo any regression testing specific skill development or training to help them adopt advanced methods of testing. This results in the following gaps:

Addressing these challenges is not easy unless undivided attention is provided to each and every point mentioned above. However, if these challenges exist today, then the question that comes to my mind is how are organizations managing their regression testing needs today? Knowingly or unknowingly, most of the organizations are meeting this demand by adopting risk-based testing, without proper risk assessment. This essentially means regression testing is executed on the basis of time and resource availability. This approach significantly increases the probability of missing several regression defects, thus increasing the production support cost considerably.  The lack of an effective regression testing strategy can be attributed to three main points:

The role of risk-based testing

A risk-based testing approach addresses the size and complexity parameters of a regression testing suite in a methodical manner. Theoretically, we all know that there are infinite number of ways in which a product can fail. Practically speaking, this is the same with applications, too. However, most business users and testers tend to test an application for every practical use. In doing so, they end up in challenges as described in the previous section-- increasing the size and complexity of regression testing required. Risk-based testing can help address the size and complexity issues by:

In spite of these positives, risk-based testing is still used or deployed in a very limited manner across organizations. The top two reasons for this are:

Making risk-based testing (RBT) a success

Most test managers have heard and known about the concept of RBT for a while now. However, its usage continues to be limited despite the fact that it can help us to measure the quality risk more quantitatively and also reduce the testing effort required significantly for regression testing. Below are a few suggestions that can help you in successfully implementing RBT in your respective organizations.

Conclusion

The importance of end-to-end regression testing is taking center stage as business and technology complexity is increasing and application sizes are growing. The effectiveness of regression testing will either experience the “law of diminishing returns” as the regression test case effectiveness will decrease, or the business value of testing will be continuously questioned due to failures in identification of regression defects. Risk-based testing is the answer to address both the above mentioned situations. The increasing distributed nature of an IT topology will swallow the old school of thought around full blown regression testing and risk-based testing (RBT) will soon take center stage as the main stream regression testing strategy across organizations.

19 Jul 2011

All Rights Reserved, Copyright 2006 - 2024, TechTarget | Read our Privacy Statement