SearchSoftwareQuality.com

Scripted testing vs. exploratory testing

By Vasudeva Naidu

I recollect using exploratory test techniques to uncover hidden defects for a software product organization, in my role as a tester 12 years ago, despite lacking formal training or awareness. It was my urge to uncover hidden defects which drove me to adopt exploratory test methods. Today, even as software testing gurus continue to debate the merits and demerits of exploratory testing, its awareness and practice, with the exception of some software product development organizations, continues to shrink rapidly. When I started thinking about the possible reasons why the majority of the testers today fail to adopt exploratory testing, I realized that the gap really lies in the design of the overall test strategy. Most of the test strategies today heavily focus on scripted test methods with limited or no scope for the usage of exploratory test methods . There is no doubt that exploratory testing does have certain advantages, but in order for us to leverage the same, we need to revisit our application testing strategies and ensure a fine balance between scripted and exploratory test methods and tools. This balance would ensure an increase in the effectiveness of the testing function without compromising testing efficiencies.

What is the Current Role of Scripted Testing Vs. Exploratory Testing?

In the last 15 years, as testing methodologies have undergone several changes, scripted testing has taken center stage. QA teams currently focus on ensuring the application under development complies with the stated requirement specifications, rather than trying to evaluate the behavior of the application under development through:

Some of the possible explanations/situations that led to the industry-wide adoption of scripted testing:

Why we need exploratory testing and its significance in today’s world

While IT organizations across industry verticals continue to improve the quality of their product, according to my experience, the defect removal efficiency of the overall IT organizations has not improved beyond 85%. However, a majority of the testing groups within the IT organizations have transformed themselves and become more efficient, reporting defect removal efficiencies of 95% to 98%. Most IT process gurus are analyzing this situation and trying to devise strategies to address this big quality gap that exists between what testing groups are claiming and what the overall IT application quality really is. It is very clear that quality is just not a testing problem, and the entire IT organization needs quality focus to improve the defect removal efficiency of the entire IT organization from the current levels. One such strategy that can be adopted to ensure the same is exploratory testing. Of course it would need to be carefully married to the organization’s master test strategy in order for it to yield the planned improvements in quality.

Implementing a combined strategy

Even though most testing professionals focus on scripted test execution, the natural tester’s instinct pushes her/him to explore the system completely in the available time. Structured methods and initiatives can improve the efficiency of exploratory testing when carefully combined with mainstream scripted execution. While attempting to devise a successful strategy one must keep in mind certain ground realities:

However, despite these challenges, testers are encouraged to think beyond their test scripts thanks to the focus of IT organizations on quality of applications. The table below outlines a few initiatives which I believe will help in successfully adopting exploratory testing to augment scripted testing:

Conclusion

While I agree with industry gurus on the importance and role of exploratory testing,  its role as the primary test strategy, as argued by many, is questionable due to its inability to support large volumes of testing and control the behavior of large test teams. At the end of the day, economics dictate what is feasible and what isn’t, and industry indicators definitely show a move towards the commoditization of the testing function. The concept of introducing quality into the product and pushing for early detection of defects is forcing QA teams to develop test strategies which adopt exploratory testing and integrate it with scripted testing seamlessly. It is also shifting the focus towards validation at the level of business transactions, grooming of SMEs and building business stewards (owners of a business process or end-to-end business transaction) who will have ability to adopt exploratory testing methods effectively. Training and awareness is also a key contributor for the success of exploratory testing methods in this ever-growing tester’s community. QA managers must implement exploratory testing techniques as an additional strategy, combined with scripted testing, to improve the overall quality of  applications.

29 Mar 2011

All Rights Reserved, Copyright 2006 - 2019, TechTarget | Read our Privacy Statement