Does Agile exploratory testing make sense? The fast answer is yes, but why? Why should Agile teams spend time testing more than just the latest changes? Why not completely rely on your back-end automated tests to uncover all the defects?
Exploratory testing is a manual testing technique that relies on both the experience and knowledge base of your testing team members. It fits within an Agile development methodology by providing improved test coverage in a rapid, flexible and productive manner. Exploratory testing doesn't require special training, just a willingness to creatively break software. We all have that instinct -- even if it's buried deep inside.
So, your Agile team members are now creatively finding defects, using their skills and stretching their brains. Is it helping?
In this article, I explain how to use Agile exploratory testing to improve test coverage by increasing defect discovery and regression testing effectiveness while retaining Agile productivity and flexibility.
What is exploratory testing?
Exploratory testing is similar to ad hoc testing, where the tester's experience steers the tester to areas where defects are suspected. Testers hammer away at functional areas using their application experience and creativity. Ad hoc is a useful testing practice, but over time, the term has become associated with sloppy, hasty testing.
Cem Kaner coined the term exploratory testing in the book Testing Computer Software, co-authored with Jack Falk and Hung Quoc Nguyen. According to Kaner, exploratory testing seeks to discover how the application actually works. It is dependent on your testers' experience with the application, their ability to create effective test cases and ferret out meaningful defects -- i.e., defects that get approved to be fixed, not those that languish in a defect-tracking system until a customer reports them.
With exploratory testing, your application becomes an open book without expectations or anticipated results. Test team members explore negative workflow paths as well as expected workflow paths and are limited only by their creativity and knowledge. After all, if your test team only executes the same regression suites over and over again, those suites tend to find fewer defects. Yes, you're proving that those scenarios work, but, honestly, for a tester, repetitive execution of the same tests is mind numbingly dull, not to mention ineffective.
Exploratory testing allows individual testers to expand their knowledge while exploring multiple workflow paths -- good, bad and bizarre. For a tester, it involves more creative energy and is intellectually stimulating. It is also useful when requirements are vague and the testing time is short. Sounds like a sure fit for an Agile team, where testing occurs continually and quickly. Exploratory testing is quick, flexible, creative and productive. What more can you ask for?
Discovering more defects
Exploratory testing finds meaningful defects that are highly system-critical or workflow-critical. These are defects your testing team needs to find. When testing is automated, it's not as likely to detect defects in complicated test scenarios because they're likely hidden in workflows that are not easily scripted. Using exploratory testing techniques, developers, testers, analysts -- nearly anyone on the Agile team -- explore all the ways customers can use the application.
Exploring the application helps your team better understand the application as the customer sees it, as well as figure out how to manipulate the infrastructure. Are there defects in the configuration? What options generate defects and in what functional areas? Are they critical areas or common workflows? Your team will find defects that they've never noticed before.
Increasing regression testing effectiveness
So, your Agile team members are now creatively finding defects, using their skills and stretching their brains. Is it helping? Is your test team more actively engaged? Are they finding more defects? With exploratory testing, your team covers more ground. Yes, you should also execute your scripted automated tests if you have them, but manual testing increases your regression testing effectiveness. Exploratory testing is more effective for regression when time is restricted or your existing regression suite fails to detect defects.
It also allows your test team and the rest of the Agile team to participate. Together they can flush out more complicated test scenarios involving security, integration points and runtime engine or white-box testing that is normally time consuming to cover and, thus, often gets skipped. Call it what you want, but manually banging your way through an application is effective regression. Automated scripts have value but, in my opinion, aren't as effective at finding workflow defects.
Manual regression testing remains the most effective method of regression testing, assuming your team members' minds are engaged in creatively disassembling the application. Exploratory testing gives you more effective regression testing coverage in an Agile team environment.