Professional testers in the agile (adaptive) space are being challenged to keep pace with accelerated development cycles and ever growing volumes of business functionality. This really breaks into two distinct but related test design challenges: testing new/altered functionality (sprint), and testing the existing body of functionality (regression). How can you stage for success in this ever growing and changing landscape? Further, how can you continue to harvest the returns from your test design efforts as the application space continues to grow?
In this article, we will describe the two most common test design challenges presented by the agile (adaptive) paradigm, and we will recommend a testing approach/process that will help address these challenges. Finally, we will discuss how to expedite functional test automation within the recommended testing approach/process.
Agile (adaptive) test design challenge
The agile (adaptive) space presents two distinct but related test design challenges – testing the content of the current release (sprint) and ensuring the content of the current release does not have a negative impact on existing functionality (regression). Much of the focus in the testing space has been on addressing the challenge of testing the current release (sprint), but if you look at it from a "functional volume" perspective the release will almost always be a minor percentage of the overall "functional volume" – especially in a mature application space.
As we said, the existing "functional volume" will almost always be a much larger but in the rare case that it is not, it will be eventually, as the application grows to meet the business need. So, what test design approach will meet both the challenge of testing the immediate sprint and the growing volume of existing functionality?If you think of an agile sprint as a "wave" and the existing functionality as a "body of water" behind that "wave" then exploratory testing addresses the wave of change...
David W. Johnson "DJ"
Senior Test ArchitectTEK Systems
Agile (adaptive) test design process
The tester is being presented with two separate challenges – any approach needs to take into account two distinct testing targets – current changes versus existing functionality. I have found that a test design process that leverages two powerful approaches works best – exploratory testing and "keyword" based test design. Here are brief definitions for both:
Exploratory testing is structured ad-hoc unscripted testing of an application space where the tester simultaneously learns the application space, evolves test cases, performs test execution, and publishes defects.
The keyword test design is founded on the premise that the discrete functional business events that make up any application can be described using a short text description (keyword) and associated parameter value pairs (arguments). By designing keywords to describe discrete functional business events the testers begin to build up a common library of keywords that can be used to create keyword test cases. This is really a process of creating a language (keywords) to describe a sequence of events within the application (test case). These test cases are then executed and any anomalies are published as defects.
I recommend a test design process that leverages both these approaches. In terms of the changes introduced by the current sprint, I recommend using an exploratory test design approach – basically creating a "named" inventory of test cases to be executed. These test case designs would have little if any content – test cases as a check list. As the functionality introduced in previous sprints matures these test cases gain additional steps (keywords) until they become fully functional test cases – this evolution would usually occur over two or three sprints. In terms of existing functionality that needs to be regression tested I would recommend using a keyword driven test design approach that supports both manual and automated functional testing. The keyword concept is not a silver bullet, but it does present a design medium that leads to both effective test case design and ease of automation.
If you think of an agile sprint as a "wave" and the existing functionality as a "body of water" behind that "wave," then exploratory testing addresses the wave of change while keyword based test design addresses the much larger body of water behind that wave.
Agile (adaptive) test automation
The test design challenge presented by agile becomes designing, building, executing, and maintaining functional tests within the context of a two to four week sprints. The solution is a combination of technique and timing. For the immediate changes in the current sprint, use exploratory testers and an itemized list of test cases with little (if any) content – basically a high-level checklist. Once the software for a sprint has migrated to and existed in production for at least one sprint, a traditional set of regression test cases can be constructed using keywords. This separates the challenge into sprint related testing and regression testing.