Alan Parkinson, founder and CEO of Hindsight Software, said he got into software quality after years as a software developer and engineer. What attracted him to the idea of software QA over development was that software QA makes sure developers keep their jobs. "Without quality software, buyers go elsewhere," Parkinson said. "If they don't buy, the sellers go out of business, and we [software developers] are all out of a job." Parkinson...
recently spoke about the use of behavior-driven development (BDD) with the open source test automation tool Cucumber in front of an audience of Java developers at the monthly Boston Java Meetup Group in Cambridge, Mass.
BDD is a development method that builds on test-driven development and acceptance test-driven development. The latter development methodologies emphasize the importance of developing tests that will prove software is functioning properly before developing the software that will complete said tests. BDD takes this concept a step further by stating that acceptance tests should be driven by the expected behaviors of the finished application. Parkinson claims that this behavior-driven approach improves requirements definition and thereby reduces the chances of introducing requirements defects.
Most defects, according to Parkinson, fall into one of two main categories. On one hand are implementation defects where the code 'doesn't do what it's supposed to. On the other hand are requirements defects where the developers have written the right code for the wrong function. Parkinson said that 50% to 60% of all defects are requirements defects and are 100% to 200% more expensive to fix. With an implementation defect, developers can usually isolate the portion of code that's wrong, fix it and be done. Requirements defects, on the other hand, usually mean redesigning and potentially recoding the entire project.
Parkinson recommends using the open source tool Cucumber to help automate the acceptance tests associated with BDD. With the Cucumber BDD tool, application development teams can use the natural language format Gherkin to represent behavior-driven requirements, produce automated tests that mirror those requirements, and then quickly produce a minimal documentation.
Gherkin is a natural language, so business representatives and stakeholders can read and understand them. However, Parkinson said writing the requirement scenarios in Cucumber is still a job best suited to a developer or QA professional. "Executives can understand the requirement scenarios in Cucumber, but that doesn't mean they'll start writing them. It's not a miracle tool," Parkinson quipped. Gherkin uses a fairly popular test scenario format called Given-When-Then. An extremely simple example would be, "Given the user is logged into my Hello World program; when the user clicks Go; then the program should display 'Hello World!'"
According to Parkinson, it's important for all players on the software development team -- including the business stakeholders and quality assurance testers, as well as the software designers and coders -- to be on the same page about requirements from the very beginning all the way through to the end. This is why using a natural language such as Gherkin for writing the final acceptance test scenarios is important. It helps stakeholders, QA testers and programmers all share the same understanding of what those requirements mean.
Cucumber reads Gherkin and outputs boilerplate code that mimics the specifications dictated in Gherkin quite well, according to Parkinson. He said that whoever is responsible for automating acceptance tests can quickly generate that test code directly from the Gherkin test scenarios, with relatively little tweaking to the code necessary. He explained that in most organizations, he's seen developers producing the automated acceptance tests, but that in some organizations testers may already be coding these tests or may be asked to learn enough code to produce them.
Once the acceptance tests have been generated, Parkinson suggested that development teams should closely follow those unit tests and run them often. This will ensure that each step of the acceptance scenario is completed. Following Mike Cohn's test automation pyramid, Parkinson explained that Cucumber BDD tests focus on the base of the pyramid, which represents 80% to 90% of all tests. Most organizations still need manual exploratory testing for edge cases, look-and-feel testing and other necessities, Parkinson said.
Parkinson also said that the Gherkin specifications double as baseline software documentation that describes regression tests and explains basic application behavior. For Agile teams insistent on minimizing documentation efforts, this can serve as a minimal form of requirements documentation, design documentation and even technical documentation. However, these specs will almost certainly be too technical to fit as user documentation.
Parkinson admits that user documentation is "not brilliant" within the Cucumber project. As with many open source projects, the Cucumber BDD project documentation "is a work in progress. It is currently being written and made available to the Cucumber community," Parkinson said.