vali_111 - Fotolia
Smoke test or regression test -- they are similar and yet different. A fine line separates what defines a smoke test, or at least a smoke test worth executing, and what defines a valuable regression test. So, as a quality assurance, how do you narrow the tests you need to execute and create a valuable smoke test suite? How do you determine the amount of coverage you need and where?
It helps if the team has performed risk analysis on the functional areas of the application so that information is up to date and available. If not, consider creating a map of all the functional areas within your application and rank them from highest to lowest priority. If you have trouble ranking them directly, try grouping them into critical, high, medium and low. Once you separate them in this manner, rank them within their group. Take the critical group and pull test cases that test some function in each area. Repeat for the high group. How many tests do you have? For a smoke test suite that's primarily executed manually you'll want to stay around 30 to 35 total test cases. The goal is to make it reasonable for a tester to execute in four hours, or half a work day. If your smoke test is automated, you should be able to increase the total number of tests to 50. I suggest keeping it small enough to allow time for failure analysis. The advantage of automated smoke tests is the ability to execute the suite more frequently while still allowing time for failure analysis.
The next consideration is taking the test cases you have ranked so far, and reading through them to gauge complexity. The smoke test suite needs to be relatively superficial but also contain enough depth or bite to actually make running it worthwhile. I am not a fan of smoke tests that only verify whether an application loads or opens. What I really need to know is if basic functions are working properly.
When reviewing your smoke test suite, look for tests on basic functions all the way through but that are not so complex they take an hour or more to execute. Remember, it's a smoke test; it's not going to touch on everything in your application. But it should at least touch on all the functionality within an application. Scope and depth is a tricky business -- use your knowledge of the application to determine how deep the test cases need to go. Keep in mind the smoke test suite is not a fully functional regression. Good luck!
Smoke testing's role in continuous delivery
Why you should automate testing, but carefully
Look into software testing's crystal ball
Dig Deeper on Topics Archive
Related Q&A from Amy Reichert
Let's explore the importance of result analysis, the right measurements and test design for application performance testing. Continue Reading
QA needs to reiterate its value to the business side of the organization. Use this tried-and-true advice to leverage documentation and automation to ... Continue Reading
Vendors have inched toward automated application testing for a long time, yet there is still room for growth. Software tester Amy Reichert offers her... Continue Reading