Q
Manage Learn to apply best practices and optimize your operations.

How do you narrow the options in a complex smoke test suite?

Trying to determine how much ground a smoke test should cover is tricky. Expert Amy Reichert offers step-by-step advice for getting the most out of this technique.

Smoke test or regression test -- they are similar and yet different. A fine line separates what defines a smoke...

test, or at least a smoke test worth executing, and what defines a valuable regression test. So, as a quality assurance, how do you narrow the tests you need to execute and create a valuable smoke test suite? How do you determine the amount of coverage you need and where?

It helps if the team has performed risk analysis on the functional areas of the application so that information is up to date and available. If not, consider creating a map of all the functional areas within your application and rank them from highest to lowest priority. If you have trouble ranking them directly, try grouping them into critical, high, medium and low. Once you separate them in this manner, rank them within their group. Take the critical group and pull test cases that test some function in each area. Repeat for the high group. How many tests do you have? For a smoke test suite that's primarily executed manually you'll want to stay around 30 to 35 total test cases. The goal is to make it reasonable for a tester to execute in four hours, or half a work day. If your smoke test is automated, you should be able to increase the total number of tests to 50. I suggest keeping it small enough to allow time for failure analysis. The advantage of automated smoke tests is the ability to execute the suite more frequently while still allowing time for failure analysis.

The next consideration is taking the test cases you have ranked so far, and reading through them to gauge complexity. The smoke test suite needs to be relatively superficial but also contain enough depth or bite to actually make running it worthwhile. I am not a fan of smoke tests that only verify whether an application loads or opens. What I really need to know is if basic functions are working properly.

When reviewing your smoke test suite, look for tests on basic functions all the way through but that are not so complex they take an hour or more to execute. Remember, it's a smoke test; it's not going to touch on everything in your application. But it should at least touch on all the functionality within an application. Scope and depth is a tricky business -- use your knowledge of the application to determine how deep the test cases need to go. Keep in mind the smoke test suite is not a fully functional regression. Good luck!

Next Steps

Smoke testing's role in continuous delivery

Why you should automate testing, but carefully

Look into software testing's crystal ball

This was last published in November 2015

Dig Deeper on Software Testing Methodologies

PRO+

Content

Find more PRO+ content and other member only offers, here.

Have a question for an expert?

Please add a title for your question

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.

Join the conversation

3 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

What trips you up when using a smoke-testing suite?
Cancel
I think if you are asking this question, you need to step back and ask yourself, what is most important in this process?  What is the purpose of any given Smoke Test, what value does it provide, and how do we make sure we aren't over checking, or running unnecessary checks.

These are healthy questions to ask, especially if automation is used for a healthy chunk of it.
Cancel
As Tim said - to avoid tripping is important to turn on critical thinking. WHY perform this or that test?

One thing for sure missed in the article: if the deployment contains fresh bug fixes the smoke testing should tackle those areas.
Cancel

-ADS BY GOOGLE

SearchMicroservices

TheServerSide.com

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

DevOpsAgenda

Close