Prioritizing software testing on little time

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Software Testing and Quality Assurance Why do performance testers write new scripts so often? How to create performance testing workload models Fixing Web application performance troubleshooting problems Expert advises on implementation of Selenium IDE for effective software testing When should regression testing occur in an automated test plan? Achieving peak performance in integration testing Getting answers about OpenSTA script problems Defining core software regression tests Breaking in functionality on UI application pages Where to find good methodology guides for software testing Advice from Scott Barber Performance testing SOA Acceptance testing for websites GUI automated testing projects How to set up a test environment Software testing processes and development methodologies Smoke and sanity testing Automating regression test cases Test plan and test strategy User acceptance testing that satisfies users and requirements Understanding performance, load and stress testing Software testing and quality assurance (QA) fundamentals How to deal with iteration issues in Agile Five steps to fostering better software tester and QA results Software Testing: New software testing technologies bring new challenges Testing strategies for complex environments Astronaut's STPCon advice: Teamwork delivers "The Right Stuff" How to make your software tamperproof Software consortium seeks standard quality metrics Demo: Using WebGoat, a free software testing tool Seven steps for a quality change and configuration management program Winning responses to "Why is QA always the bottleneck?"

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary build  (SearchSoftwareQuality.com) code review  (SearchSoftwareQuality.com) conformance testing  (SearchSoftwareQuality.com) error handling  (SearchSoftwareQuality.com) garbage in, garbage out  (SearchSoftwareQuality.com) load testing  (SearchSoftwareQuality.com) NUnit  (SearchSoftwareQuality.com) quality assurance  (SearchSoftwareQuality.com) stress testing  (SearchSoftwareQuality.com) white box  (SearchSoftwareQuality.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary

This question pops up in various forms all the time. It boils down to "We don't have enough time to test everything, so what do we test?" Not having enough time, of course, is not only the status quo for testing software, it is a universal truth for any software that will ever go into production.

Given that, here's my advice.

1. Start by forgetting that you have any test cases at all.
2. Make a list (quickly -- remember we don't have enough time to test, so let's not waste what little time we have making lists) of each of the following usage scenarios. I usually limit myself to five on the first pass, but no matter what, move on to the next category as soon as you find yourself thinking about the category you are on. If you have to stop and think, whatever you come up with isn't important enough.
1. What things will users do most often with this application?
2. What areas of this application are most likely to contain show-stopping defects?
3. What parts of this application are critical to the business?
4. Are any parts of this application governed by legal or regulatory agencies?
5. What parts of the application would be most embarrassing to the company if broken?
6. What parts of the application has my boss said must be tested?
3. Prioritize the list. If you've made the list in a word processor or using note cards, this will take under 60 seconds (if you have to write a new list by hand and you write as slowly as I do, it will probably take a little longer. Here are the rules for prioritizing.
1. Count the number of times a scenario appears in any of your categories. The more times the scenario appears, the higher the priority.
2. In case of a tie, 'a' comes before 'b' comes before 'c,' etc.
4. Now scan your test cases. Note which ones are covered and which ones aren't. On the ones that aren't covered, ask yourself, "Can I live with not testing this?" If the answer is no, add it to the bottom of the list.
5. Start testing.
6. If you complete these tests before time is up, do the same exercise again without repeating any usage scenarios. If not, at least you have a defensible list of what you did and did not test and lost all of about 15 minutes of testing time creating that list.

Software testing resources: How to define a test strategy How to make testing estimation more accurate Ten skills of highly effective software testers

In case you're wondering, this approach is derived from my FIBLOTS heuristic for deciding what usage scenarios to include when developing performance tests. FIBLOTS is an acronym representing the words that complete the sentence "Ensure your performance tests include usage scenarios that are:

• Frequent
• Intensive
• Business critical
• Legally enforceable
• Obvious
• Technically risky
• Stakeholder mandated."

I guess for functional testing, it would be "Ensure you test usage scenarios that are:

• Frequent
• Risky
• Business critical
• Legally enforceable
• Obvious
• Stakeholder mandated."

Too bad the acronym FRBLOS isn't as easy to remember as FIBLOTS.