Q

Expert advice on test case management and reporting

Simplify your test case management reports with this simple, straightforward advice from software test expert Amy Reichert.

Help! My manager wants me to track the number of test cases versus the number executed. What should I do?

Amy ReichertAmy Reichert

A manager's job is to track metrics, measure execution and create all sorts of charts and graphs to show progress. If they want you to track the number of test cases you have in your test suite against the number you've executed -- do just that. If you're using a test management tool, there are canned reports that can give you this information with a little setup involved. You can also create a custom report and really impress them.

If you don't have a test management tool, simply use Excel or another spreadsheet application and input all the test cases in your suite, one per row. I'm going to assume you either have periods of time where you execute a regression testing suite or you're executing regression tests continuously. Either way, it's the same general process.

First, write down all the test cases you have. Next, sort them by validity. Some of the test cases will either be out of date or will need to be updated for other reasons. Those tests are no longer valid. Put those invalid tests in a group separate from the test cases that are ready to execute against.

If they want you to track the number of test cases you have in your test suite against the number you've executed -- do just that.

Now you have a count on the total number of tests you have, and a count on how many of those are invalid because they need to be updated. Estimate how many new tests your test team is writing. This number is going to change, but as long as you estimate reasonably it'll be accurate. We now have the following information:

  • Total number of test cases
  • Number of test cases that need to be updated
  • Number of new test cases being developed

Next, you'll want to define the date up to which you are tracking data. Pick a date a few days before your manager wants the report delivered and use that as your defined date. This gives you time to compile the data your manager wants up to a recent date without having to do real-time calculations.

I'm hoping you track your execution either in a tool or manually in a spreadsheet. If you do either of those, then you can get a count on how many of the tests have been executed quickly by manipulating canned reporting functions in the tool. If you're not tracking execution, then you need to talk to each tester and get an estimate of which tests cases they've executed up to your defined date.

Now we have:

  • Total number of test cases
  • Number of test cases that need to be updated
  • Number of new test cases being developed
  • Total tests executed up to a defined date

These four numbers should give your manager an accurate view of the work you and your team have been doing.

If you're not creating test cases, but are testing in an ad hoc or exploratory manner, you can still generate counts. Your counts will just be estimates. You'll need to interview each tester and note what functions they're testing and ask them to estimate the number of tests they execute daily. Add them up to give counts similar to these:

  • Total number of test cases
  • Number of tests executed (per tester per day)
  • Total tests executed up to a defined date

Your best estimates for these three numbers should give your manager a workable estimate of the work you and your team have been doing.

Do you have a question for Amy Reichert or any of our other experts? Let us know and we'll post your answer in a future response. Email the Editor@SearchSoftwareQuality.com or leave a question in the comments.

This was first published in April 2014

Dig deeper on Software Regression Testing

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

Have a question for an expert?

Please add a title for your question

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.
Related Discussions

Amy Reichert asks:

Do you think test execution reports are a valid way to represent a team's testing efforts?

4  Responses So Far

Join the Discussion

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchSOA

TheServerSide

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

Close