Answer

Expert advice on test case management and reporting

Help! My manager wants me to track the number of test cases versus the number executed. What should I do?

Requires Free Membership to View

Amy Reichert

A manager's job is to track metrics, measure execution and create all sorts of charts and graphs to show progress. If they want you to track the number of test cases you have in your test suite against the number you've executed -- do just that. If you're using a test management tool, there are canned reports that can give you this information with a little setup involved. You can also create a custom report and really impress them.

If you don't have a test management tool, simply use Excel or another spreadsheet application and input all the test cases in your suite, one per row. I'm going to assume you either have periods of time where you execute a regression testing suite or you're executing regression tests continuously. Either way, it's the same general process.

First, write down all the test cases you have. Next, sort them by validity. Some of the test cases will either be out of date or will need to be updated for other reasons. Those tests are no longer valid. Put those invalid tests in a group separate from the test cases that are ready to execute against.

If they want you to track the number of test cases you have in your test suite against the number you've executed -- do just that.

Now you have a count on the total number of tests you have, and a count on how many of those are invalid because they need to be updated. Estimate how many new tests your test team is writing. This number is going to change, but as long as you estimate reasonably it'll be accurate. We now have the following information:

  • Total number of test cases
  • Number of test cases that need to be updated
  • Number of new test cases being developed

Next, you'll want to define the date up to which you are tracking data. Pick a date a few days before your manager wants the report delivered and use that as your defined date. This gives you time to compile the data your manager wants up to a recent date without having to do real-time calculations.

I'm hoping you track your execution either in a tool or manually in a spreadsheet. If you do either of those, then you can get a count on how many of the tests have been executed quickly by manipulating canned reporting functions in the tool. If you're not tracking execution, then you need to talk to each tester and get an estimate of which tests cases they've executed up to your defined date.

Now we have:

  • Total number of test cases
  • Number of test cases that need to be updated
  • Number of new test cases being developed
  • Total tests executed up to a defined date

These four numbers should give your manager an accurate view of the work you and your team have been doing.

If you're not creating test cases, but are testing in an ad hoc or exploratory manner, you can still generate counts. Your counts will just be estimates. You'll need to interview each tester and note what functions they're testing and ask them to estimate the number of tests they execute daily. Add them up to give counts similar to these:

  • Total number of test cases
  • Number of tests executed (per tester per day)
  • Total tests executed up to a defined date

Your best estimates for these three numbers should give your manager a workable estimate of the work you and your team have been doing.

Do you have a question for Amy Reichert or any of our other experts? Let us know and we'll post your answer in a future response. Email the Editor@SearchSoftwareQuality.com or leave a question in the comments.

This was first published in April 2014

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Expert Discussion

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest