Serious misunderstandings and miscommunications can come from inadvertently using different meanings for the same seemingly simple term, such as the fundamental element of testing--"test case." A tool is provided to reveal the extent of such discrepancies and reach common consensus in your own organization.
What is a test case?
Students in my testing seminars frequently express frustration that different instructors use the same terms to mean different things and vice versa. The problem actually is more complicated, as I discovered while reading a book by a prominent testing authority colleague.
I created a survey as a less confrontational way to address the test case definition question. Without explaining the reason for my questions, I sent the survey to about a dozen fellow testing authors and instructors, including the author who had prompted my question. The survey asked for each recipient's definition of "test case" and several other test terms.
I changed the survey as shown below to include a concrete example.
The survey example above shows five different ways to interpret how many test cases there are in testing Enter an Order for a Customer. The survey asked recipients to examine the example and determine which one, and only one, of the five indicated possible interpretations of a test case fits what they think of as a test case. That is, they could pick Level A, where there's a white arrow with an "A" in it, which means they feel the entire example is one big test case for Enter an Order for a Customer. At Level B, where each place there's a blue arrow with a "B" in it represents a separate test case, the example consists of seven test cases. At Level C, where each place there's a pink arrow with a "C" in it represents a separate test case, the example consists of 14 test cases. At Level D, where each place there's a yellow arrow with a "D" in it represents a separate test case, the example consists of 20 test cases. At Level E, where each place there's a teale arrow with an "E" in it represents a separate test case, the example consists of 24 test cases. I asked the survey participants to pick one level: A=1, B=7, C=14, D=20, or E=24 test cases in the example. I use the same exercise in several of my testing seminars; and people also can do the exercise on my website. I encourage you to pause at this point in this article before reading how others have answered, figure out your single (A, B, C, D, or E) answer, and send it to me on my website or at .
Only a few of the authors/instructors provided the requested one-letter answers. Most who previously had given lengthy verbal definitions declined to give a one-letter answer to this concrete example. Several, including the book author, nonetheless waxed on about various philosophical testing topics unrelated to the example, such as how long should be required to run a test case.
Broader survey results
I acknowledge I haven't been able to total actual A-B-C-D-E counts from the hundreds of testers who have taken the survey example. Most response counts get buried in the bustle of classes. I can say with certainty that each choice has received many votes and none reliably dominates. Each class exhibits variability among the choices, and patterns of responses differ from class to class.
What is a requirement?
The need for such test case interpretation choices often occurs because of the level of detail in which requirements are defined. Testers generally create test cases based on however the requirements are defined. The problem is that requirements very often are defined at only the grossest Level A.
Robin F. Goldsmith, JD is President of Needham, Mass., consultancy Go Pro Management, Inc. He works directly with and trains business and systems professionals in requirements, quality and testing, metrics, ROI and project and process management. A subject expert and reviewer for IIBA's BABOK, he is author of the Proactive Testing and REAL ROI methodologies and Discovering REAL Business Requirements for Software Project Success.