Measuring a software testing effort is a difficult thing. There is a lot of literature about the abuse of test...
By submitting your personal information, you agree that TechTarget and its partners may contact you regarding relevant content, products and special offers.
metrics, but surprisingly little about test metrics done well. This is a story from early in my career, when I was first learning how to do software testing, from a time when the testing world itself looked very different than it does today. A colleague and I devised an interesting way to measure our work, and we learned something important. That lesson still informs my work today.
Exploratory testing and test planning
Exploratory testing (ET) is a style of software testing that I enjoy very much, and I have become quite good at it. But to someone looking over my shoulder while I work, what I do may seem disorganized or chaotic. What is happening is that I am creating a mental model of the system, and in order to do that, I often jump around in the application in ways that could very well confuse an outside observer. I know from long experience that it is an effective way for me to work, but not everyone enjoys working like this.
What to measure?
We knew in advance that we were both going to be working on a specific project with very clear requirements to be completed in a particular amount of time. We would get code to be tested dropped in the test environment on a specific day, and the project was required to be complete a significant time later. We agreed that we each would work only in our own style. I would do ET while she carefully and conscientiously executed the test plan. At the end of the project, we would count the number of issues/defects reported, and we would also measure the rate at which each of us reported these issues.
The results come in
Early in the project my issue-reporting rate was significantly higher than hers. I had this wild upward spike of bug reports, finding many issues very quickly, while her issue-reporting rate gradually increased to a certain level and then remained stable.
The results of the test metrics exercise allowed us to plan our work much more effectively. I was usually in charge of testing on shorter projects with critical release dates, because I was better at finding more bugs more quickly. For longer more complex projects that required more planning and foresight, she was usually in charge.