I'm sometimes criticized for taking a more heuristic approach to performance testing as opposed to a more mathematical one. My response has always been that I'm biased toward a commercially driven environment where even heuristic approaches frequently take more time, require more information and need specialized expertise the teams don't have. Adding complex mathematic equations that require more time, fewer unknowns and (in many cases) exceed the mathematic training of all of the performance testers in the team simply isn't likely to make matters better.
Even so, I think I have finally come up with a significantly complex mathematic formula for my critics:
[TABLE][TABLE]For those of you who have forgotten (or never had a need to learn) integral calculus, the formula roughly says this:
The maximum value for the formula within the function is 100; the minimum is 0. Based on my experience, a good score would be 50, and a typical score would be 10. "+/- X" acknowledges that whatever the function predicts could be wrong.
Brilliant, huh?!?
OK, I admit it. I'd never use this formula. In fact, the entire notion of a magic formula is intended as a parody. I came up with this while reviewing a paper from an IEEE conference. The paper is now several years old and was related to the performance of software systems. I'm not going to name the specific paper because it's not important, and I don't want anyone to think I'm trying to
To continue reading for free, register below or login
To read more you must become a member of SearchSoftwareQuality.com
attack it. I'm not.
Formulas that look (and to me feel) like the one above are all over books and papers that claim to be written for performance testing practitioners. While I know that there are some performance testing practitioners who work in environments where formulas that look like this are not only valuable but also essential to their job, my experience suggests that these are the minority.
That being said, the formula is neither shallow nor is it entirely without merit. If we look closely, it calls out many of the reasons that performance test projects aren't as successful as we would like them to be. Consider the following points that the formula makes:
Feel free to use my formula if it helps you communicate the relative importance of some aspect of performance testing to your team. If you use it, remember that every input value (with the possible exception of time and the number people on the team) is an estimate that you created, making the formula no more accurate than your input data. I suspect that you'll get more value out of using the formula to demonstrate that even the most complicated looking formulas won't relieve you of the need to think.
----------------------------------------
About the author: Scott Barber is the chief technologist of PerfTestPlus, vice president of operations and executive director of the Association for Software Testing and co-founder of the Workshop on Performance and Reliability.
