To continue reading for free, register below or login
To read more you must become a member of SearchSoftwareQuality.com
This is an interesting series of questions. I count at least 18 questions embedded in these three lines with five question marks. And it looks very much like the questions as a whole were inspired or at least shaped by a commonly believed myth about performance testing.
Let's start with the questions.
1a. What are performance, load and stress testing?
This is at least 3 questions. Performance testing, load testing and stress testing are three different things done for different purposes. Certainly, in many cases they can be done by the same people with the same tools at virtually the same time as one another, but that does not make them synonymous.
Performance testing is an empirical technical investigation conducted to provide stakeholders with information about the quality of the product or service under test with regard to speed, scalability and/or stability characteristics. It is also the superset of other classes of performance-related testing such as load and stress testing.
A load test is a performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes anticipated during production operations.
A stress test is a performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes beyond those anticipated during production operations. Stress tests may also include tests focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes while the product is subjected to other stressful conditions, such as limited memory, insufficient disk space or server failure.
1b. Could you please explain it with examples?
Ok, for simplicity, let's take one example. Imagine your favorite Web site that sells things. Anyone that accepts credit cards and has a shopping cart paradigm will do.
Performance testing would involve things like determining the end user response time under various amounts of traffic and scenarios. For example, you may be interested in finding out what response times a user would encounter during an hour where there were 2000 searches for items, 500 items added to carts and 50 purchases were made. Or you may be interested in finding out what response times a user would encounter during a different hour where there were only 500 searches for items, 400 items were added to carts and 250 purchases were made.
Load testing the same site might involve determining how many searches can be conducted in a given hour before the response time slows to 5 seconds. Stress testing could involve determining how many searches can be conducted in a given hour before the users stop receiving results.
2. What are the prerequisites for performance, load and stress testing?
The only think I need to start conducting any of three kinds of tests is an interest in learning about the performance characteristics of an application or a concept of an application. I don't need tools, I don't need a mostly functioning application. Some of the most useful performance testing I've ever done has been with an architect, in a bar, using a bar napkin and a borrowed pencil. He was sketching and brainstorming a design for an application. I was testing his design by pointing at various objects/components and asking questions. About 3 of 5 questions resulted in him starting over on a new napkin.
|
|
 |
The point is that this is an impossible question to answer without first knowing why you want to do performance testing in the first place. Do you want to know what the response time will be in production? Do you want to determine compliance with contracts? Do you want to help the developers tune the application? Do you want to determine what hardware is necessary to support the target load? Each of these objectives requires different things to achieve.
3. Do we write separate test plan & test cases for performance, load & stress testing? If yes, where would I get those templates?
I have no way to answer that. I can think of situations where that would be at least a reasonable idea. I can think of many more where that would be a colossal waste of time and energy. I avoid performance test plans like the plague and I don't even know what folks mean when they use the phrase "performance test case." I tend to work from an overall performance test strategy (which is usually little more than a prioritized list of tasks that the team would like to have accomplished via performance testing on a white board) and test designs. To me, a plan implies dates, sequences, resources and reports. A test case involves things like expected results, pass/fail criteria and mappings to requirements. None of those things makes any sense in performance testing. At least not to me.
As for templates, there are templates all over the Web for those things. It is possible that I could point you to one that you'd find valuable, but it would be dumb luck. I cannot imagine a template that could be generic enough for you to take all the particulars of your situation into account while still being specific enough to actually provide any value.
The underlying myth:
Experience tells me that these questions stem from the myth that "performance testing is just functional testing, only with more users." This simply is not the case. One will not meet with success while performance testing, except by accident -- which does happen quite often -- by treating it like functional testing. Not only are the skills, purpose, planning, scheduling and tools different, but the entire thought process is different. To tell the truth, you are probably more likely to succeed by simply asking the members of the team "what would you like to know about performance?" and then figuring out how to collect that data than by treating performance testing like a functional testing project on steroids.
|
