A year ago, I was working on a project where we were doing a failure modes and effects analysis (FMEA) related to failover and recovery. As I was thinking about how to best start my analysis, I recalled that in the past while doing performance testing work I looked at many of the same aspects of the system while planning. As a way to generate ideas, I did some research to identify sources that could help me with my planning. You can take a look at some of the resources I found, or use different taxonomies if you have any that you particularly favor.
Here’s an example of how you might use a resource like this. Let’s take the risks listed in chapter three of Performance Testing Guidance for Web Applications. In the following figure from the book, you’ll see a summary of the risks presented in that chapter.
Figure 1: Performance testing risks, from the book Performance Testing Guidance for Web Applications.
I prefer working with the list of questions the authors have outlined in the chapter, but the graphic does a nice job summarizing things. For each specific risk listed, you want to:
- Ask yourself if you’ve accounted for that risk with your current plan. If you haven’t, figure out if you should. If you think you should, figure out what type of testing would be most appropriate for you. One nice thing about this particular taxonomy is that they give you some guidance there.
- For each risk, move from the generic to the specific. The risk “Can the system be patched or updated without taking it down?” is a great question, and an initial answer might be “yes.” But when I look at the system I currently work with, there are several major systems all working together. I might ask if I can patch all of them. And patch them in what ways; via software, database, run-time dependencies, services, etc.?
- For each risk, ask yourself if there are any slight variations on that risk that might be important to you. Good examples of the practice are the two risks listed in the book: functionality could be compromised under heavy usage; and the application may not stay secure under heavy usage. And you can vary different parts of the same question. In those two risks, they varied the quality criteria — functionality and security — but kept the risks, such as heavy usage, static. You could add other quality criteria or other risks.
The general idea is that you’re using lists like these to help you generate test ideas. In a way, you’re also using them to test the planning work you’ve done so far to make sure you haven’t forgotten or overlooked anything.