Last week O’Reilly’s 97 Things Every Software Architect Should Know: Collective Wisdom from the Experts was released. In a recent excerpt published by Sara Peyton, Thoughtworks CTO Rebecca Parsons provides some insight into performance testing.
In early testing, you may not even try to diagnose performance, but you do have a baseline of performance figures to work from. This trend data provides vital information in diagnosing the source of performance issues and resolving them.
I like this quote because it looks at an often overlooked aspect of performance testing. Many times when people think about performance testing, they only think they can test if they have requirements. While those are helpful, they aren’t the only tools you have available. Sometimes trending is enough to recognize you might have found issues. If something was working with a response time of 5 seconds, and degrades to 10 seconds, you don’t need a requirement. It’s worse. When running your early tests, that’s often good enough to get going.
A bit later in the excerpt, another small gem is highlighted:
Technical testing is also notoriously difficult to get going. Setting up appropriate environments, generating the proper data sets, and defining the necessary test cases all take a lot of time. By addressing performance testing early, you can establish your test environment incrementally, thereby avoiding much more expensive efforts once you discover performance issues.
It’s incredibly difficult to build out even seemingly simple performance test environments. It’s not even that physically setting up the environment is always hard (however many times it can be), it’s often figuring out what’s different from the production environment. You know, those little problems you discover after everyone has said it’s “the same” but it is really different due to minor changes in configuration, version, or dependencies. Ironing out those kinks, or identifying the key differences can take time.
If the rest of the book is like the excerpt, it might be worth picking up. Some other topics covered in the book (as noted by the publisher) include:
- Don’t Put Your Resume Ahead of the Requirements (Nitin Borwankar)
- Chances Are, Your Biggest Problem Isn’t Technical (Mark Ramm)
- Communication Is King; Clarity and Leadership, Its Humble Servants (Mark Richards)
- Simplicity Before Generality, Use Before Reuse (Kevlin Henney)
- For the End User, the Interface Is the System (Vinayak Hegde)