How to conduct performance, stress, load testing without tools

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Software Testing and Quality Assurance What are the top free and not-free automated test tools? Test case preparation for a Web-based application The difference between functional testing and regression testing Free load/performance testing tools for Java-based Web applications Mapping results in test environment to production environment What are some debugging shortcuts to reduce frequent debugging? How to get a software testing job in a recession How to write a test strategy document What are the risks in using open source test automation tools? What is system integration testing? Software performance, load and stress testing Budget-friendly Web app performance testing, monitoring tips Testing functionality, performance of mobile Web applications Free load/performance testing tools for Java-based Web applications Software testing deliverables: Developing a software testing strategy Is functional testing sufficient to determine code coverage? Why the quality assurance department should be involved in testing What are the different software testing methodologies? Testers: Time to gear up for mobile software testing Two-minute guide to determining software testing coverage Best load and stress testing tools

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary integration testing  (SearchSoftwareQuality.com) performance testing  (SearchSoftwareQuality.com) shotgun debugging  (SearchSoftwareQuality.com) stress testing  (SearchSoftwareQuality.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary

First, I want to express my sympathy to anyone who finds himself in a position of being asked to create multi-user simulations (i.e. the load part of performance/load/stress testing), requesting a load-generation tool, and being denied. In my experience, the excuse of not being able to afford a load-generation tool is almost always just that -- an excuse.

Performance testing resources: Top 10 performance testing tips Testing for performance, part 1: Assess the problem space Testing for performance, part 2: Build out the test assets Testing for performance, part 3: Provide information

Only once in my career have I found the only tool capable of generating a production-like load against the application I was testing to be prohibitively expensive. (In that case, the tool cost more to purchase than the application was anticipated to earn in a year.) In every other case, either the risk justified the cost or an adequate, inexpensive, or free tool has been available.

The only really good reason to be in this position is if all available tools were considered and none of them supported the application under test (even with customizations and extensions) and you don't have access to the skills (internally or externally) to build a tool of your own -– and that seems unlikely to me.

The simple truth is that if a company is building an application that is realistically expected to have enough users to justify the expense of performance testing, even if that expense is just the time of an employee, that company ought to be projecting enough revenue from the application, or lose enough credibility by having a poorly performing application, to justify either the cost of a tool or the risk (in some company's eyes) of using a free or open source tool.

Now, after saying all of that, I must admit I have found that the vast majority of value that is gained by quality performance testing comes outside of the load-generation tool. Some of my favorite techniques (assuming you are testing websites):

1. If you are testing a website, odds are that you slice response times in half (sometimes more) by performance testing the front end. For more on how to do that, see the article "Right Click -> View Source and other Tips for Performance Testing the Front End" in the December 2007 issue of AST Update magazine (PDF).
   
   
2. Use browser plug-ins or online tools to capture page load times. Several such tools are referenced in the article above.
   
   
3. Ask functional testers and/or user acceptance testers to record their opinion about performance while doing their testing. It may be useful to give them a scale to use, such as "fast, acceptable, tolerable, annoying, unusable."
   
   
4. Have the developers put timers in their unit tests. These won't tell you anything about the user-perceived response times, but developers will be able to see if their objects/modules/classes/functions/etc. take more or less time to execute from build to build. The same idea can be applied to various resource utilization (like memory and CPU utilization), depending on the skills and/or tools available to the development team.
   
   
5. Employ what I frequently refer to as the "hire a bunch of interns method." Basically, get increasing numbers of your co-workers to use the application during a specified period of time and ask them to note both the response time (which is easiest to do using the aforementioned browser plug-ins) and their opinion about the application's performance. (Give them the same scale used for the functional and/or user acceptance testers.)
   
   
6. Have special performance builds made with timestamps strategically output to log files. Analyze the log files build after build and track the trends.