Performance testing in the production environment

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Software Testing and Quality Assurance Why do performance testers write new scripts so often? How to create performance testing workload models Fixing Web application performance troubleshooting problems Expert advises on implementation of Selenium IDE for effective software testing When should regression testing occur in an automated test plan? Achieving peak performance in integration testing Getting answers about OpenSTA script problems Defining core software regression tests Breaking in functionality on UI application pages Where to find good methodology guides for software testing Software performance, load and stress testing Software Testing: New software testing technologies bring new challenges Drilling deep into performance testing at STPCon STPCon: Do reality checks on performance test products, panelists advise Ways to approach application performance testing on a tight budget Data warehouse/BI performance testing tool recommendations Software testers facing six big challenges today, StarWest keynoter says Is manually testing a software project for flaws too risky? At the movies: Exploratory, performance, security testing a kiosk Why do performance testers write new scripts so often? The case for software tester, analyst partnerships Advice from Karen N. Johnson How testers can convince developers of software errors Should you retest closed defects during regression testing? Eliminating testing environment troubles Do software testers need developer experience? Software testing best practices vary by context Functional testing: Unit testing, integration testing and beyond Building automated tests for legacy applications The benefits of user acceptance testing The scope of user acceptance testing Regression testing: How to select test cases

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary integration testing  (SearchSoftwareQuality.com) performance testing  (SearchSoftwareQuality.com) shotgun debugging  (SearchSoftwareQuality.com) stress testing  (SearchSoftwareQuality.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary

It can be both inaccurate and dangerous to compare performance results obtained in the test environment to the production environment. The two most likely differences in the environments are system architecture and volume of data. Other differences might include: class of machines (app and Web servers), load balancers, report servers, and network configurations. This is why making a comparison can be inaccurate. It can be dangerous if production planning is made from performance transaction timings obtained in an environment that might be very different.

Review the system diagram for both environments and see if there are additional differences you can identify. Be clear to communicate these differences if anyone suggests using results from the test environment to imply the performance timing results would be the same in production.

I advocate testing in production whenever possible. In order to execute performance tests in production, I've typically worked in the middle of the night -- from 2am to 5am, for example -- while a production outage is taken. I've worked middle of the night on holiday weekends in order to gain test time in production as well. If you can't execute tests in production and you are left to execute performance tests in the test environment, then I recommend learning the performance behavior from your test environment and then communicating test results in terms of performance characteristics versus transaction timings.

Software testing resources: Performance testing in context Developing an approach to performance testing Entering the realm of performance testing

Performance characteristics might include knowledge of CPU usage or performance degradations. For instance, you might be able to discover that the report performance exceeds the acceptable range defined when generating a report with some specified amount of data (such as 2 months of accounting numbers). Or you might learn search performance begins to degrade when X number of users are logged into the system and X numbers of users are executing searches at the same time. You can look for high level information and learn overall performance characteristics that can be helpful but won't provide performance timings that should be used to presume production will behave in the same way.