Home > Ask the Software Quality Experts > Software Testing and Quality Assurance Questions & Answers > Mapping results in test environment to production environment
Ask The Software Quality Expert: Questions & Answers
EMAIL THIS

Mapping results in test environment to production environment

John Overbaugh EXPERT RESPONSE FROM: John Overbaugh

Pose a Question
Other Software Quality Categories
Meet all Software Quality Experts
Become an Expert for this site


Software quality news and advice
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google


>
QUESTION POSED ON: 08 April 2009
We are currently executing tests in an environment that is approximately 50% of the size of production. We need to map these test results to production. As an example, if one of the processes is returning an average response time of 3 seconds with average hit rates of 100 per hour, how will the same process behave in the production environment?

>

There is no accurate way of estimating this, frankly, that is better than testing the production environment itself. One highly theoretical approach to sizing which I have been involved with in the past uses the concept of CPU cycles per action to calculate total performance capabilities. The concept is that each action on a site uses a consistent amount of CPU cycles in each tier. If you can isolate the action and record its cycles, then extrapolate this out to a set of common usage patterns, you can arrive at the number of cycles needed for a given time frame. Theoretically, that cycle count can be mapped to processor, RAM and other hardware constraints. In my opinion, the time and effort required to gather this level of detail is greater than simply testing the site.

A second approach is to use a guesstimate ... if your production site is 2x your test site, you can assume it'll handle 2x the load. This, however, is also highly theoretical. There are so many variables at play. First of all, what does 2x mean? Twice the number of servers? Twice the RAM per server? Twice the CPUs? Twice as many U's taken up in a server rack? Also, production sites and test sites can have numerous subtle differences, including network subnet traffic, machine services (sometimes test machines are running different services such as an antivirus or firewall application), machine configuration, etc.

The best approach is to schedule and carry out a baseline test, where you run similar performance tests in your test environment and your production environment. This can be inconvenient to the user and the testers involved, but it is the best way to establish a multiplication factor. On a Web application I used to own, we did all of our performance testing between 11:30 p.m. and 5:00 a.m. (our lowest service window). Our firewall served up an Unavailable message and all traffic was shunted away. If you can carry this out, and schedule it for regular intervals (say every 12 or six months) it will help you understand the relationship between both sites as well as how that relationship changes over time.


Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google



RELATED CONTENT
Software Testing and Quality Assurance
Why do performance testers write new scripts so often?
How to create performance testing workload models
Fixing Web application performance troubleshooting problems
Expert advises on implementation of Selenium IDE for effective software testing
When should regression testing occur in an automated test plan?
Achieving peak performance in integration testing
Getting answers about OpenSTA script problems
Defining core software regression tests
Breaking in functionality on UI application pages
Where to find good methodology guides for software testing

Software testing and quality assurance (QA) fundamentals
Software Testing: New software testing technologies bring new challenges
Testing strategies for complex environments
Astronaut's STPCon advice: Teamwork delivers "The Right Stuff"
How to make your software tamperproof
Software consortium seeks standard quality metrics
Demo: Using WebGoat, a free software testing tool
Seven steps for a quality change and configuration management program
Winning responses to "Why is QA always the bottleneck?"
Where to find good methodology guides for software testing
5 ways to answer executives' unfair software test, QA questions

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
build  (SearchSoftwareQuality.com)
code review  (SearchSoftwareQuality.com)
conformance testing  (SearchSoftwareQuality.com)
error handling  (SearchSoftwareQuality.com)
garbage in, garbage out  (SearchSoftwareQuality.com)
load testing  (SearchSoftwareQuality.com)
NUnit  (SearchSoftwareQuality.com)
quality assurance  (SearchSoftwareQuality.com)
stress testing  (SearchSoftwareQuality.com)
white box  (SearchSoftwareQuality.com)

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary



Search and Browse the Expert Answer Center
Search and browse more than 25,000 question and answer pairs from more than 250 TechTarget industry experts.
Browse our Expert Advice



Software Quality - Software Maintenance, Software Requirements, Software Standards
About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations' technology projects - with its network of technology-specific websites, events and online magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Site Map




All Rights Reserved, Copyright 2006 - 2009, TechTarget | Read our Privacy Policy
  TechTarget - The IT Media ROI Experts