What is the biggest software testing challenge in terms of creating and maintaining a staging environment?
Not creating a staging environment that exactly mimics the production environment.
I've seen too many companies skip staging and move from a test environment directly into production. It's a high-risk mistake, especially as applications move into the Web services space, where availability and performance are key and there is no acceptable amount of downtime. The mission-critical nature of software that must be available at all times doesn't allow room for error. Creating a staging test environment that mimics your company's production system becomes all the more critical to your success.
Close is not good enough. As testing professionals and leaders, we must demand an exact replica of production, including any third-party software, open source code and data. Every little thing needs to be an exact, but independent, copy of production to truly qualify as a staging environment. This includes data for the most part, although there is a need to work around sensitive data to keep it safe.
If you have sensitive data, or personally identifiable data that is controlled by regulations, then that data has to remain protected and shouldn't be copied to any test environment. However, you can use data that has been masked or scrubbed so that the type of data remains intact but the value does not. By masking data in a staging environment, you are not exposing sensitive data to potential threats. It's imperative to verify that the data is masked or scrubbed before it moves into a staging test environment.
The software testing challenge is increased by the fact that most customers use varying configurations. Multiple platforms are available and there is a variety of hardware and software combinations. A staging environment cannot realistically mimic all of those possibilities. It's up to the software development team to decide which configuration is most important to test, based on a combination of risk analysis and which setup is most often used.
In addition, the team may review secondary setup options, such as configurations that are more problematic, result in greater support calls or possess higher-risk security vulnerabilities. You don't have to stop at one staging environment; create as many as you can handle. Keep in mind the time involved not only in executing tests, but also in setup, maintenance and upgrades to the staging environment(s).
This was first published in June 2013