In Agile development, is testing done in the same environment as development, or do you recommend separate environments?
Each member of an Agile team needs their own “sandbox,” whether they’re working on production code, test code or doing testing activities. A tester needs control over her own testing environment. She needs to know what build is deployed there, what database schema the code is using, what hardware and software are used.
I once worked on a team where we started out with no test environments available. The programmers told me to just use their machines for testing. However, they were constantly changing the code. It’s not productive to test a moving target like that.
Some teams use an “iteration zero” as a time to set up all their development and test environments. I’ve worked on teams where builds were automatically deployed to the test environments. However, I prefer to have control over which builds to deploy. There may be dozens of builds in one day. If I have a test charter or mission that requires several hours to complete, I don’t want new code deploying while I’m in the middle of that.
For most teams, one test environment won’t suffice. On my team, each tester (and programmer) has an individual test server with its own test schema. The schema uses canonical data which can be refreshed from seed at any time, so if your testing junks up your test schema, you can start with a new clean one. The canonical data is representative of production, but does not include every possibility. So we also have a test environment which mimics production, using a scrubbed copy of production data.
A staging environment is also essential. When we’re ready to release, we first deploy to the staging environment. It mirrors production, but we can do any testing that we want on it without fear of mucking up production data. This gives us confidence that we’ll deploy the right database and code changes to production.
As applications grow and needs change, test environments may need attention. When deploying a new build starts to take too long, or response time grows because the test database is overloaded, we may need to invest in new hardware and software. I’ve found it useful to add environment issues to the impediment backlog, preferably on a big visible chart where our system administrators and DBAs can notice and address it.
Dig Deeper on Topics Archive
Related Q&A from Lisa Crispin
Agile leader Lisa Crispin explains a more organic, more Agile approach to test reporting. Continue Reading
When it comes to Agile planning, average time over many iterations is a more important metric than individual story estimates. Continue Reading
Most inexperienced Scrum teams overcommit on what they will deliver, and when. Agile leader Lisa Crispin says that does more harm than good. Continue Reading