Performance testing of a large-scale ERP application requires specialized skills. Does it make sense for performance...
testers to be part of the Scrum team or is it better to have them be brought in as consultants? If they are brought in as consultants, how can I optimize their time? Can I bring them in later in the release cycle?
When software performance is critical, performance testing needs to happen early. My team does design spikes so we can do performance testing and verify that a new code architecture scales appropriately. Some performance testing expertise on the Scrum team is required, but it doesn’t necessarily have to be a performance testing specialist. In the case of my team, we budgeted time to acquire the necessary tools and expertise in order to do the performance testing ourselves. This was a significant but necessary investment.
I’ve worked with larger organizations that have tried a variety of approaches to making sure each Scrum team gets the specialized expertise needed at the right time. Some large companies with many Scrum teams have dedicated “performance, stability and reliability” teams, whose members rotate among the Scrum teams, helping with performance testing and also transferring their skills to the regular team members. It might be ideal to have a performance specialist on each Scrum team that needs one, but this can be an expensive solution.
You may need post-development testing cycles for additional performance testing, especially when testing embedded software, or in the case where your team’s test environments don’t include all the systems with which it must interact in production. If this is the case, be sure at least one team member coordinates testing activities from project start to finish for continuity and to minimize duplicated effort.
You’ll need to experiment to find what works best for your situation. Do you have enough performance testing specialists that you can farm one out to each team that needs it in a timely manner? Or would it be better to train each team in performance testing skills? Choose performance test tools that are appropriate to the existing expertise of the teams. For example, my team searched for a performance tool where the test scripts could be written in Java, because our production code is written in Java and our developers wouldn’t have to learn a new language or task switch between languages.
One of the most difficult challenges may be providing appropriate performance test environments. For meaningful test results, you need to test in an environment that matches the production environment. This may require a significant investment of money and time to set up.
As you plan your next release, discuss the performance testing that needs to be done, and make sure there is a plan to provide the right hardware, software and people to accomplish it at the appropriate time.
Dig Deeper on Topics Archive
Related Q&A from Lisa Crispin
Agile leader Lisa Crispin explains a more organic, more Agile approach to test reporting. Continue Reading
When it comes to Agile planning, average time over many iterations is a more important metric than individual story estimates. Continue Reading
Most inexperienced Scrum teams overcommit on what they will deliver, and when. Agile leader Lisa Crispin says that does more harm than good. Continue Reading