Performance testing of a large-scale ERP application requires specialized skills. Does it make sense for performance testers to be part of the Scrum team or is it better to have them be brought in as consultants? If they are brought in as consultants, how can I optimize their time? Can I bring them in later in the release cycle?
Requires Free Membership to View
When you register, you'll receive targeted emails designed to keep you informed of the most relevant information on Agile development, application security, testing & QA, software requirements, and more.
Hannah Smalltree, Editorial Director
When software performance is critical, performance testing needs to happen early. My team does design spikes so we can do performance testing and verify that a new code architecture scales appropriately. Some performance testing expertise on the Scrum team is required, but it doesn’t necessarily have to be a performance testing specialist. In the case of my team, we budgeted time to acquire the necessary tools and expertise in order to do the performance testing ourselves. This was a significant but necessary investment.
I’ve worked with larger organizations that have tried a variety of approaches to making sure each Scrum team gets the specialized expertise needed at the right time. Some large companies with many Scrum teams have dedicated “performance, stability and reliability” teams, whose members rotate among the Scrum teams, helping with performance testing and also transferring their skills to the regular team members. It might be ideal to have a performance specialist on each Scrum team that needs one, but this can be an expensive solution.
You may need post-development testing cycles for additional performance testing, especially when testing embedded software, or in the case where your team’s test environments don’t include all the systems with which it must interact in production. If this is the case, be sure at least one team member coordinates testing activities from project start to finish for continuity and to minimize duplicated effort.
You’ll need to experiment to find what works best for your situation. Do you have enough performance testing specialists that you can farm one out to each team that needs it in a timely manner? Or would it be better to train each team in performance testing skills? Choose performance test tools that are appropriate to the existing expertise of the teams. For example, my team searched for a performance tool where the test scripts could be written in Java, because our production code is written in Java and our developers wouldn’t have to learn a new language or task switch between languages.
One of the most difficult challenges may be providing appropriate performance test environments. For meaningful test results, you need to test in an environment that matches the production environment. This may require a significant investment of money and time to set up.
As you plan your next release, discuss the performance testing that needs to be done, and make sure there is a plan to provide the right hardware, software and people to accomplish it at the appropriate time.
This was first published in January 2011