During post-mortem analyses of failed or poorly-performing software projects, many companies realize that they underestimated how a new application would impact their IT infrastructure. Systems were undersized, not configured for new scales of load and the application's performance issues not understood. In hindsight, it's obvious that better load testing could have helped them avoid many of these issues.
Traditional load testing practices are often inadequate in this age of Rich Internet Applications (RIAs), which are placing heavier loads on IT infrastructures. Load testing today requires a solid understanding of the topology challenges of both conventional Web applications and RIAs, including Web 2.0 applications. This tip will cover the traditional challenges, as well as new challenges created with RIA. Further, the tip will provide insight and solutions to the particular challenges for both the existing challenges and those created by RIA in particular.
Load testing for Web applications, due to its distributed nature, has the basic challenge of load testing the application from end to end in addition to the individual elements involved in providing the application logic and infrastructure. Typically these challenges exists around the disciplines involved in the various elements of the application: presentation (Web), logic (application), data (database), security (policy and firewall) and networking.
While many of the challenges are related to the disciplines themselves or communication across the groups that support the various platforms, the common problems faced are both interdisciplinary and across disciplines. They can be organized as the where, how, and when of the five W's.
- Problem location identification (where): Where is the load test failing. Within the whole application, it comes down to these things: What element is not scaling: the firewall; the presentation code; the database table; or something else? Once it's identified as, say, the firewall, is it the network address translation, the order of the rules, or the amount of RAM? If it's the presentation, is it the actual code, or the Apache Web service?
- Base lines (how): How does the system react under normal load? Ironically, despite the emphasis on load testing, especially for enterprise and e-commerce applications, many tests end up showing that there were performance issues prior to new code being deployed. Unfortunately, many organizations simply start loading up test scripts prior to baselining the application prior to load testing. Rapid application development may discourage the baseline; however, even in shops with traditional method techniques the failure to base line is a common issue.
- Event correlation (when): Event correlation during load testing distributed system is both a communication and technical challenge. On the technical side, the largest challenge with event correlation is caused generally by two factors: the way various systems scale and reach bottle necks; and the shared nature of many systems.
Scaling challenges can be caused directly by lack of resources, poor code, or even mis-configuration that was not noticed until testing. Shared system resource issues are also technically scaling issues; however, additional complexity is added as the effects of other applications and traffic may need to be considered.
The world upside down
Load testing Rich Internet Applications like Web 2.0 apps shifts the where, how, when to a who-and-what paradigm. However, before considering the who-and-what paradigm, it is also important to recognize that the where, how, and when also become much more complex:
- Where? It's the Web, dummy. The problem is, what is the Web? RIA creates a situation where cloud computing becomes an asset and risk. Distributed computing becomes hidden computing. Upgrades, downgrades, and just unknown changes become the norm.
Identifying where the problem occurs becomes one of removing variables from the equation in large unmanageable pieces. For example, an app may be to be changed to test it against local storage versus an Amazon S3 storage solution. Then a conclusion is made: it's S3. Now is it Amazon's firewall, our Internet connection or an intermediate network?
- How? Baselines become tricky with RIA. Are you breaking the terms of service by creating a load test against a cloud resource? Do you have visibility into the third party's service level agreement (SLA)? What about traffic patterns? When is a good time to batch load data? Does the API support un-throttled access to the service for load tests? Load testing will take on a new life as capacity reporting, as historical data will have to be created at various times of the day.
- When? What about event correlation within a cloud? Are you kidding? Well, your customer is not going to understand that the issue "is the cloud." Identifying load issues timely within the cloud during testing -- and production issues -- is even more critical to the testing than traditional support models, as response times will be extended identifying resources to support cloud issues. Load testing will inadvertently thrash out these support issues.
As I mentioned earlier, the paradigm shifts with RIA to who and what in load testing. Let's look at those issues.
- Who: Load testing in a cloud environment will unfold to the who from two aspects: who can not provide the services as originally perceived and promised; and who can address the issue. If a cloud resource is limited, the development effort must determine if the application can be further distributed to using multiple resources. Or, perhaps, internal resources can be used to provide the service, or a redesign could provide a tiered approach. A tiered approach has become very popular in the networking and storage disciplines.
On the networking side, some data can be tiered via caching. The caching can address network latency issues to cloud and internal resources.
On the storage front, tiered storage is used to provide fast access to "current" data on fast storage area networks, and slower access to data that may not be needed immediately; providing access to data not unlike caching.
- What: RIA creates an additional distributed computing model that must be considered when load testing. Although similar to traditional distributed challenges mentioned above, cloud resources are often presented as a complete solution from a marketing and service standpoint; however, in practice using the resources still requires understanding and testing individual elements within the solution.
It is not uncommon, for example, for cloud storage providers to offload web calls to their storage to a third party, and for application tester to bump into load issues on the Web interface where margins are minimum -- or even negative if the provider has it as a loss leader -- before seeing any concerns with the storage interface.
Today, there's a double challenge in load testing. Traditional problems haven't been completely solved, although diligence can help. Meanwhile, load testing has become more complex, as RIA load testing must adapt to test environments that do not lend themselves well to testing or, in some cases, may not encourage or even prohibit load generation.
About the author: Ronald McCarty is a freelance writer and consultant specializing in systems, networks and information security. He received his bachelor's degree in computer and information systems at the University of Maryland's international campus at Schwaebisch Gmuend, Germany, and his master's degree in management with a specialization in information technology at Capella University. Ron's company, Your Net Guard, offers IT consulting and integration services in the Dallas/Forth Worth area.