Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Mobile load testing tools review: Appvance vs. HP StormRunner

Appvance PerformanceCloud and HP's StormRunner Load are two of the most advanced load testing tools on the market today. See how the two compare.

Customers are impatient; it's a fact. Especially when using their mobile devices. If your mobile application's performance does not meet your users' expectations, they will find another application that does. This makes performance and load testing critical to every release. Today's load testing tools are improving to help pick up some of the slack created by the shortened time frames that Agile testing brings.

In this article, I'll discuss the features of two newly released load testing tools that are designed for Agile performance testing and work well for mobile application development. These tools are Appvance's PerformanceCloud and Hewlett-Packard's StormRunner Load.

Let's start with the nuts and bolts. Both Appvance's PerformanceCloud and Hewlett-Packard's StormRunner Load are cloud-based. Both allow the tester to run tests locally or on external clouds, such as Amazon AWS, RackSpace OpenStack, Azure, Savvis, etc. PerformanceCloud supports the HTTP, SOAP, REST, ESB and BPM protocols, while StormRunner Load supports the TruClient, HTTP and TruAPI protocols. HP StormRunner Load recommends running on Firefox, while PerformanceCloud supports a wide range of browsers including IE 8 or higher, Firefox 3.6 and higher, Google, Chrome, and Safari, and offers a patent pending uxAvatar technology to drive Ajax apps in IE, Chrome, Firefox, Safari and Opera.

Now let's look at the advantages of these load testing tools. Both products offer ease of use and speed in both scripting and executing tests, which is a critical feature in Agile testing and continuous integration testing. Both products offer scalability, allowing for ramp up from one to 10 million or more geographically distributed users.

So, what makes them different? Obviously HP's StormRunner Load integrates with the ever-popular HP testing tools suite, including Quality Center ALM. This will be a huge advantage for teams that are already used to testing with these tools. StormRunner Load also offers a real-time predictive analytics tool that allows developers to find their bottlenecks easily during Agile iterations. PerformanceCloud's counterpoint is a multitude of reports and a drill-down capability for assessing performance issues.

HP's StormRunner Load's ease of use is a big improvement to the Performance Center and Load Runner products and provides a nice Agile addition to the HP test suite. However, Appvance's PerformanceCloud's test script reusability, and most importantly, the ability to test the presentation layer make this product a very strong competitor.

On the other hand, Appvance's PerformanceCloud integrates with a variety Agile tools including Hudson, Rally, Jenkins and Cloudbees. Through PerformanceCloud integration with Rally, test cases can be mapped to user stories. This is a nice feature that really supports Agile testing. In addition, one of the most exciting features about Appvance's PerformanceCloud is that a single test script can be repurposed as a unit test, functional test and regression test, as well as a load, performance, stress and scalability test. This is truly Agile.

And load testing tools aren't everything. One of the most important considerations in mobile application performance testing is the end user experience. Testing the presentation layer is critical because the presentation layer is where the user first meets your application. If your potential customer gives up and moves on to another website, it doesn't matter how well your back end performs. Appvance's PerformanceCloud goes further in testing the presentation layer. Load testing tools for the back end are still important, and HP's StormRunner Load does perform well there.

Next Steps

Test yourself on APM and metrics knowledge

Check out these tips on optimizing performance

Learn about cloud APM

Dig Deeper on Topics Archive

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

Have you used either of these load testing tools? Let us know what your experience has been. Tell us the good, the bad and the just plain weird.