Smartphone usage for accessing and using business applications is on the rise as is the usage of smartphones in general. According to recent Nielsen surveys,
Today, CIOs, software architects and project and DevOps managers are identifying and implementing mobile performance testing tools and techniques to meet the mobile performance challenge. Analysts and experts weigh in on current mobile performance management technologies and strategies for performance management on mobile devices.
Challenges specific to mobile computing performance
Mobile technologies present software testers with a complex set of challenges, as issues can arise in several different places along the delivery chain. Issues can appear at the server level, the network level and at the end user level, says Gal Tunik, senior product manager at HP Software. The expectation is also different for mobile than desktop application performance. “With mobile, the user experience is very, very significant. Users are impatient about slow application performance,” says Tunik.
Developers and testers must work with “five or more platforms as well as a myriad of devices, taking into account service provider differences,” according to Linh Ho, vice president of corporate marketing at OpTier. She also acknowledges that users might quickly move to another app or service if they do not experience the performance they are expecting.
Furthermore, mobile devices pose new challenges that organizations must quickly adapt to. While desktop applications require a certain set of testing processes, mobile applications invite a wider and more complex set of performance validation challenges than those found in the desktop environment. They do not imitate the issues found in the desktop environment and have their own requirements, according to a recent HP white paper.
Tools available for managing mobile devices
Numerous monitoring and reporting tools are available for mobile device management. “Common capabilities for all these solutions includes the ability to deliver applications and updates, track and restrict application use and remotely configure devices and applications,” says Steve Brasen, managing research director of systems management at Enterprise Management Associates, Inc. He cites standalone solutions such as Odyssey Athena and solutions that are fully integrated with enterprise management suites offered by Kaseya, Symantec, IBM and Novell.
Linh Ho of OpTier emphasizes the “always-on” approach, and explains that OpTier has also partnered with testing vendors such as Keynote DeviceAnywhere and SOASTA, which offer solutions that complement offerings available in the application performance market space. Keynote DeviceAnywhere provides a real-time monitoring product, Test Center Enterprise Monitoring™ (TCE Monitoring), which is accessed over a cloud-based platform.
In addition, HP and partnering organizations such as Shunra offer complementary tools for mobility performance testing, says Gal Tunik. HP’s solutions include HP Mobile TruClient and HP Mobile Applications, which integrate with HP LoadRunner.
According to Jonah Kowall, research director at Gartner, “many tools exist, including both active and passive technologies. The main difference is that active technologies will simulate a user doing a specific set of steps to build a transaction SLA; this is something we call synthetic end user experience monitoring.”
However, Kowall emphasizes that “passive technologies (or what we call real end user experience monitoring) are typically more useful since they passively monitor the user interacting with the applications.” He explains that this passive monitoring catches all user interactions, whereas many issues go unseen in active monitoring technologies, which cannot provide the full coverage of an application’s features.
In Test automation tools for mobile devices, SearchSoftwareQuality.com contributor Nari Kannan describes available tools related to automation.
Factors to consider for mobile device performance
With so many different devices on the market, how can testers measure performance on all of them? Linh Ho of OpTier suggests prioritizing based on what the majority of end users are using. Likewise, Gal Tunik recommends examining your organization’s business goals and your “addressable market.” Test on the devices your addressable market is using, he says. He also highlights the importance of testing on older versions of phones, as those users who have not yet upgraded to the latest and greatest will still expect strong performance on the previous model device.
Businesses must also consider several other performance variables, such as the operating system, the other applications a user has on the device and the impact of third party content or components. Tunik recommends testing in real network conditions and on various carriers, since user experience can vary widely depending on these factors.
Best practices for implementing a mobile performance testing strategy
Enterprises can take steps to create a performance testing strategy that works with their existing resources. Linh Ho suggests that solutions need to be cross-siloed, providing value to developers, testers, QA managers and the operations team.
Gartner’s Jonah Kowall recommends examining the best of breed technologies, keeping in mind that the many available technologies are becoming available at varying price points. He advises, “Include the other buying centers in the decision-making processes to ensure the tool is leveraged across the entire organization. APM is moving increasingly towards a SaaS delivery model, and there are attractive solutions for those adaptable to that delivery method.”
In order to improve employee productivity, Gal Tunik supports the idea of doing functional testing with the “BYOD” (bring your own device) approach. However, there are a few security measures to take into account with BYOD.
Steve Brasen suggests that if organizations allow BYOD programs, they use sandboxing or virtualization to separate personal applications from work applications. He says, “Any APM strategy for mobile devices must be sensitive to enterprise security and compliance requirements. The portable nature of mobile devices makes them ideal targets for theft and for enabling inappropriate access to sensitive corporate information.” He further stresses that business applications and data must be secured across the server, the network and the devices themselves.
Work with existing technologies
While looking for ways to improve mobile device performance, businesses must examine the technologies they have already invested in, particularly in monitoring, according to Ho. The new solutions should work to complement existing technologies.
Similarly, Kowall suggests looking at how the newer technology fits into the larger offering, and the advantages and disadvantages that come with it. How your existing technologies work together with the newer solutions will ultimately determine the efficacy of real user experience monitoring.
What challenges does your organization face in optimizing performance on mobility applications? Follow us on Twitter at @SoftwareTestTT and let us know your thoughts.
This was first published in February 2012