Mobile application testing is both a critical and a complex component of mobile application development. It is crucial to have a clearly defined and well-developed mobile testing strategy and framework. The main components of a mobile application testing strategy include usability; performance; security; and functional and nonfunctional testing across multiple platforms, devices and browsers.
A complete mobile testing strategy must also account for testing across differing network connection speeds and geographical locations, as well as address the use of Wi-Fi, 3G or 4G connections. Testing must confront such issues as screen resolution and brightness, CPU, memory and OS optimization. The mobile testing strategy must be geared to the architecture of the applications under test whether they are Web, mobile Web, native applications or hybrids. Finally, an organization must consider the test approach, primarily the use of emulators versus actual devices, or even real user monitoring.
Once a program manager has determined the team's mobile needs and developed a complete mobile testing strategy, the question becomes how to effectively execute such complex testing as a part of an on-going mobile application lifecycle strategy. Although cloud-based testing using emulators may have been sufficient in the early years of the mobile device age, the sheer explosion of mobile technology has made this an incomplete test strategy. Effective mobile application testing now needs to include cloud-based mobile test automation tools that use real devices. Due to the specialized skills required for this testing, hiring professional testing services should also be considered.
Moving from strategy to implementation
Finding vendors to support a defined test strategy is critical. When looking at device farms, approach such vendors as Perfecto Mobile or Xamarin Test Cloud. Crowdsourcing, such as the service offered by Applause (formerly uTest), may be an option for certain types of applications. While crowdsourcing can provide a lot of data quickly, it may also create incorrect initial impressions of the application's quality and utility.
Emulation remains a strong candidate, at least for initial testing, with popular and largely inexpensive emulators widely available for both iPhone and Android devices. However, most emulators remain incomplete in important ways. Experienced testers looking for accurate and repeatable results might not fully trust an emulator.
Automation has to play a key role in any enterprise testing strategy. Whether for tracking test case scripts and results, collecting and analyzing real user data, or analyzing performance on the client device and load on the server, the job is just too large for a manual approach. Vendors such as Soasta offer testing solutions in the cloud, while Hewlett-Packard has added mobile capabilities into its highly touted product line.
The cloud can fit in as the source of a real device farm, in providing resources for real user monitoring, and as a source for increasingly improving test automation tools and data analytics. Count on leveraging cloud resources from a provider like AWS or RackSpace in one or more of these ways when you embark on a mobile testing initiative.
Many application builders are still content with incomplete testing or even no testing, in part because mobile testing is still in its infancy and can be difficult to do correctly. The options and tradeoffs might seem to present insurmountable complexity. Inexperienced project managers might be tempted to simply let customers and users provide primary feedback on features and bugs.
Fortunately, the mobile testing landscape is changing for the better. In response to the specific needs of mobile testers, niche vendors including Soasta, Perfecto Mobile, Tricentis and Keynote have emerged, supplementing the standard test automation tool sets provided by the market leaders. Thanks to more options, better tools and more comprehensive data, organizations can have far more confidence in the ability of their applications to meet enterprise needs.
The three most important criteria for choosing mobile testing tools
Test automation tools for mobile devices
Automated testing tools: Four reasons why projects fail
AWS mobile development tools hold steady lead
How Amazon CloudWatch provides automated APM tools