Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Five mobile testing tools for developers to consider

This review of five mobile testing tools, Appium, Selendroid, ios-driver, Sauce Labs and Perfecto Mobile, will kick-start your research in a growing software market.

Mobile testing tools are experiencing a growth spurt. New products and services emerge nearly every day, and the time and energy needed to evaluate them may make all the free trials in the world seem not so free. This article looks at several mobile testing tools, listing their benefits, features and tradeoffs to help testers and IT managers make smarter investment decisions.

The first major challenge with mobile testing is the host -- that is, the device the tests will run on. Unlike a laptop or desktop, where the tests are developed and run on the same device, automated tests are typically planned and programmed on a laptop and run on some mobile device that may be plugged into the laptop and emulated. Or the mobile device might run somewhere else and be connected to the test driver over the Internet. Most large testing programs require testing on a handful, if not dozens, of machines, making physical plug-and-play untenable. These challenges should receive special attention when reviewing tools.

Mobile devices
Automated tests are often run on mobile devices

Appium

Appium is a small server written in node.js. Appium allows users to access their Android or iOS environment where they have the server running.

Since Appium uses the WebDriver framework, it can run tests in any language that WebDriver supports. Webdriver is a popular Web API that developers can program in many languages, including Ruby, Python, JavaScript and many more.

Appium interacting with iOs
Appium interacting with iOs

On the plus side, Appium is free and based on open source components. On the downside, that means setup can be a hassle; you have to install open source prerequisites and make sure the versions are compatible.

Selendroid and ios-driver

Two other options fairly similar to Appium are Selendroid (which recently replaced AndroidDriver) and ios-driver. They are both device-specific implementations of Selenium WebDriver for mobile devices. Selendroid is a framework for testing software on Android devices; ios-driver applies specifically to iOS devices.

With Selendroid, testers can run tests on an emulator or on an actual device. As of version 0.6.5, ios-driver does not support running tests on real devices. But that feature may be added in the next release.

Testers using Selenium products can benefit from the massive community of people that are also Selenium customers. Several websites offer free community-based support and testers can rest assured that most problems they experience will have been covered by someone else.

Sauce Labs

Sauce Labs offers a mobile testing product based on the Appium framework. Strictly speaking, Sauce Labs uses the Appium tool to host the tests, providing the environment for running digital tests. This approach provides the same benefits of freedom of programming language and dual (iOS, Android) platform support while removing the need for physical hardware.

In other words, Sauce Labs uses the Appium framework, but gives testers the device to run the tests on a virtual mobile device, then bills the device as a metered service by the CPU hour. This service is built in the same way as other Sauce Lab products, so testers get similar features, such as the potential to watch videos of tests as they run, or to link those errors reported in a test to a particular point in the video, making debugging much easier.

However, unlike the previously mentioned tools, mobile testing supported by the Sauce Labs platform is not free. The manual subscription plan is $12.00 a month, the individual plan is $49.00 a month and the small team plan is $149.00 a month. Enterprises should contact Sauce Labs for further pricing information.

Perfecto Mobile

Perfecto Mobile is not based on WebDriver. It has a native UI based on a proprietary TCL-based programming language. This approach lets testers who are not programmers select behaviors from a simple, no-coding development environment and write expected results in simple English. Tests designed with these proprietary tools may be more costly to move away from than tests written with open-source tools.

In addition to its native UI, Perfecto Mobile provides a RESTful API for integration with other interfaces including WebDriver and JUnit. Like Sauce Labs, Perfecto Mobile can provide either simulators or cloud-based real devices to run tests on. Perfecto Mobile recommends running tests on real devices. Perfecto Mobile provides live scheduled demos and prerecorded videos, but less extensive written documentation than the other products listed here.

The choice is yours

Driving the user interface with mobile testing tools is massively popular in the software world, and automation of mobile websites is quickly becoming common practice. This growth has caused an explosion in companies creating mobile testing tools. Now that testers have so many tools to choose from, the choice is more complicated, often made worse by folklore surrounding test automation and marketing material. Hopefully, this article provides a starting point for testers' research.

 

Update: This article was updated on November 13, 2014. The previous version stated that Perfecto Mobile provides simulations of cloud-based real devices. That has been corrected to read "either simulations or cloud-based real devices." Other clarifications about Perfecto Mobile have also been made.

Next Steps

Tips for testing mobile applications

Explore three types of security testing tools

Learn more about Sauce Labs

How to master mob programming

This was last published in November 2014

PRO+

Content

Find more PRO+ content and other member only offers, here.

Conference Coverage

DeveloperWeek 2015: Dev techies unite in San Francisco

Join the conversation

6 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Have you encountered hurdles when evaluating mobile testing tools? If so, what were they?
Cancel
In the article, nicely covered all the popular tools but still
Its is very difficult to evaluate an automation tool for mobile. 

1. Appium and Selendroid - Under development, not released completely, faced lots of challenges, though its easy to automate.
2. Perfecto mobile- I found very less clues on how to automate the mobile apps. Support is not good.
3. Robotium - I am including here though it is not mentioned in presentation- Its a good tool but it needs some development detail about the app unlike other tool worked on only apk. 
Cancel
@agnitesh, thanks for your feedback. I'm sure other readers who are looking at mobile automation will benefit from reading about your experience. Thanks especially for bringing up Robotium. We always like hearing about new tools we can explore. Your response makes me curious about what successes you've had. Have you learned any tips or tricks you could share here?
Cancel

I have explored few of the Mobile automation Tools which were mentioned:

Appium and Selendroid: Required heavy settings and configuration. Required MAC machine for iOS automation.

Perfecto: Its good tool but support is not very much pleasant.

See Test: Its good tool with good support for all the masior platforms like Android, iOS, Window and Black berry.

Robotium : Till now only support for Android platform.

Cancel
Thanks Parkash, for that good concise feedback.
Cancel
Thanks for compiling this list! I take on the occasional small mobile testing project, but it is far from my area of expertise, so it's great to have a nice, concise overview of some of the tools that are available out there. This list provides an excellent starting point for the next time I decide to consider a mobile project.
Cancel

-ADS BY GOOGLE

SearchMicroservices

TheServerSide.com

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

DevOpsAgenda

Close