Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Finding the best automated testing tools for your business

Testing experts Michael Larsen and Matt Heusser explore the top automated functional testing tools in the market today like Selenium, Hewlett Packard Enterprise UFT and TestComplete Platform.

Application testing's wide variety of technologies and styles make creating a single, universal test tool impossible -- companies need a toolbox. In this article, we focus on automated functional testing tools, a type of software testing tool that replaces human activities with a computer.

Automated functional testing tools typically drive the UI, mimicking the actions of a human, and then check for expected results. Record-playback tools let a user record interactions. This creates a literal script that can be played back again and again. The drawback to record-playback tools is how they work; they can often fail when a code change alters the appearance of a screen, making it different than the expected results. Because of this, many of these scripts are stripped down and made less literal.

Checking the text on a button, instead of the entire window or webpage, makes the tests less brittle. In that process, we replicate the other approach often used with these tools: keyword-driven tests. Keyword testing is often more robust and less error-prone because it is able to look at the specifics of interactions, rather than every aspect for each line in the script.

In this area, there's an important question to ask: What type of application am I testing? Are we looking at a compiled desktop application, a website, a contained web application, a mobile website or a mobile application? Each requires a different approach and different tools. Native mobile applications can be extremely challenging to automate or test in a desktop; mobile web applications can run in a desktop, but may look or behave differently. Unless it's an all-in-one tool, it is unlikely that a web automation test tool supports mobile applications.

All-in-one automated functional testing products

TestComplete Platform

Less-technical testers who work on Windows machines and don't want to write code should consider SmartBear's TestComplete. Based on record-playback, TestComplete has two views: visual and code. The visual view of TestComplete shows the commands, such as click, wait, type and so on followed by parameters -- where to click, how long to wait and what to type. That means a business user can look at the test and read it, agreeing that it's what the software should do.

Programmers can dive into code in TestComplete, but wrap that code in a function. For example, the login function can be a single line in the visual view, taking a username and password as inputs.

TestComplete is ideal for testers who don't want to program and work on Windows devices, and who want the flexibility to test Windows, web and mobile -- iOS and Android -- applications all in one package. Teams that want to version control tests right along with code will want to use Git, Subversion, Mercurial or Microsoft Team Foundation Server; the application uses its own format for data, and you'll want to check code in from the TestComplete development environment.

TestComplete provides additional paid support, including a training package of videos you can purchase; a monthly, two-day remote video training course; in-person training; and hourly support. The biggest issues we have experienced with TestComplete are supporting just-released versions of browsers, especially Internet Explorer, along with Java Applets running in the browser. SmartBear does work actively to support new browsers, but automatic updates can sometimes trigger before the SmartBear team has a new version in place.

Pricing for the TestComplete Platform is based on two models. You can purchase a Node-Locked License, meaning the software is tied to a single machine, or a Floating User License, which can be installed on multiple machines, but is tied to a single user account. All users purchase the TestComplete Platform license first, and then choose to purchase licenses for web, desktop, mobile or all of the above.

TestComplete has been expanding into the open source world, adding recent support for Jenkins and Selenium by way of Java's JUnit. If your team is using those tools already and you want to run every application test in one place, TestComplete might be the right choice.

Hewlett Packard Enterprise UFT

If your concerns go beyond automating the software tests to providing evidence the tests ran, when they ran and on what build and server, you might consider Hewlett Packard Enterprise's UFT, which stands for Unified Functional Testing.

Hewlett Packard Enterprise's product, formerly known as QuickTest Pro, is designed to integrate with Quality Center, an application test coverage tool that measures what was tested, along with Sprinter, an exploratory test coverage tool. Together, the three apps provide documentation and evidence that an insurance, medical or financial services company might need, especially to stand up to auditors. The applications work together to create a sort of quality suite.

Better known as part of a suite than a stand-alone application, UFT becomes a clear recommendation for organizations that already work in a Hewlett Packard Enterprise environment. Companies that already rely on Hewlett Packard Enterprise for service and support may find UFT the clear winner. Like TestComplete, UFT uses a visual model. But its model is more like a flowchart than a grid, with the possibility of loops and conditionals -- if statements -- visualized.

Telerik Test Studio

Teams using Microsoft Visual Studio to create web or desktop applications might be interested in Telerik Test Studio. This is especially true for the more technical testers who are not afraid of code, or teams that want to have the programmers contribute to the automated testing.

That's because Telerik's code is the same native programming language as Visual Studio. "Drop in" to the code in Telerik to extend it, and you can find yourself in the Visual Studio development environment, writing code in the exact same language as the programmers on the team. That means the programmers and testers can even share code libraries, which can be very popular. TestComplete's C# and VBScript modules are not true Visual Studio code and have minor syntax differences.

Teams that create in Linux or on Apple, for example, creating open source applications, will have problems with all the tools above, because they will need to have application testers using Windows. Because of the tight Visual Studio-Telerik integration, we do not recommend Telerik for teams on Linux and Mac OS X.

EggPlant Functional

Teams writing code on Linux, Mac or in a mixed environment will want to look at eggPlant. EggPlant records the user interface, like the record-playback tools above, then creates a simple script. The script looks very much like BASIC code; testers can add simple if statements, variables and for loops. In our experience, most testers are able to pick up this kind of programming very easily -- although it is possible for testers to get out of their depth very easily, creating complex test code that is hard to maintain and debug.

TestStudio, UFT, eggPlant and TestComplete all have free trials. If the winner is not clear -- UFT for a Hewlett Packard Enterprise organization and eggPlant for Linux -- then explore, learn, pick and then build a proof of concept with the winner.

Stand-alone web and mobile automated testing tools

Selenium

Teams that are comfortable with code and only need to test web applications through the UI are bound to look to Selenium. Selenium is a free, open source project for testing websites and web applications. Its most popular component is WebDriver, a code library to use in many programming languages that automates the web browser with commands like open, click, type and so on. Selenium provides a number of components that need to be installed to work effectively.

The on-ramp to Selenium WebDriver is the Selenium integrated development environment (IDE), which is an in-browser record-playback tool that can not only produce, but record-playback scripts and also convert them to WebDriver code in Java. As a record-playback tool, the IDE only runs in FireFox; WebDriver runs in Internet Explorer, FireFox, Chrome and Safari.

There is a large development community behind Selenium WebDriver, and a variety of popular programming and scripting languages are supported. Teams that want to write test code in the same language as the production code will find support for Java, C# Ruby, Python, JavaScript -- Node.js -- and more.

Commercial software support exists around Selenium. Sauce Labs provides virtual test environments to simulate any modern browser and the grid to run many application tests at once. Learn more about Sauce Labs by visiting their site.

Appium

Companies that develop mobile applications and mobile websites will want to look beyond Selenium to Appium, an open source mobile testing tool. Appium uses the same API as Selenium WebDriver, which means the application testing frameworks available for WebDriver also are available for Appium. Sauce Labs provides Appium support, as well.

Perfecto Mobile

Although it started out offering a virtual lab of real devices for users to test on, Perfecto Mobile has gone on to extend its features to support automated testing against that network of real devices, along with boot camps and training options. The company has a proprietary interface to design software tests easily -- claiming little to no coding required -- which also supports Selenium and other open source frameworks.

If your product will be used by a wide variety of mobile devices -- Android, iOS, tablet and phone -- and you need to support both older devices with smaller screen resolutions or newer devices with larger screen resolutions, you might want to look into what Perfecto Mobile has to offer.

Sauce Labs

Managing a library of test devices is one of the first problems makers of mobile software run into. If you are working with Apple only, there are four iPads, three iPhones and the watch. Those are just the devices running the latest version of iOS. Add older versions and then the entire Android ecosystem and you have a problem. Sauce Labs simplifies this problem by offering a large set of emulated or real mobile devices that are available as a service when you need them. This service works for hands-on and automated testing.

Next Steps

While enterprises are recognizing the need for mobile, few are recognizing the need for mobile application testing.

Getting out of your own way by implementing automated testing is a big step in releasing apps faster.

Here are the top three testing objectives to help direct your tests.

TestComplete Platform allows automated app testing

 

This was last published in March 2017

PRO+

Content

Find more PRO+ content and other member only offers, here.

Buyer's Guide

Choosing the best software testing tools for your business

Join the conversation

14 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Which automated functional testing tool do you think is best for your company?
Cancel
Why stop at GUI level?
Functional testing tools at API level don't have many disadvantages comparing to the GUI tools. I'd vote for SoapUI Pro (free version has no parameterization, too much time to waste hardcoding).
Cancel
Of this list, I've only used Selenium WebDriver, and it worked well for our purposes. We implemented the same test using WATIR for comparison purposes, and we ended up preferring Selenium.
Cancel
I support the vote for TestComplete. The value of its powerful engine, libraries, and robust interfaces outweighs the license cost. These "free" open-source tools come with extensive costs of in-house development and time-to-solution.
Cancel
You should also look at SeeTest by Experitest. I've used it in the past year for my mobile testing and i am very pleased!
Cancel
I think the term automated is a bit relative here.  We aren't exactly doing a lot of testing with state diagrams and flow charts these days.

However, of them Selenium seems to be the choice for end to end UI web testing, Telerik may be favored by devs more, and the others are kind of a pick which solution fits your context better.
Cancel
Need to test all integration points as we replace our POS in a Nonstop Cobol, Hybris, Oracle RetL & PeopleSoft FCinance environment - Any guidance is welcome.
Cancel
Hi, Look at Opkey, a much more complete tool that automate across web, mobile, web services, databases, desktop apps, Oracle, service now, salesforce etc. under one platform. Opkey in association with its sister tool pCloudy can seamlessly test POS as well.
Cancel
Hello ALL,

Am Looking for an automation testing tool that helps to test a complete Text- based application. We are using a Telnet application which is built using forms in notepad. Each screen in the application is created using a notepad in complete text based. Am only used to web automation testing where in i can choose the field and the tool records the exact feel and property. But for text based i do not know how to proceed. Please help me. Thanks in Advance.
Cancel
Qualitia Soft is also best test automation tool leverages the power of both Selenium and QTP. 
Cancel
The analysis is very HL and incomplete in several ways so I'd like to add some details:
1. Selenium is an API not a tool so there is coding experience required. If you are stuck then you are on your own looking for solutions from forum to forum. If your app going responsive using angular then you'll have to go with Protractor which uses java script and you can't use existing Selenium framework.
2. UFT/QTP: There is no future support for VB since MS is out of the picture UFT is going LeanFT. 
3. Test Complete is behind in tech/features compare to other tools in the market, they'll get there in few year I hope. 
4. The solution that is on top of my list after detailed POC of all the solutions out there is SilkTest for functional testing. Newer version uses WebDriver and Appium so you get the best of both worlds and for TCM I'd vote for Silk Central. It supports all sorts of fucntional tools like QTP/UFT/Selenium/Mobile executions. 
thanks
Cancel
Hi All Look at Opkey, a much more complete tool that automate across web, mobile, web services, databases, desktop apps, Oracle, service now, salesforce etc. under one platform. Opkey in association with its sister tool pCloudy can seamlessly test POS as well.
Cancel
https://github.com/wangyudongdom/automation-test-tool

Recommend a very useful automation test tool for RESTful API
Cancel
I think both sauce labs and perfecto are good tools. I will also recommend pcloudy, this will give you the best service.
Cancel

-ADS BY GOOGLE

SearchMicroservices

TheServerSide.com

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

DevOpsAgenda

Close