The editors at SearchSoftwareQuality regularly recognize project management and software testing technologies for their innovation and market impact. The Sauce Labs' automated testing platform is the July 2015 editors' choice selection.
What it does
Sauce Labs' new continuous integration (CI) user interface (UI) is a much-needed tool for developers, one that's unusual -- if not singular -- in software test automation products, according to Chris Riley, DevOps consultant for Fixate IO. It's about time, he said, that a test UI showed the big picture. Many test interfaces only show the status of individual tests. This UI shows the status of whole builds.
Sauce Labs' goal, said CTO Steve Hazel, is to give developers a "CI interface that can provide better insights about, say, 50 tests running simultaneously."
While the CI UI is the big news, said Riley, the other features in Sauce Labs' June 11 release build out the company's strategy of increasing support for continuous and concurrent testing on multiple platforms and methodologies. These updates include new tunneling and archive pages and management tools, as well as single sign-on authentication capabilities. The tools can help developers test Web and mobile apps faster and manage many tests at once.
Sauce Labs is both the name of the company and its cloud-based, Web and mobile application testing platform as a service (PaaS). Based in San Francisco, the company was founded by Jason Huggins, the creator of Selenium, a Web browser automation software suite. As of spring 2015, 250 million tests have been done on the Sauce Labs platform since its first release in 2008.
Why is it cool?
Sauce Labs' cool factors are the new continuous integration UI and its broad support for many Web and mobile platforms, said Riley and user Yusuke Tsutsumi, senior software development engineer at Zillow, a real estate portal provider.
"This UI makes it easier to take on quality and all the other areas developers now have to be aware of, like security and compliance," said Riley. In addition, he said, its red, yellow and green alert design is easy to follow.
Having a vendor who looks at the big picture for enterprise development, as Sauce Labs does, is great and not the norm, said Riley.
The depth of features attracted Tsutsumi. "Sauce Labs is pretty much the biggest external tool we've used for test automation," he said. "Not only do they provide every commonly used desktop browser/OS combination, [but] they also provide a wide variety of mobile browsers for us as well. That was really the clincher for us."
What a user says
Sauce Labs' CI and continuous delivery workflow features and UI help Tsutsumi's team accelerate build and release cycles by significantly enhancing integration between Sauce Labs and Zillow's CI/CD environments.
"We're always looking for solutions to get feedback as quick as possible," Tsutsumi said. "We really care about quality and don't want to ship without testing everything and making sure it's up to our standards." The integration with Sauce Labs has helped Zillow reduce the test times and retain code quality. "I want to get to the point where we can validate the whole website within 10 minutes. We're not far from that now," he said.
Like other cloud-based services, Sauce Labs accommodates relative ease of scaling up and down. Its pricing model simplifies scaling, too, said Tsutsumi. At the enterprise level, "Sauce Labs charges by minute or [virtual machine], which is completely agnostic to the browser variation," he said. "We can easily ramp up 20 Android Chrome browsers one minute, and then move straight to 20 Windows IE11 browsers the next. That's flexibility that would have been very difficult for us to reach internally."
Sauce Labs pricing ranges from $12 per month for one user to $149 for small teams of 10 users. Enterprise prices vary.
Tools and trends for continuous integration
Agile best practices for continuous integration