Users of the open source Selenium automated functional testing platform are getting some commercial support with startup company Sauce Labs. Today the San Francisco-based company, co-founded by Selenium creator Jason Huggins, is rolling
Sauce IDE joins the company's existing offerings, Sauce RC, a distribution of Selenium RC that allows users to write test scripts in various programming languages, and Sauce OnDemand, the company's cloud based testing service.
According to CEO John Dunham, the intersection of software development and deployment along with agile methodology revealed a need that was not being answered by traditional functional testing tools. Building on the popularity of Selenium was a way to address that need, he said.
"This comes at an interesting time in terms of how enterprise and other applications are being developed and deployed," said Jay Lyman, an analyst for open source, The 451 Group. "We have virtualization and cloud computing that change things, and Web app development being applied to mobile devices and software. A few things have converged, and we're reaching a point where this type of solution makes sense."
He continued, "It matches some of the trends we see, and part of that is development of those applications is coming together with deployment; and running those apps in the environment where they were intended. In all of these things, speed becomes paramount. [Sauce Labs is] focused on making developing and testing applications in different browsers fast for developers and ISVs; they're focusing the on right things."
However, he said, "At same time, they're not the only ones focused on this. They're entering a very competitive market." Lyman said SpringSource, Sun/Oracle, JBoss and Red Hat "are also on the cusp of these trends." Engine Yard, he said, is another company addressing these trends, offering on-demand deployment and management of Ruby on Rails applications. "It's a crowded space, but there is plenty of room for a number of different approaches and solutions."
Sauce Labs, Dunham said, is addressing some issues users have with Selenium. "People using Selenium, despite being fans, felt the headache of maintaining all the versions of the browsers to be configured in the test lab, and the underlying OSes and patches, was really burdensome," Dunham said. "It's not the best use of time for QA or developers. Once you have an inventory of scripts to check the features of an application, the duration of time to complete the tests, the pure number of tests driven by the number of features, and the number of browsers to run them on, is really substantial in short order. Offering a cloud service for Selenium was a slam-dunk way to solve the problem."
And now with Sauce IDE, the company is attempting to bridge the gap between coders who use Selenium RC and non-coders who use Selenium IDE. Sauce IDE tests can be run via Sauce Labs' Sauce OnDemand cloud service. "For our target group using Selenium IDE, it's a big conceptual leap for them to go from recording some smoke tests to having to learn a programming language and the things you have to do to run tests," explained Huggins, who is Sauce Labs' CTO. "Some people want to stay in the IDE but get more benefits, so they can get the advantages of our cloud service without having to learn lot of new things in one [offering]."
Sharethrough, a video distribution network for branded online video content in San Francisco, was an early user of Sauce OnDemand. Co-founder and CTO Rob Fan said the company initially used Selenium to get some basic browser coverage. However, as their testing grew more complex, requiring hundreds of tests running simultaneously, he said they looked into Selenium Grid, "but we realized in playing around with it, you could get it running but to have it usable by the entire team it would become a lot of hassle. With testing you want the results to occur as quickly as possible, so it wouldn't have been worth the investment of our resources."
When he came upon Sauce OnDemand, "I liked how they took that concern out of my hands. I could focus on writing the tests I needed to write, plug them in, and run them on their system."
An issue with open source products is that users are somewhat on their own, despite the open source community. "One thing that would help as a whole, me and anyone out there, is for them [Sauce Labs] to provide better training and materials on Selenium testing. If we could have a one-stop shop to tell us the best ways to test, that would be great," Fan said.
Another issue with open source tools is that although they can hone in on the hottest features, Lyman said, "the flip side of zeroing in on the hottest features is that you may have to sacrifice some features/functionality in testing and frameworks. Depending on the ISV or enterprise shop, there may be some bells and whistles they require. That's why there's not a one size fits all, and another reason why you see Rational/Borland/Mercury [testing tools] alongside some open source options."
Going forward, Huggins said Sauce Labs is working on the pre-alpha code for testing mobile apps on the iPhone and Android, and expects that capability to be available in the Sauce 2.0 code base.