Ask the Expert

What are the risks in using open source test automation tools?

Hi, I would like to follow up on a question posed on March 2, 2009: "What are the parameters you take into account while selecting an automation tool?"

How do you feel about open source solutions? Would like to hear your takes on both "community-driven" (Selenium) and "commercially driven" (soapUI).

I'm responsible for the test automation frameworks and I found myself trying to go the "open source route" as much as possible. Are there any potential risks of doing that, besides the obvious lack of "enterprise guaranteed support"?

Requires Free Membership to View

I'm actually a solid proponent of open source software (OSS) testing tools. In a previous position, I used both Selenium and Watij to drive automated testing and found both of them to be solid testing solutions. The lack of enterprise support is often considered a drawback to using OSS solutions, although I found I received great support from user groups for both tools -- I was never really blocked in my work.

I think a few keys contribute to the value and success of tools like Selenium and Watij. First, these tools are often developed by more than one engineer or built for more than one project. As the community uses the tool, it becomes more broadly applicable. Secondly, the people building and using these tools are generally top-notch testers and have a good idea what it takes to succeed. Finally, I find these tools rarely force you into a given test methodology.

Commercial software testing tools definitely have their place. For instance, I'm a happy use of WAPT (Web Application Performance Test tool), which is a commercial Web load generation tool. It's flexible, dependable, and above all it gets the job done quickly. There are some commercial test tools, however, that have a huge, steep learning curve. These don't lend themselves to use in short-term projects or where testers aren't technical enough to use them. Some tools also force organizations to organize and carry out their work in a specific process, which may or may not be applicable to the organization or project.

One thing to keep in mind is that, given the choice between an open source solution and a commercial solution, the decision should be made based primarily on the needs of the project. In some cases, the cost of the tool (a six-figure license agreement, for example) needs to play into the decision, but for the most part I recommend teams make decisions based on how well the tool will meet their needs.

Rocky Nook Publishing has released a book called Software Testing Practice: Test Management (Spillner et al), which has a detailed chapter on evaluating test tools. It's a bit more analytical than I generally like to be, but it does provide a solid overview on how to rate tools based on a number of criteria. I recently reviewed this book on my blog, Thoughts on QA and Engineering, if you'd like more information about it.

One last word about open source solutions. I like them because they often seem to be more agile in terms of implementation. They're often developed and used in continuous integration environments, which appeals to me as a fan of agile.

This was first published in March 2009

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: