Most software testers are familiar with open source tools like FitNesse; but consultant Randall Rice thinks they should also know about other lesser-known free, inexpensive and handy tools. Rice named and demoed his favorite test tools, in a StarWest 2009 session in Anaheim, Calif., this week. He recommended tools in various areas, including test design, execution and management.
As a proponent of open source testing tools, Rice -- founder of Rice Consulting -- has run into management's resistance to using open source software in some companies.
"I realize that many of you are in companies that prohibit open source tools," he told the packed-room audience. Reasons vary. Often, management may not bring in open source because it could be seen as an admission that their purchase of expensive commercial tools was a mistake. Also, executives say they're worried about the lifespan of open source products. That excuse makes Rice laugh, because so many software vendors have been acquired and their products retired or changed drastically.
One company's development managers told Rice they needed new testing tools but couldn't afford more licenses. Yet, they would not permit the usage of free, open source tools. Rice was dumbfounded when they chose no tool rather than a free tool.
Rice advised testers to make the business case in terms of cost versus benefits and of having a free tool vs. no tool. In situations where existing commercial software is being used, note contributions of major vendors to open source and free software offerings.
Next, Rice gave his reviews of some tools you may not be aware of that are good alternatives."
Test design tools
CTE-XL, a free set of classification-tree-based test design tools, "is one of the most fantastic bargains I know of," Rice said. "These tools give you so much power for test design. This tool can give you a great grip on identifying variable conditions…and gives you every possible combination of dependencies. It's very graphical and makes it easy to do the tree structure." Also, he said, it provides pair-wise and triple-wise test combinations.
Rice has also tried and liked Unified TestPro Extended for Manual Testing, a framework for keyword and other types of test designing. Unified Testpro brings automation's benefits – such as scalability and manageability – but doesn't carry the automation startup costs common in keyword test products. Reports are easy to set up, and test time can be reduced with its reusable test case options.
Test execution tools
Running macros is the base function of Macro Scheduler. In Rice's opinion, it "is a cool tool to do basic test automation; but you can do a lot of things with it. It has a really rich scripting language." Its row/column orientation helps in change tracking. It isn't an advanced-usage tool, and Rice advises using a comparison tool with in. Though not a free tool, it's still inexpensive, Rice said, at $110 per individual license and $995 for enterprise license.
Testopia, a test case management extension for Bugzilla, can be used for tracking test cases. With it , testers can integrate bug reporting with their test case run results. Rice likes Testopia's versatility, noting that it can be extended beyond software testing to track testing on "virtually anything in the engineering process."
Speaking of engineering, qaManager helps quality assurance (QA) managers track engineering and customer releases of software projects and resource allocations. This web-based release-tracking tool is powered by OpenXava and Java, Rice said.
Zeta Test is a free test management environment that gets Rice's enthusiastic nod. "It's a full-featured test management and test execution environment," he said.
The Zeta Test environment can be used for black-box and white-box testing, as well as regression and change-management tests. Its user interfaces are easy to use, Rice said, for all phases of testing from planning to documentation to evaluation.
The free XQual Studio (XStudio) is a test and QA lifecycle management application. "It's comprehensive and innovative," said Rice, who praised its graphical and modular design and the flexibility delivered by its organization of data within trees. Its integrated management framework covers every part of QA and testing projects, which simplifies report generation.
Test data and evaluation tools
Rice used the word, "cool," several times when describing Generatedata.com, which delivers test data in text, cvs, SML, SQL and other forms. "It's more of a service than a tool," Rice said. "It's very cool because you can put hundreds of rows of data in so many formats and do in-depth customization."
Rice's favorite test evaluation tool was not designed for that purpose. The free Microsoft Windows Media Encoder is intended to be a video editing tool; but it also captures screen video in the wmv format. With that capability, a tester could capture on video a test being run.
The screen capture feature is just right for situations in which the tester sees something weird going on in software, and the developer doesn't see the same thing. "Don't you hate that? With the video, though, you can demo how you did the test," said Rice. He advises testers not to try to capture long tests and to focus instead on specific tests in short iterations.
Free test tools' value
Rice concluded his review of tools, which included more than those covered in this article, as he began it: stressing the value of free software. Don't sell free software short, he said. "Free and cheap tools allow you to obtain and learn tools on your own," he said. "With test tools, you don't always get what you pay for; so, regardless of the price, do a decent evaluation.