Conformiq Inc., a provider of automated software test design solutions, recently announced that Conformiq Qtronic can now utilize multi-core, distributed and parallel computing for faster automatic test case generation. Qtronic automates test design by generating black-box functional tests from high-level models expressed in Java and UML-compatible notations, without user intervention.
"These days companies have already solved the test execution problem, and have platforms for executing tests, and use tools like [HP Mercury] QuickTest Pro, Parasoft, etc., or their own platform," said Conformiq CTO Antti Huima. "The big problem is how to get these test cases in the first place; it takes time and is error prone. It's easy for engineers to create the wrong test cases."
Conformiq, which originated in Finland, launched Qtronic in 2007. The company established U.S. headquarters in Saratoga, Calif., in February of this year. Conformiq has received $4.2M in venture capital funding from investors in Europe and the United States, led by Nexit Ventures and Finnish Industry Investment Ltd.
Now with the ability to utilize multi-core and parallel computing, the company said it is paving the way for deploying Qtronic on grid and cloud platforms.
Previously, Huima explained, Qtronic executed mathematical algorithms in a sequential manner on a single CPU or a single core. "So even if you had a multi-core desktop with the previous version you could only use one of the cores," he said. For a big, complex model, that could be time consuming, he explained.
"Now we have implemented a distributed algorithm, so automatically you can deploy all the cores on your desktop." For example, he said, if it took four hours to generate a complex test case on a single-core desktop, the time would drop to one hour with a quad-core processor.
Huima said Conformiq is now exploring the potential use of grid and cloud platforms with customers to speed the generation of test cases even further. The benefits are twofold, he said. "You use less manpower and have test cases ready earlier; and second, it improves communication between the test/dev team. When you see a model you can spot holes in the development specs, so you contact the developers and see what they mean. This seems to happen more often in model-based testing, because it's more difficult to walk around holes in the specs. You need to know what the system is supposed to do [with model-based testing]."
Another new feature of Qtronic is the ability to calculate test case dependencies, Huima said. "You can see that test case 10 depends on test case 3; and you can order in dependency order, so you can speed up test execution when test cases fail." Huima said Qtronic is intended for use by test engineers and test automation engineers, but added that the engineers who create the test automation frameworks also need to be involved because the test cases will run on those systems.
Competitors to Qtronic include Paris-based Smartesting and Microsoft Research, according to Huima. Smartesting is a director competitor with the Smartesting Center Solution, he said. Microsoft Research is doing automated test design research, and "they have some plans to make tools available to the public; when that happens they'll be a competitor."
Rachel Chalmers, research director at The 451 Group, said automated test design is a new area and calls Conformiq "an outlier."
"The bigger picture over the last five to eight years is the rise of test-driven development, which revolutionized the way you turn specs into software, with each spec turned into a test and you write to the test," she said. "This broke down the specs into modular units, which is a powerful notion."
However, she added, "My issue with automated test design is that it seems like there's a lot of [human] intelligence involved in designing tests so they meet the specs. On the other hand, there's an enormous amount of labor in designing tests, especially if you're looking at large-scale grid and complex applications. There may be a role for automated test development under the role of humans."
Taking a parallel from the ops world, Chalmers said automated provisioning tools automate 80% of the boring work, enabling ops personnel to spend their time adding value to the remaining 20% of the work. Automated test design "will be a useful supporting tool in the development process; I don't think it will automate all of the job."
Qtronic, like any new tool, does have a learning curve, Huima said. "If a test engineer has some programming skills they can become productive in one to two weeks; without programming skills it will take longer. The models are basically small computer programs. When teams start to use them they become productive, but they still use them in a straightforward fashion. After a couple of projects they'll start to create more object-oriented models, but they can still get benefits early."
To ensure that the test cases generated are correct and relevant, "those who know how to create models can review each other's models, but more importantly you can generate test cases in human readable format and they can be reviewed by a domain expert who can tell if the models don't make sense."
Huima said testers can also trace the test cases back to the requirements. However, he said, "as you run [the test case generation] you gain confidence, and start to trust it like you trust your compiler."
Conformiq Qtronic runs on top of Eclipse; it is available as a plug-in or standalone desktop product. It is licensed on an annual basis for named users or as floating licenses.