The testing market is riding a new wave of innovation—and it's been a long time coming, according to Voke inc....
analyst and founder Theresa Lanowitz. In a recent report from the analyst firm on testing platforms, among the drivers of this wave are creative start-ups with innovative technology, virtualization and the cloud, and Microsoft's entrance to the testing market with Visual Studio 2010, along with the convergence of traditional apps and embedded systems.
"Software is always on, and often life-and business-critical. Software is truly driving the business, and the business executive has to promise he can deliver," Lanowitz said. That goes part and parcel with low tolerance for failure, she said, as user expectations are at a new high.
"The threshold for failure in the past was fairly high," she explained. "Now you get a free app on your iPhone and you expect it to work and perform even if it's free. That's why we're seeing performance becoming so important in the market again. You don't expect defects to be interrupting service."
Until recently, however, "the testing market has been in a slump," Lanowitz said. She said there have been only two big innovations since 1999: Rational's Rational Unified Process (RUP), followed by Mercury's Business Technology Optimization (BTO) introduced in 2002.
HP, which subsequently acquired Mercury, is the market share leader of the traditional testing market, according to the report, but Lanowitz said the company's bold move to the cloud is significant.
The cloud is "pushing people along [toward innovation]," she said. "HP putting LoadRunner on the Amazon EC2 cloud was a big strategic move for HP and the industry. HP had all of its products already in a hosted environment, so they were the first ones to take advantage of what's transforming into the cloud. Putting [LoadRunner] in the cloud for everyone is saying, 'No longer is the monetary issue a barrier.'"
HP's announcement this month that it is acquiring Fortify Software, a security testing provider, "is a good move," she said. "They have to respond to the innovation of the smaller vendors. It's also good for the market."
Innovation isn't just coming from the small players, however. Among the larger players, "Microsoft is the one to watch in the testing market," she said. "They will shake things up with Visual Studio Test Professional 2010. This will challenge the market. They have an incredibly large developer base, and if that base can help get test professionals [onboard] and offer value to testers searching for an alternative, they will be a major player for years to come."
Of significance, she said, after several mis-hits with testing going back 10 years, Microsoft is targeting the 80% of testing that is still manual, she said, and bringing innovation to that problem. Of note, she said, are Visual Studio Lab Management, an integrated platform for virtual lab testing and management; the IntelliTrace feature in Visual Studio 2010 for rapid debugging; and Visual Studio Team Explorer Everywhere 2010 (rebranded from the Teamprise acquisition) for collaborating across platforms.
Microsoft, she said, "knew it missed the mark before, so they looked at the testing market and saw it was stagnant, and they created an organic product to satisfy not just .NET users." With Team Explorer Everywhere, teams working in Java and other platforms can collaborate and share common project artifacts.
Going forward, she said, Microsoft has to shore up its marketing to really talk to test professionals and show the value of Visual Studio. "What they done with Team Explorer Everywhere and Lab Management is really show they're serious about providing multiplatform support, and helping Java developers and testers. And virtual lab management is a differentiator."
In the voke report, the analyst firm uses what it calls the Market Mover Array to evaluate testing vendors. The array is plotted against two axes: "Innovation and Technology" and"Marketing Ability." Each vendor rating was charted and placed into one of four bands on voke's Market Mover Array for testing platforms: transformational, pivotal, influential and transitional.
The report rated Coverity as "transformational." San Francisco-based Coverity offers a line of static and dynamic analysis tools for software integrity.
Coverity, she said, "has changed the conversation from technology to business, which is a big pivot point. It's not about the practitioner—the tester sitting in the app lifecycle—or testing through the lifecycle, but about the quality of software you put out as a company and a business. That's where they're taking the message."
Replay Solutions, based in Redwood City, Calif., also received a market mover rating as "transformational." The company offers ReplayDirector, a software record/replay tool. "Replay is using virtualization to solve an age-old problem," she said. ReplayDirector can replicate defects with or without the original environment, using virtualization technology, the report details.
Finally, TOMOS, created by professional services firm RTTS in 2008 and based in New York, offers a lightweight SaaS solution for application lifecycle management and is rated as "pivotal." The report says the company "represents a new breed of testing solutions—those created based on years of professional services experience with larger, heavyweight solutions."
Lanowitz said TOMOS is "an integrated app lifecycle play, but comes from a professional services company, so they have a good eye on testing, and they've figured out the model of what cloud and a lightweight solution can deliver. TOMOS attracts smaller companies that might not be able to afford a larger solution from a larger vendor."
While testing is a big part of the application lifecycle market, Lanowitz said not everyone will embrace or want an ALM solution, and therefore she doesn't believe ALM will consume the testing market. "We will always have a need for testing components.
Moreover, she added, "as we see more [application] complexity and embedded systems being developed, testing will continue to be a really important component."