When it comes to finding and fixing software quality defects and security vulnerabilities, "you can never say you're really done," said Ian Gordon, vice president of product management for Klocwork Inc., a provider of automated defect and vulnerability detection software in Burlington, Mass.
Just ask Jeremy Allison, co-author of the open source Samba software suite. Klocwork recently analyzed Samba using its K7 static analysis tool and identified more than 150 defects -- after the software had been analyzed by another static analysis tool and subsequently fixed.
"That's the wonderful thing about static code analyzers," Allison said. "They find bugs others may miss."
Klocwork analyzed Samba, in addition to two other open source projects, Amanda and XMMS, using the latest version of K7, announced this week. The K7.1 product adds 44 new vulnerability checkers across Java, C, and C++, as well as features ARM Compiler and Java 1.5 support.
Gordon said K7.1 also has improved integration capability with the software build process. "As part of the analysis, we have to understand how the software is built/compiled/assembled. We've done work to simplify that. We don't have to change their build process and put our commands in."
In addition to analyzing code at the system build level, K7.1 allows developers to do analysis at their desktops via plug-ins that work with their integrated development environments. Gordon said K7.1 features plug-ins for the Wind River version of Eclipse and the QNX version of Eclipse, in addition to K7's previous support for standard Eclipse and Visual Studio.
Finally, Gordon said, K7.1 offers scalability through its built-in ability to distribute analysis across multiple machines. "If it took 10 hours to analyze [a build] before, now it's roughly two hours," he said.
Tool detects defects and vulnerabilities
K7 analyzes code for both software defects and security vulnerabilities. "You have to worry about both," Gordon said. "Customers want to find both types of problems -- defects that will cause a crash or have your system exposed."
As part of the proving ground for K7.1, Klocwork analyzed the three open source projects. In Samba's case, Allison said they had previously fixed 270 bugs identified by Coverity as part of a research project by the Department of Homeland Security. None of the bugs were security vulnerabilities, Allison said, but rather "generic, stupid bugs like memory leaks."
Klocwork scanned the software after Samba fixed the 270 bugs and identified additional 150-plus defects, Allison said. Again, there were no security vulnerabilities found. However, Allison said, Klocwork "flagged some interesting anomalies that we looked at really heavily; they turned out not being security flaws, but could've been under different circumstances." He said the two products "appeared to catch slightly different subsets."
Before Klocwork did any public announcements it submitted the defects back to each open source community to allow them to comment and fix them, according to Nick Allen, director of marketing at the company. Samba has been the most responsive so far, he said. The most recent release candidate of Samba includes fixes for the defects Klocwork identified.
"We love it; beat the crap out of us," Allison said. "Find every flaw you can and obviously please tell us about it first, or any potential flaw people think they find. It's massively in our interest to get this as bug free as possible."
Allison said security is built in to the software development life cycle at Samba. "Security is not a separate part of the process; it's built in. Every piece of code checked in is reviewed by at least two people. At some point you have to trust the human beings who write code. There's no magic bullet in security. You have to review code and use what automated tools you can."
Klocwork's goal, according to Gordon, is to make the use of static analyzers part of the developer's everyday life. Allen added, "The market has not yet embraced the static analysis tool to improve quality throughout the software development life cycle. The next big frontier is making sure we're addressing software quality and security problems from end to end. Every developer using static analysis tools is the holy grail."
Still, Gordon acknowledged, it's a difficult issue. Organizations want to find problems, but without taking too much time or degrading performance, he said. "It's a very complicated problem to solve, and I don't think it will ever be solved completely. It's a matter of degree."
The K7.1 Defects plus Security Suite is available for $2,995 per user. The K7.1 Development Suite is available for $3,995 per user.