What sort of open source tools are out there for software security testing? How do they compare to proprietary tools?
There are a number of open source tools available that can help with software security testing. Some of the freely-available static analysis tools that can help with security testing include FindBugs and PMD for Java, FxCop for .NET and Brakeman for Ruby on Rails. Freely available dynamic security testing tools include OWASP ZAP, w3af, arachni and skipfish. All of these tools can be successfully employed as a component of a comprehensive software assurance program. Even though these tools do not have licensing fees associated with them, it is important that organizations looking to use these tools understand the differences between these freely available tools and commercial analogs.
Broadly speaking, there are two major differences between most open source security testing tools and comparable proprietary offerings.
First, open source security testing tools often lag behind commercial options in maturity. This relates to both the sophistication of the analysis the tools perform as well as other factors, such as the quality and comprehensiveness of the tools' documentation. For example, few freely available static analysis tools provide the kind of thorough dataflow analysis that can be very useful in finding common vulnerabilities like SQL injection and cross-site scripting (XSS). There are several available that have this capability -- such as Microsoft CAT.NET and Google's CodePro Analytix -- but development and support for these tools has been spotty and their future is uncertain. For the most part, open source security testing tools are targeted toward experts who can deal with the relative complexity involved in configuring and running them and intend to use them as a jumping-off point for further analysis. Commercial tools tend to offer more "wizard-like" capabilities targeted at allowing more junior analysts to get up and running quickly.
Second, open source tools tend to be focused on testing specific applications, whereas commercial tools often have "enterprise editions" that can help with testing across an organization's application portfolio. Although an individual analyst may be interested only in a specific application, enterprises need to be concerned about the security state of their application portfolio as a whole. Unfortunately, most open source tools are focused on single application analysis and have limited, if any, portfolio management capabilities.
In an attempt to give more organizations "enterprise-class" software security capabilities, Denim Group released ThreadFix as an open source software vulnerability aggregation and management tool. ThreadFix lets organizations import and consolidate the output of both commercial and open source scanning tools across their application portfolios and track those scanning results over time. This lets security managers collect data and start looking at their software assurance programs in a quantitative, rather than qualitative, way. So, open source tools are advancing in their maturity and capability, but most specific testing tools still lag behind their commercial equivalents and organizations just need to be cognizant of these limitations so they can plan to compensate for them.
This was first published in August 2013