Although black box security testing is extremely important to do, researchers at Fortify Software have found it isn't enough to help developers find and repair code flaws.
In a report released Monday about black
Requires Free Membership to View
When you register, you'll receive targeted emails designed to keep you informed of the most relevant information on Agile development, application security, testing & QA, software requirements, and more.
Hannah Smalltree, Editorial Director
box security testing -- sometimes referred to as penetration testing -- Fortify researchers found that such tests failed to provide adequate results in three areas:
Black box tests don't tell you what percentage of the code was hit, said Barmak Meftah, vice president of products and services at Fortify. "Without that parameter, the gauge of security isn't clear," he said.
|  | ||||||||||||||||
| |||||||||||||||||
"While black box security testing is an important tool for analyzing the security of deployed applications, its scope is limited by the fact that it resides outside of the application," Meftah said.
To remedy that, Fortify has created a product to complement black box testing and give developers and testers greater details about test results. Fortify Tracer, whose announcement coincides with the release of this report, sits inside an application and provides "more measurable and actionable output," Meftah added.
For example, Fortify Tracer injects monitors in all of the attack surfaces and around all the functions of the application. Then when a black box test finds issues with an application, Fortify Tracer tells how much of the code was hit and where specifically the problem is.
"Once the issue is found, we can give more information about the cause of the problem because it sits inside the application," Meftah said.
Charles Kolodgy, research director of secure content and threat management products at IDC, reiterated the importance of making applications as secure as possible. The key to application security tools, however, isn't just what they can find but how accurate the tools are at finding real vulnerabilities while minimizing false positives. Following that, it's important to be able to remediate the discovered vulnerabilities, he added.
"This is exactly what Tracer does best in cooperation with the 'black box' testing," Kolodgy said. "It can isolate the exact location of the vulnerability identified by the application scanner in the source code. This should make it easier to be fixed and should also allow people to determine if it is an actual vulnerability."
Fortify Tracer currently works on any J2EE executable (.war/.ear) files. Dashboards communicated key metrics and allow users to compare runs, inspect issues and find flaws. In addition, it generates detailed reports showing vulnerabilities according to their categories, such as cross-site scripting (XSS) and SQL injection.
Meftah said Fortify Tracer will be integrated with Watchfire's AppScan, but the product will also work with any black box security tester, he said.
Available immediately, Fortify Tracer costs $24,000 per named end user.