This content is part of the Essential Guide: Change agents: Leaders in enterprise application architecture
Manage Learn to apply best practices and optimize your operations.

Security test researcher funded by US Department of Homeland Security

The U.S. DHS is funding Dan Cornell's efforts to improve hybrid analysis of results of various different security testing tools.

Dan CornellDan Cornell

Application security expert Dan Cornell sees two problems in the software quality industry rising above the others. One problem is a disconnect between information security officers, software quality assurance professionals and application developers. The other is a lack of integration between various security testing tools. More than anything else, Cornell says, these two factors keep application developers from seeing and fixing major security vulnerabilities. That's why Cornell started ThreadFix, an open source software vulnerability aggregation and management system.

Cornell splits his efforts as principal at the Denim Group -- a software security firm based in San Antonio, Texas - consulting with customers and researching and developing new application security tools and methods. Cornell's research on security testing tool integration recently won Denim Group funding from the U.S. Department of Homeland Security (DHS) to research hybrid analysis mapping (HAM) techniques.

The HAM project

The hybrid analysis mapping (HAM) project's most important purpose is to assimilate different forms of security testing, according to Kevin Greene, a program manager in the DHS Science and Technology Directorate, Cyber Security Division. Application security testing includes independent assessment activities, such as penetration testing, static code analysis and dynamic code analysis. "Let's marry these all together into one risk management framework that presents vulnerabilities in a consistent and meaningful way," Greene said.

Research shows the best any one tool can do on its own is to find about 14% to 17% of the weaknesses and vulnerabilities in software systems and applications. One focus of this project, for Greene, is to merge well-established commercial security tools and emerging open source alternatives that may not have attained the breadth and depth of commercial security suites. "While open source tools may not have the same value as commercial, they certainly do have value," Greene said. "Let's see if we can leverage the strength of each tool to improve the accuracy of results."

Hybrid analysis is no trivial matter

A major challenge for the researchers working on HAM, according to Greene, is that each tool expresses weaknesses in a different way. They have different outputs. Depending on the tool, vulnerability data may output to .CSV, text, .xml or possibly other file types. "What normalization points will work to correlate this unstructured data from these tools?" asked Greene. "If we can get that, then we can get some real automation in the threat management process; then we'll be able to test more religiously with better relational context of vulnerabilities."

Finding vulnerabilities is easy; the challenge is fixing those vulnerabilities.

Dan Cornell

The normalization tools for security testing are somewhat limited at the moment, according to Cornell. Different testing tools cater to different problems and therefore present test results in different ways. For instance, Cornell said the way static code analysis tools work is different from the way dynamic code analysis tools work. Right now, normalization tools can accurately match several different static code tools together, or they could accurately match several different dynamic tools together.

The current tools would run into trouble, though, matching one static tool with one dynamic tool and providing results that are both accurate and relevant. Cornell said that the ThreadFix project is already "doing crude approximations of hybrid analysis," and that he is excited that "now we have the opportunity to find very sophisticated ways to do it."

If successful, the HAM project will be added to the DHS software assurance marketplace (SWAMP). "The SWAMP will be a great research infrastructure for improving our software development capabilities," Greene said. "We're doing very exciting things with software assurance in the SWAMP."

Phase I of the HAM project includes funding for the Denim Group as well as two other performers -- North Port, N.Y.-based Secure Decisions and Herndon, Va.-based Data Access Technologies Inc. Greene expects Phase I to wrap up in late September or early October. From there, one of the three performers that are working on Phase I will move on to Phase II and, hopefully, Phase III.

Phase I should see HAM research taken to technology readiness level (TRL) where the concept proves feasible and shows value. Phase II would give researchers two years to take the technology to a more mature TRL (TRL 4 to TRL 6), where it can be effectively demonstrated in an operational environment. Phase III would be to get the technology beyond TRL 7, where the systems are launched and in use.

Focus on improvement

"Finding vulnerabilities is easy," Cornell said. "The challenge is fixing those vulnerabilities." While many security testing pros -- including ethical hackers, penetration testers and static or dynamic code analyzers -- can discover a wide array of potential weaknesses, relatively few can actually fix them. The fixing part is ideally done by the developers who write the code to start with, according to Cornell. In most organizations he's worked with, the process breaks down at the point where security professionals hand the vulnerabilities over to developers.

At this point, one of two things usually happens. Either the developers don't understand the vulnerabilities or they don't know how to work those vulnerabilities into their normal workflow. Unlike other software defects, security vulnerabilities are usually tracked and managed by a single information security officer using their own ad hoc tools -- usually spreadsheets and PDFs. "Vulnerabilities have to get turned into software defects in order to get proper attention from developers," Cornell said.

An introduction to ThreadFix

ThreadFix is an open source software vulnerability aggregation and management system. It takes the results of several different security tests, normalizes and deduplicates the data, and then creates software defect reports that can be integrated and prioritized right along all the other software defect reports that the development team is already working on. Assuming the defect tracker works well for development and QA, Threadfix can potentially ease security requirements by integrating them into the existing software quality processes.

ThreadFix is used by security testers who run a series of individual tests, upload results from several different scans and rely on the open source software to pull it all into the same place. "A cross-site scripting vulnerability might be found twice, by two different tools," Cornell said, "but that's still really the same vulnerability." He said that ThreadFix will naturally overlap those two and present only one defect to the developers. Cornell said that several instances of a single vulnerability can also be batched together into a single defect where appropriate.

Threadfix is a REST-based API that can be accessed via a command line client. This allows developers using continuous integration or a DevOps approach to implement Threadfix programmatically, according to Cornell. He said it is possible to integrate Threadfix with automated testing that runs at every build. Automated security tests could be run, the results automatically fed into Threadfix and security defect reports would be produced.

The focus of Threadfix is to be a tool that can consistently integrate all the latest and greatest security testing tools. Threadfix does not automate the security testing process, however, nor does it run any security tests on its own. It aggregates vulnerability information from existing testing tools and produces actionable defect reports based on the results. Cornell said Threadfix could be packaged with other open source software testing tools, but building those tools directly into Threadfix would be counterproductive. "There are already a lot of great tools out there, and more are being built every day," he said. "For us to try and build them into Threadfix would be reinventing the wheel."

Kevin GreeneKevin Greene

Kevin Greene is a program manager at the Department of Homeland Security (DHS) Science and Technology Division. He has over fifteen years of experience in the field of information assurance and application security, writing and consulting for federal agencies and with private organizations working toward attaining and maintaining government security regulation compliance. He is also a major player in DHS efforts to fund successful software assurance research.


Do you have an interesting software quality project that you're leading or working on? Would you like to share it with the world? Let us know, and next time you may be reading about yourself on

Dig Deeper on Topics Archive

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.