Should security defects that are detected from scanners or static analysis be tracked under change control?
Requires Free Membership to View
When you register, you'll receive targeted emails designed to keep you informed of the most relevant information on Agile development, application security, testing & QA, software requirements, and more.
Hannah Smalltree, Editorial Director
The high level answer is “it depends.” Some organizations have strictly-defined change control processes which include defect source, defect type, and change size as parameters for consideration in change control. In my personal opinion, I believe the answer is that organizations should only track issues reported in automated tools which meet the “fix” bar.
The key struggle faced by automated code and application scanning tools is the number of “false positives.” These generate noise and require effort in researching, yet they yield no increase in the security for the application -- they are a waste of resources. First generation scanning tools were barely more than advanced grepping tools, which searched code or applications for common strings typically associated with security vulnerabilities. False positive rates of 30%, 40% and even 50% were not unheard of. A Web application with 30,000 lines of code scanned by an automated tool with a 40% false positive rate resulted in a high number of issues -- and if each issue were automatically entered into a change management or defect tracking tool, that tool would be quickly overwhelmed. The sheer volume of incoming issues would exceed the capacity of the engineering or operations team to review.
Second-generation security tools have improved dramatically, cutting the false positive rate sometimes into the single digits. But even then, I would not personally recommend logging every issue as a defect. Many second-generation tools report “information” level issues which document stylistic and usage patterns which have no impact on the security of the application. Also, even with a 5% false positive rate, 30,000 lines of code can still generate more defects than an application team can manage.
The approach I advocate is to store the results of a scanning tool in a separate database and allow the team to review HIGH and MEDIUM issues. Upon investigation, the team can make the decision to open defects in the change management system, on a one-by-one basis. This prevents the team’s change management system from being overwhelmed and allows the team to prioritize and organize their security response based on their specific security development lifecycle and security defect bar.
This was first published in June 2011