Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Change management: Tracking security defects found with scanning tools

Expert John Overbaugh explains how testing teams can maintain change management system efficiency.

Should security defects that are detected from scanners or static analysis be tracked under change control?

The high level answer is “it depends.” Some organizations have strictly-defined change control processes which include defect source, defect type, and change size as parameters for consideration in change control. In my personal opinion, I believe the answer is that organizations should only track issues reported in automated tools which meet the “fix” bar.

The key struggle faced by automated code and application scanning tools is the number of “false positives.” These generate noise and require effort in researching, yet they yield no increase in the security for the application -- they are a waste of resources. First generation scanning tools were barely more than advanced grepping tools, which searched code or applications for common strings typically associated with security vulnerabilities. False positive rates of 30%, 40% and even 50% were not unheard of. A Web application with 30,000 lines of code scanned by an automated tool with a 40% false positive rate resulted in a high number of issues -- and if each issue were automatically entered into a change management or defect tracking tool, that tool would be quickly overwhelmed. The sheer volume of incoming issues would exceed the capacity of the engineering or operations team to review.

Second-generation security tools have improved dramatically, cutting the false positive rate sometimes into the single digits. But even then, I would not personally recommend logging every issue as a defect. Many second-generation tools report “information” level issues which document stylistic and usage patterns which have no impact on the security of the application. Also, even with a 5% false positive rate, 30,000 lines of code can still generate more defects than an application team can manage.

The approach I advocate is to store the results of a scanning tool in a separate database and allow the team to review HIGH and MEDIUM issues. Upon investigation, the team can make the decision to open defects in the change management system, on a one-by-one basis. This prevents the team’s change management system from being overwhelmed and allows the team to prioritize and organize their security response based on their specific security development lifecycle and security defect bar.

Dig Deeper on Topics Archive

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.