Manage Learn to apply best practices and optimize your operations.

Tips to locate reliable security testing services

Developers without a security expert often rely on a third-party testing service. Software expert Dan Cornell provides some tips for the search.

I don't have a security expert on staff. Does it make sense to rely on a security testing service?

In the absence of having security expertise on staff, you will likely have to pick a provider from outside security testing services. Developing security expertise in-house is too expensive and time-consuming to be a realistic option. This expertise also has to be maintained over time.

Since relying on an external partner is likely the only viable option, the main considerations in choosing a third-party security testing service include:

  • Understanding your organization's application attack surface;
  • Addressing budget issues;
  • Understanding the depth of testing analysis; and
  • Deciding on the frequency of analysis.

The application attack surface

First, it is important to understand the scope of what needs to be tested. There are questions a security tester will need to have answers to, including how many applications need to be tested, where they are hosted and who is developing them? Program managers should also be prepared to answer questions about ranking risks. Questions include which applications manage the most sensitive data, which are responsible for the most valuable operations and which represent the greatest risk? This ranking will help prioritize testing activities going forward. Without it, further decisions will likely misallocate resources.


Raw budget dollars may be the critical constraint in a testing program. Especially in a smaller organization without the budget to develop in-house security expertise, budget is probably an overriding concern. Going into an evaluation of outside vendors with a budget scale can help to quickly narrow the field, especially when making decisions about the depth of testing analysis to procure.

Depth of analysis

All assessment and testing activities are not created equal. When evaluating third-party testing services, it is critical to understand specifically what types of analysis will be performed. This determines the level of security insight the assessments will provide.

When evaluating third-party testing services, it is critical to understand specifically what types of analysis will be performed.

Static testing looks at application code or binaries at rest. Dynamic testing examines a running system and performs tests to try and determine behaviors that indicate the existence of application vulnerability. Automated analysis such as static and dynamic testing relies solely on tools trying to match coding patterns or request and response pairings.

Automated analysis is comparatively inexpensive, but it also has limitations. For example, automated testing can only identify certain classes of vulnerabilities and is usually powerless to determine those that depend on the application's business context. In addition to false negatives being introduced because of the limitations of automated analysis, automated security testing can often identifies false positives, where the analysis highlights supposed vulnerabilities that are not actually exploitable.

Manual analysis is comparatively expensive because it relies on security analysts performing tests. This increases the types of vulnerabilities that can be identified, and it is reasonable to expect manual testing to filter out false positives. However, the cost of comprehensive manual analysis can be prohibitive, even for organizations with significant resources.

Frequency of analysis

The security landscape is always changing, and the most important applications are usually under some sort of active development. Security testing is not a one-time activity.

Using outside testing firms or services is a common strategy for organizations both large and small that need to bring on security testing capacity and expertise. However, it is important to engage these firms with a solid understanding of what testing is going to be performed to properly set expectations. Also, having an understanding of the organization's attack surface and an application risk ranking can help ensure that the testing budget is allocated optimally.

Next Steps

Comparing in-house testers and outsourced security testing services.

Dig Deeper on Topics Archive

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Which consideration is the most important for choosing a security testing service?
Having provided penetration testing services to client companies, I think one of the things to consider is the current level of vulnerability in the application. If the outside security testing partner manages to find several critical issues within the first few hours of testing, then the application's security measures should be thoroughly revised anyway. In the mean time, the testing partner can either look for more low hanging fruit or risk conducting an investigation into code that will get revised anyway. This seems like a waste of resources.

I would recommend that the client company should at least attempt to conduct some form of security analysis on their own - especially in situations where cost is an issue. Automated tools can only find a small part of the possible issues, but they can be a start. Manually inspecting relevant fields and requests for Cross-Site Scripting and Cross-Site Request Forgery issues can reveal several major problems. Once the low hanging fruit has been explored, it might be time to call in the experts.