I don't have a security expert on staff. Does it make sense to rely on a security testing service?
In the absence of having security expertise on staff, you will likely have to pick a provider from outside security testing services. Developing security expertise in-house is too expensive and time-consuming to be a realistic option. This expertise also has to be maintained over time.
Since relying on an external partner is likely the only viable option, the main considerations in choosing a third-party security testing service include:
- Understanding your organization's application attack surface;
- Addressing budget issues;
- Understanding the depth of testing analysis; and
- Deciding on the frequency of analysis.
The application attack surface
First, it is important to understand the scope of what needs to be tested. There are questions a security tester will need to have answers to, including how many applications need to be tested, where they are hosted and who is developing them? Program managers should also be prepared to answer questions about ranking risks. Questions include which applications manage the most sensitive data, which are responsible for the most valuable operations and which represent the greatest risk? This ranking will help prioritize testing activities going forward. Without it, further decisions will likely misallocate resources.
Raw budget dollars may be the critical constraint in a testing program. Especially in a smaller organization without the budget to develop in-house security expertise, budget is probably an overriding concern. Going into an evaluation of outside vendors with a budget scale can help to quickly narrow the field, especially when making decisions about the depth of testing analysis to procure.
Depth of analysis
All assessment and testing activities are not created equal. When evaluating third-party testing services, it is critical to understand specifically what types of analysis will be performed. This determines the level of security insight the assessments will provide.
When evaluating third-party testing services, it is critical to understand specifically what types of analysis will be performed.
Static testing looks at application code or binaries at rest. Dynamic testing examines a running system and performs tests to try and determine behaviors that indicate the existence of application vulnerability. Automated analysis such as static and dynamic testing relies solely on tools trying to match coding patterns or request and response pairings.
Automated analysis is comparatively inexpensive, but it also has limitations. For example, automated testing can only identify certain classes of vulnerabilities and is usually powerless to determine those that depend on the application's business context. In addition to false negatives being introduced because of the limitations of automated analysis, automated security testing can often identifies false positives, where the analysis highlights supposed vulnerabilities that are not actually exploitable.
Manual analysis is comparatively expensive because it relies on security analysts performing tests. This increases the types of vulnerabilities that can be identified, and it is reasonable to expect manual testing to filter out false positives. However, the cost of comprehensive manual analysis can be prohibitive, even for organizations with significant resources.
Frequency of analysis
The security landscape is always changing, and the most important applications are usually under some sort of active development. Security testing is not a one-time activity.
Using outside testing firms or services is a common strategy for organizations both large and small that need to bring on security testing capacity and expertise. However, it is important to engage these firms with a solid understanding of what testing is going to be performed to properly set expectations. Also, having an understanding of the organization's attack surface and an application risk ranking can help ensure that the testing budget is allocated optimally.
Comparing in-house testers and outsourced security testing services.
Dig Deeper on Topics Archive
Related Q&A from Dan Cornell
Is it safe to move from on-premises application lifecycle management tools to cloud-based tools? Read this expert answer to find out. Continue Reading
Can security impact application performance? What security vulnerabilities might be slowing us down? Continue Reading
As our developers incorporate more and more third-party software components and partner APIs that we don't have direct control over, how do we test ... Continue Reading