Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

What to expect of a third-party Web application security tester

If you don't have the expertise in-house, you may turn to an outside consultant to do your Web application security testing. Don't go about it haphazardly, though. Kevin Beaver outlines three key things you need to check before you hire a third-party expert or firm.

If you don't have the expertise in-house, you may turn to an outside consultant to do your Web application security testing. Don't go about it haphazardly, though. Kevin Beaver outlines three key things you need to check before you hire a third-party expert or firm.

Kevin Beaver

A significant part of security assessments involves properly managing expectations for everyone involved. What do network administrators expect to take place during the testing process? What do managers of internal audit expect to get in the final report? How should end users expect to be affected by the testing and subsequent changes brought on to tighten things down? And, most important, what does management expect to get out of the assessment that ultimately benefits the business?

Answers to those questions are pretty straightforward, especially when security testing is done internally. But what about when you bring in a third-party expert or firm? Do the rules change? Should expectations be managed differently? The simple answer to both is yes. Whether you're hiring a third party because you don't have the expertise in-house or you're looking to get an independent assessment (like what certain regulations require), you're opening up a significant part of your environment to an outsider.

Sure, you likely hooked up with this third party through a referral or got a good gut feeling based on your initial discussions. For all intents and purposes, though, you're still bringing in an outsider who knows very little about your application, network and business operations. It's also a person or firm you haven't yet built trust with. That's fine -- businesses do it all the time, and it's often the only way get a fresh perspective on your security vulnerabilities. Just be smart about it. You don't want to bring in someone who has won you over through fast talk and fancy marketing materials only to find out that his limited experience and low-end tools may very well bring down your Web site and provide limited value based on the price you're paying for the assessment. Or worse, he isn't looking out for what's in your and your business's best interest.

Here are some hot button areas that have stood out to me over the years being on both sides of the table on this topic.

Depth of testing
A mistake that's often made is the assumption that a penetration test is enough to find every possible vulnerability within a Web application. That's absolutely untrue -- at least how I've seen people scope them in their contracts. A penetration test is not an all-out assessment of a Web application from every possible angle. It's merely what an untrusted outsider with no credentials can do to exploit the application. But what about the other 75% of the application that's available only to authenticated users?

In a lot of cases I've seen a Web security assessment (whether it's called a penetration test or an audit) boils down to just running a vulnerability scanner against the application. If nothing's found with the tool, then everything's assumed to be safe and secure.

That is a big mistake for three reasons:

  1. With the tools we have today, anyone can run an automated scan -- even with many of the kludgy freeware tools available.
  2. Vulnerability scanners do make mistakes.
  3. Easily more than half of Web security weaknesses can be found and exploited only by a human being manually interacting with the application.

The bottom line: Make sure you're getting in-depth testing equal in value to what you're paying for and never assume that just because someone can't get into your application from the outside that it's safe and secure from rogue insiders or an outsider that's obtained valid login credentials. Ask your third-party tester what their typical approach to this is.

Good communication contributes directly to expectations. Be it in the contract, during the testing, or after everything's said and done, you're going to want to make it clear that you expect to know exactly what's going to take place and want to be kept in the loop (via email or telephone) -- especially if any problems arise.

You'll also want to know how they're going to handle their test results during the assessment and once everything is completed. There's often a lot of sensitive material contained in scanner results, screenshots and report documentation that you don't want mishandled.

More information on Web application security testing
Web application security testing checklist

Web application testing: The difference between black, gray and white box testing

Penetration testing versus code review

Look for a solid -- yet fair -- statement of work and contract that lays out all of the details and places equal responsibility on both of your organizations. It may say that problems could arise during testing and there's no guarantee that every vulnerability is going to be found, but that's reality. As long as the third party seems to have done good work for others in the past, promises to provide the best work possible to you, and agrees to keep you in the loop during the testing phase, things should work out fine.

Final deliverables
This is where a lot of people are let down, and it points right back at communication and what was signed off on before the testing started. Will the third party simply hand over the scanner report and say, "Good luck with all that . . ." or does it plan on creating a report that outlines the vulnerabilities that you should focus on and provide some basic guidance on getting started?

Again, a monkey can run an automated scan against a Web application. The true value -- and most of what you're paying for -- comes from the expertise the third party provides in showing you what the results actually mean (via proof-in-the-pudding screenshots, information obtained, or theoretical explanations of what can/will happen), why you should be concerned, and what you should concentrate your efforts on first. The best gauge of what you'll be receiving from the third party in the future is what they've done in the past. Ask for a sample report to check the quality of their work.

Be it testing, communication or final deliverables, ask tough questions and listen until you get reasonable answers. Most important: Get it all in writing. Most of us learn the hard way at some point in our careers that the "gentleman's handshake" of the past is not dependable. That's too bad, but it's reality in business today. If amnesia ever does set in and something's not clear, signed contracts and email records are great things to fall back on.

By focusing on those areas, once you make it through a round or two of testing with a third-party security tester, you'll know whether or not they're a good long-term partner to trust and depend on.

About the author: Kevin Beaver is an independent information security consultant, speaker, and expert witness with Atlanta-based Principle Logic, LLC. He has more than 19 years of experience in IT and specializes in performing information security assessments revolving around IT compliance. Kevin has authored/co-authored six books on information security including Hacking For Dummies and Hacking Wireless Networks For Dummies (Wiley) as well as The Practical Guide to HIPAA Privacy and Security Compliance (Auerbach). He's also the creator of the Security On Wheels series of audiobooks. Kevin can be reached at kbeaver@principlelogic.com.

This was last published in July 2007

Dig Deeper on Software Security Test Best Practices

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.