Software security testing: Finding your inner evildoer

By Joe Basirico 06 Aug 2007 | SearchSoftwareQuality.com

Software quality news and advice Digg This!     StumbleUpon     Del.icio.us

In my experience as both a software security engineer and trainer, it has become clear to me that some students catch on to the concepts faster than others. Often, those who tend to excel don't immediately show signs of brilliance. Frequently, a seasoned tester who can hunt down functional bugs in the weirdest of places can't make the transition to security testing very easily. In the past few years, having trained development teams from some of the world's largest software vendors, I've examined their behavior and attempted to distill what makes a great security tester. I've come up with three pillars that every security testing foundation must have:

The most important pillar of expertise for a great security tester is being able to understand how the system can fail and to think maliciously once you've got your foot in the door. Joe Basirico Security Innovation Inc.

Imagination -- Many times we don't have all the information we'd like to have as security testers. When exploiting an SQL injection vulnerability, for instance, the security tester has to make certain leaps of faith about the underlying system and make educated guesses about what is really going on to create an effective test.

Complete knowledge of the system -- A great security tester must know about each component of the system he is testing. For Web applications that often means in-depth knowledge of JavaScript, XML, server-side code (ASP, JSP, Ruby, PHP, etc.), databases, Web services and more. The tester must be able to recognize when things are out of place and when components may be used incorrectly. This complete knowledge comes with time and expertise, but it can be aided by intense research of each subject with a security focus in mind.

Evil streak -- The previous two pillars of expertise will take a security tester only so far in his quest for security testing nirvana; the pillar that is a game-changer is the ability to think like an attacker. Being able to anticipate the way an attacker will visualize the system is an integral part of testing the system. Similar to mapping out the many ways a burglar might be able to break into your house, the same thought process is needed for security testing so that you cover all the creative ways an attacker could exploit your application.

In this article I'll discuss each of those pillars in more depth and provide a few tips on how to become an expert in each field.

A great security tester has a great imagination
A great imagination extends beyond the ability to imagine a system as it could be. It also includes the ability to envision the truly interesting bugs and vulnerabilities in a system. Most security assessments are performed black box -- without source, documentation or access to internal systems. When a security tester approaches a security assessment with little information he must make certain assumptions and inferences about the system he is testing. Sometimes those can be verified later through focused testing, but often they cannot.

Take for example an Ajax-enabled Web application. Things that are immediately obvious may be the use of JavaScript, the server language or the use of SSL. The tester must then imagine what the server topology looks like and ask himself a few questions:

• Are they using Microsoft SQL server or MySQL, or are they storing everything in an XML datastore?
  • If they're using SQL, are they using Stored Procedures or Parameterized Queries?
• Where/What is the input checking being done?
  • On the client in JavaScript?
  • On the server?
  • Did they write their own input validation routines or are they relying on controls provided by the language?

There are many other questions that a seasoned security tester will ask to better understand what is going on behind the scenes in this software system. Once the architecture and topology has been discovered, it's time to start discovering how secure the system really is.

SQL injection is an exceptional example of a vulnerability that requires a creative imagination to be discovered. For these vulnerabilities, a tester must be able to envision how certain features in the Web application would be executed on the database.

More information on security testing software Application security defense in depth: Strategies to lock down your Web apps How to attack (test) software yourself How things break: Securing your software

Take for example a login control that requests a username and a password from the user and uses that information to query the database to locate and log in the user. What would the SQL look like that would perform this function? After constructing a hypothesis of the query, the security tester can verify the guess against the live system by trying certain inputs in the username and password fields. If the test passes and a vulnerability is found, the tester may choose to do further testing to leverage this vulnerability against the system to discover vulnerabilities deep within the system.

In this case the security tester is presented with an SQL statement that queries the database and returns some information. This is an exciting statement because it means that a well-crafted SQL injection string may allow the tester to arbitrarily read from the database. An improperly secured database may relinquish all control to the user at this point by allowing multiple SELECT, JOIN, INSERT, DROP or EXEC statements to be executed in succession.

Without the ability to imagine the SQL statement or the backend architecture of the system, this attack may not be possible. Even if a less imaginative tester were to stumble across this vulnerability, he may miss other vulnerabilities with higher risk.

Great security testers have complete knowledge of a system
The most common Web application vulnerability by far is cross-site scripting (XSS). At Security Innovation our engineers maintain a knowledgebase of all security vulnerabilities we have found over the years of security testing. More than 85% of vulnerabilities found in Web applications are due to XSS. Often they are so ubiquitous that after finding dozens of them, we actually stop looking and instead provide guidance to our customer's development team so they can fix them and we can focus our testing efforts on more mission-critical issues.

For that reason XSS is a great example for this subject. Ideally the system is protected by defense in depth. Initially any user input should be checked in the Web browser, then validated on the server using a whitelist regular expression. Finally, when that data is displayed back to the user, it should be whitelist-encoded to make sure no errant characters slip by and are executed on the client's browser.

Let's examine each of the stages of defense in depth individually and discuss what should be done at each stage.

The first step is to provide the user immediate feedback on the input provided. This is done by checking input in JavaScript in the browser. A good security tester must be able to recognize where error messages originated and when they can be easily bypassed. JavaScript provides no mechanisms for security, since it resides in the browser and can be easily turned off. It does, however, provide a great mechanism to give the user specific feedback as to what quality of input is expected on a form. This makes users happier, speeds up the form completion process by not requiring a complete post back to the server, and relieves server workload by performing checks on the data before being passed along to the server.

As I mentioned earlier, client-side input checking is easily circumvented and cannot be used for a security defense mechanism. JavaScript provides no security benefit. For that reason, every input must be checked as thoroughly or more thoroughly on the server.

In this stage the security tester must know how to circumvent the client-side input-checking mechanisms to exercise each server-side check individually. The security tester must also be able to recognize what layer this error message is coming from. Is this error being returned because of an input validation routine or is it trickling all the way into the system and being stopped by some internal component as it is executed? If the security tester is able to bypass the input validation routines, he must finally understand what type of attack to apply in this case.

Often XSS vulnerabilities are the result of improperly encoded tags or attributes, so closing a string or tag may be necessary before beginning the exploit JavaScript. Alternately, this input string may be loaded into a JavaScript block directly, so opening another JavaScript tag would break the system.

Finding your inner evildoer
The final and, in my opinion, the most important pillar of expertise for a great security tester is being able to understand how the system can fail and to think maliciously once you've got your foot in the door. The moment a potential vulnerability is discovered it must be assessed for risk. The most common risk rating system is DREAD, which stands for Discoverability, Reproducibility, Exploitability, Affected users and Damage potential. A tester with a healthy understanding of the latest exploits and a bit of an evil streak may be able to persuade developers and managers to escalate the vulnerability to a higher risk rating and increase the likelihood of getting it fixed quickly.

Thinking like an evildoer also means keeping abreast of the current security issues in the area that may affect your software system. There are dozens of helpful mailing lists, Web sites and conferences that are great resources for this information. Security testers will find a community to learn and discuss the latest security vulnerabilities and exploits. This gives testers advanced knowledge of the kinds of vulnerabilities that are now common and surfacing in similar types of applications. Additionally, if the software system relies on other systems, it's a good idea to search through the mailing lists for those systems to look for vulnerabilities that may be inherited from less secure components.

Conferences offer a great way to gain expertise in the field and learn the latest techniques for security testing. Some great security testing conferences include RSA, Blackhat and the Software Security Summit, which each occur annually.

Summary
Security testers have one of the most exciting and creative jobs in the industry. They are tasked with finding elusive security bugs in complex software systems and convincing the rest of the team of their importance. They must prioritize their time and efforts to make sure the best (or worst) security vulnerabilities are found and fixed.

To do that, the security tester should have the best resources available: access to external classes, conferences, magazines and books. The security tester has to find as many critical security bugs with limited resources before the major ship deadline; the attacker has to find only one – and has all the time in the world after ship to do so. Is it a level playing field? No. That is what makes the tester's job so exciting, critical and challenging.

Every great security tester has these three qualities: a great imagination, complete knowledge of the system they are testing and an evil streak so they can think like an attacker -- and beat him at his own game. By mastering those three pillars of expertise a tester will be well on his way to becoming an exceptional security tester.

-----------------------------------------
About the author: Joe Basirico is a senior security trainer at Security Innovation Inc. in Wilmington, Mass.

Tags: Software security testing and techniques,  VIEW ALL TAGS

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Software security testing and techniques Web application security best practices: Tips on implementation Testing strategies for complex environments How to make your software tamperproof Ways to approach application performance testing on a tight budget How can I tell if my software security has been breached? Is online application testing for smartphones different from other software testing? Software testers facing six big challenges today, StarWest keynoter says Lesser-known free software testing tools testers should try Is manually testing a software project for flaws too risky? Affordable automated testing tools for securing websites

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary