Manage Learn to apply best practices and optimize your operations.

Defining and preventing buffer overflows

Kurt Seifried describes buffer-over flow attacks and how you can guard against them.

What is a buffer overflow?

Software is typically written so as to manipulate data in some useful fashion. This data may be text, images, video or sound; however, internally -- as far as the program is concerned -- it is simply a string of data, typically in the form of bytes (8 bits of data) that represents some value (e.g. a color or text character). Many popular languages such as C and C++ leave a lot of decisions in the hands of the programmer -- such as buffer sizes, which are explicitly declared to be a certain number of bytes. The result is that it becomes quite easy for a programmer to declare a buffer as a certain size that may not be large enough at a later point in the program, or may accept data that is larger than the buffer itself if input is not properly validated.

This is a problem because when you put more data into a buffer than it is configured to hold, the data often ends up in the executable area of the stack. Often the attacker accomplishes this by overwriting the return address, allowing them to interrupt the flow of the program and run custom code. Even with limited space (less then a hundred bytes of executable content) most attackers can insert enough code to compromise a system. This is especially dangerous because many applications, especially network services, run with elevated privileges in order to access user files and other sensitive operations -- thus the attacker's code runs with the elevated privileges.

In order to avoid buffer overflows programmers must keep buffer sizes in mind when accepting data. All incoming data must be checked for size to ensure it is not too large. Unfortunately, not all buffers are accessible or known; often third-party applications and libraries that interact with the program will contain buffers that do not check incoming data sufficiently, all of which are outside the control of the programmer.

Here are some resources for programming securely:
Building Secure Software: How to Avoid Security Problems the Right Way

Preventing buffer overflows

So now you know what a buffer overflow is, but if you're like me, you don't have time to audit every single line of source code and recompile your applications -- assuming you have access to the source code. Fortunately, there are a variety of software packages and operating system capabilities that can be used to prevent buffer overflows. Even with these packages installed, an attacker will usually still be able to execute a denial of service; most systems that can detect a buffer overflow stop the program from executing any further, effectively killing it. There are three primary approaches to protection: modifying the system kernel (thus protecting all applications), modifying the compiler or source code (thus protecting a single application and requiring recompilation) and adding a middleware layer (thus protecting all applications). Which approach you take will depend on the accessibility of source code and how much you can modify your system.


Windows has no inherent capability to detect and prevent buffer overflows. There is a third party package from SecureWave called SecureStack which will detect many buffer overflows and stop them, the major caveat being that it requires a modern CPU to work effectively and is not cheap, at $249 per system.


Solaris has built-in capabilities to detect and prevent some types of buffer overflows by setting the stack non executable. Keep in mind, however, that some programs rely on the stack being executable, and configuring this option may break these programs. You can detect and log buffer overflows by adding the line:

set noexec_user_stack_log=1

to /etc/system and reboot. This will report buffer overflows. To block them add the line:

set noexec_user_stack=1

to /etc/system and reboot. This will actively prevent many buffer overflows.

OpenBSD on Solaris

OpenBSD has built-in capabilities to detect and prevent some types of buffer overflows. StackGhost on Sparc platforms uses a hardware feature that allows a "cookie," or small piece of data, to be XOR'ed into the return address. It is later removed when the buffer is used, thus making it unpredictable for attackers and breaking many attacks. For more information, please see

Linux -- libsafe

Libsafe acts as an interface to the system and protects a number of vulnerable functions from buffer overflows and other problems such as string format exploits. It is particularly handy as no recompilation of the system kernel or the applications that are to be protected is required. For more information, please see

Linux -- Openwall Kernel Patch

The Linux Openwall kernel patch requires recompilation of your kernel and is only available for older 2.2.x kernels; however, a 2.4.x test patch has been released. It adds a non-executable stack feature -- available only on x86 systems -- that will break many buffer overflows. For more information, please see

Linux -- StackGuard

StackGuard prevents buffers overflows by placing a word at the end of the buffer; if it is altered due to a buffer overflow, then the system knows a problem exists and the program is stopped. StackGuard is an addition to GCC and unfortunately, programs must be recompiled to be protected. For more information, please see

About the author: Kurt Seifried is an Information Security Analyst with interests ranging from Microsoft and UNIX systems to network protocols and encryption (to name but a few). He has written a large number of articles (available online) and maintains many resources on his Website. He was formerly the senior analyst and main writer for SecurityPortal. Visit his site at

This article originally appeared on

Dig Deeper on Topics Archive