There is a lot of concern about the damage that a virus can do to your PC. With the prevalence of Microsoft Windows, this has certainly emerged as the primary platform for virus writers to target. While the anti-virus crowd has managed to stay one step ahead, there are radical virus protection features that are starting to make their way to your desktop.
In particular, automatic detection for buffer overflow conditions is becoming a primary feature. Popular anti-virus software vendor McAfee provides buffer overflow protection as a new feature in version 8.0, and Microsoft is planning to release Data Execution Prevention (DEP) in a future update to MS Windows that will perform a similiar function.
There is a significant problem here, buffer overflows are lurking in virtually every software program ever written for the Windows platform. This includes both commercially distributed and internally developed applications, many of these programs will require updates to function on a DEP enabled MS Windows platform or with the latest release of McAfee.
A buffer overflow results when a program attempts to use more memory then it has allocated. In a typical program there are numerous allocations that take place on an ongoing basis. As long as the overallocation doesn’t step into some critical memory, your program continues to run happily. If, however, an overallocated segment of memory happens to be adjacent to an active program function, then the function will get overwritten by the number of bytes you are overallocating.
Virus writers can exploit buffer overflows to compromise your system. Essentially, if an application can always be found following a specific data memory allocation, then you can send the program more data then it can hold and when this memory is full it will start overwriting the memory representing the program. At this point, you could overwrite the privileged application memory with your own virus and thereby gain elevated system access.
Fortunately, every Intel CPU since the 80286 has supported protect mode operation. When running in protect mode, an application only has visibility to the memory that belongs to that application. So it is not possible for a program running locally to allocate itself into the same memory the operating system kernel is using. This means that the only way for virus to inject themselves are by relying on intrinsic functions of a defective application. For example, the Microsoft JPEG library had a buffer overflow that could be exploited by embedding your virus into an image.
Detecting buffer overflow conditions require sophisticated software that track memory allocations from your program. Not only that, some programs rely on proprietary memory management strategies that would be immune to this detection. One might argue this is not the kind of thing you would want to detect; rather, the defective application should be properly tested by the author of the software to eliminate buffer overflow conditions.
In either case, software vendors are going to be forced to update their code. Applications that are stopped as a result of buffer overflow will require an update in order to cotinue functioning. This technology won’t fix the problem and a lot of software is going to be nonfunctioning because of it.
A more appropriate solution is to adopt new memory management strategies, in particular the automated management facilities native to platforms like Java and the .NET CLR are ideally suited to this task. If computer users are going to be forced to update all of the applications on their system, why not use a new technology that already addresses this issue. Rather than update Microsoft Office to fix memory bounds issues in C/C++, why not install Microsoft Office .NET and rely on the .NET CLR to provide reliable memory functions that don’t require explicit allocation/deallocation.