DEP to keep buffer overrun at bay

Since the service pack 2 (august 2004), Windows XP supports the hardware DEP (Data Execution Prevention). This feature is hardwired in the modern CPUs, it allows to intercept and lock programs before buffer overruns lead to issues. It requires a compatible BIOS, CPU and operating system.

By default the DEP is only applied only to the Windows core programs and services. The main reason behind this default behavior is that back in august 2004 a bunch of legit apps were locked by hardware DEP (Adobe Reader 6, Jasc Paint Shop Pro 8, Pinnacle Systems Studio 9,...). These programs were patched out of the bugs / performance tricks to be hardware DEP compliant.

If your gear supports hardware DEP, it's highly recommended to activate it. To do so, you'll need:

  • Open the system applet in the control panel.
  • Select the advanced tab then click on parameters in the Performance section.
  • Select the Data Execution Prevention tab.
  • Select the Turn on DEP for all programs and services except those I select.
  • Click on Ok then reboot to apply the changes.

DEP options:

070208_dep_us.png

From now on your PC is protected against most buffers overruns. If a program tries to execute code in a forbidden area, the following window will appear.

DEP warning:

070208_dep_us_crash.png

In this example, Windows Explorer is not behaving properly... If a legit program is stopped by DEP, you still have the option to add it to the Opt-out list. But you'll have to understand that disabling DEP leaves a breach in the security wall created by DEP.
 
In case you don't know if your PC supports hardware DEP, Steve Gibson from grc.com made a program called Securable. Securable checks your system for the following security features:

  • Hardware DEP: for Windows XP SP2 32/64 bits and up.
  • 64 bits: for the Kernel Patch Protection of Windows Vista 64 bits.
  • Virtualization: for the future HyperVisor of Windows Vista 64 bits.

Securable ran on my laptop:

070208_dep_securable.png

Additional informations: for the advanced users
The DEP mode is a parameter of the operating system in the boot.ini file.

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\\WINDOWS
operating systems]
multi(0)disk(0)rdisk(0)partition(1)\\WINDOWS=\"Microsoft Windows XP Professionnel\" /fastdetect /NoExecute=OptOut

The possible values for the NoExecute parameter are:

  • ALWAYSOFF turns DEP off.
  • OPTIN activates DEP only for the main Windows programs and services and those added to the list. (Windows XP SP2 default mode)
  • OPTOUT activates DEP for all the programs et services except those added to the list. (best mode available from the DEP options window)
  • ALWAYSON activates DEP for all programs and services. (only available by manually editing the boot.ini file)

Remark: If a bug in a critical driver triggers a DEP lock in OPTOUT or ALWAYSON modes it might prevent your PC from booting. In that case you'll have to access the recovery console and edit the boot.ini file to switch to OPTIN mode.