Rootkits

A "Rootkit" is a software program that is designed to install itself in a target host Operating System and gain control of it as the "root" or "super user" of the system.

Once the root module is installed, it is capable of modifying portions of the Operating System's kernels data structures and code to achieve "stealth" or invisibility by causing the system to behave differently than it was designed. The intuitive byproduct of this is that the Rootkit cause the underlying operating system to behave differently than dependent anti-virus and anti-Spyware applications will expect. For example, a Rootkit can alter the behavior of a file-search system-call by causing the function to ignore specifically infected files. An anti-virus program that depends on the operating system API would use the altered system call during its analysis of the file system. With the system call altered to filter out the infected files, the anti-virus program will never know that the infected files exist! Hence, a Rootkit makes the Operating System untrustworthy and unreliable.

Rootkits are deployed primarily for purposes of eavesdropping and remote control of the targeted computer system. For example, a Rootkit could be deployed to "sniff" network packets, record user keystrokes (steal passwords, credit card numbers, etc), read emails, or access the system shell.

Rootkits do not depend on software bugs (also known as "exploits") being present in a system. Typically, a computer attacker will use an exploit, such as a buffer overflow bug to load the Rootkit code into the Operating System kernel. However - a Rootkit, in and of itself, does not depend on software exploits in order to function as designed once it is installed.

A Rootkit is also not synonymous with a computer virus as it does not reproduce and spread or "self-propagate" itself under its own control. The purpose of a Rootkit is to install itself and maintain a low profile; not to spread itself virulently and draw attention to itself.

While Rootkits are not computer viruses or exploitations of system software bugs, they can be (and are frequently) incorporated by virus authors and attackers to produce highly effective and destructive assaults on targeted computer systems which makes their detection and removal an actively researched technology by modern anti-virus and anti-Spyware application vendors.

Rootkits are the most dangerous threat as they can disable all detection methods and become totally hidden. They are stealthy and non-destructive threats that are intended to provide an attacker with a backdoor for ongoing remote access to the operating system. Normal anti-virus and anti-spyware software tools available in market cannot detect rootkits. We need specialistic tools for detecting and cleaning rootkits. Removal of rootkits is a widely debated topic because elimination of these malware programs, which themselves reside on the operating system may leave it unstable and non-functioning. Therefore, removing Rootkits is an important focus of study which presents two separate problems: the removal of the Rootkit itself and removal of the malware that the Rootkit was “stealthing”.

Rootkits can be implemented for various operating systems including Windows, Linux, FreeBSD, etc. Due to time constraints of the project, our study of Rootkit implementation was restricted to Windows Operating Systems due to Windows being the most commonly known software platform by the general public. Rootkits can be implemented in various ways, including kernel Rootkits which modify Kernel data structures as well as user space Rootkits which modify the memory bytes of a target process on the system. Rootkits typically modify the operating system by installing function "hooks" which modify the results returned from system functions so that applications running in user-space receive false results. Kernel level Rootkits are typically much more powerful than user space Rootkits as kernel level Rootkits are able to access global kernel structures and consequently modify the behavior of all processes running on the system. User space Rootkits are restricted to modifying the behavior of only a single process on the system. Kernel level Rootkits also run in privileged mode which also gives them the upper hand on evading Rootkit detection software which typically runs in user mode.

The Windows operating system architecture follows a hybrid micro-kernel/monolothic-kernel architecture where the system is partitioned into "rings".

A ring designates a privilege level which is associated with each of the software components that is running on the system, including the operating system itself. Windows operating systems partition all of the running processes into two rings: ring 0 (kernel processes) and ring 3 (user processes). Processes that are running in ring 0 are said to be executing in "kernel mode" whereas user processes running in ring 3 are executing in "user mode". The Windows kernel and system software (such as device drivers) execute in kernel mode whereas normal applications execute in user-mode. There is an implied level of privilege which corresponds to each of the rings. Processes executing in kernel mode are allowed to access memory associated with user-mode processes. Processes executing in user-mode are not allowed to access memory associated with the kernel.

The Windows operating system implements virtual memory management. A distinction is drawn between a "physical address" (an actual address in physical memory) and a "virtual address" (an address that only has meaning within a running process). Windows processes always use virtual addresses which means that they do not explicitly read from or write to "real" physical addresses. Instead, the windows kernel prepares a special table called a "page table" for each process that defines the mapping of a processes virtual addresses to their corresponding physical addresses in main memory. Modern Central Processing Units (CPU) provide a Memory Management Unit (MMU) which uses the kernel supplied page table to provide automatic address translation of addresses without having to preempt the processor whenever a virtual address needs to be translated into a physical address.

In 32-bit editions of Windows, the representation of a memory address is restricted to 32 bits. Therefore, an address can range from 0 to 4,294,967,294 (also called 4 gigabytes or 4GB). Every program or "process" running on Windows operating systems is given its own private 4GB virtual address space. The reason that each process is said to have its own "virtual" address space is because the running processes are not actually accessing the same locations in physical memory even though it appears that they both have the same address space. For example, suppose program A and program B attempt to read the content of memory at address 0xCAFEBABE. Each program supplies the address 0xCAFEBABE to the MMU which translates 0xCAFEBABE into the actual physical address in Random Access Memory (RAM) which is then used for the actual memory read. When the MMU translates a virtual address into a physical address, it does so from the context of the program that is trying to perform the memory access via a data structure called a "page table" which maps virtual addresses into physical addresses. Different processes have their own unique page table with different mappings of addresses. Therefore, if Program A and Program B both attempt to access memory location 0xCAFEBABE, they will both be accessing different areas of actual memory because the MMU could translate A's 0xCAFEBABE into 0xFEEDFACE and then translate B's 0xCAFEBABE into 0xDEADBEEF, assuming that Program A's page table contains a mapping of 0xCAFEBABE to physical address 0xFEEDFACE and Program B's page table contains a mapping of 0xCAFEBABE to physical address 0xDEADBEEF.

Virtual memory allows the operating system to protect itself as well as its applications from poorly behaved processes. All processes have their own virtual address space; as far as a single process is concerned - it is THE only process running on the system. Since different processes cannot physically address each other's memory, one process cannot overwrite (corrupt) the memory of another process. In order to protect kernel mode memory from user mode processes, the page table also defines privilege levels associated with an address. If a user-mode process attempts to access memory that is designated as reserved for the kernel then the MMU will raise an exception and the offending process will be terminated.

While all Windows processes are initialized with their own page table and distinct physical address mappings, the upper region of every process's address space is mapped into the same region of physical memory designated as the kernel space. For all processes, the upper gigabyte of memory is always mapped to the kernel area which contains the data structures maintained by the kernel. Therefore, if program A and program B both attempt to access virtual address 0xC0000000 then both programs will access the same location in physical memory because it is in the kernel region and the page tables for all processes map the kernel region to the same physical region in memory. In other words, all processes share the same physical region of kernel memory. However, since this area of memory is reserved for the kernel, user mode processes cannot access it. The upper gigabyte can only be accessed on behalf of the kernel. For example, if a hardware interrupt is raised while user-process A is running, the CPU will transition into kernel mode and jump to some interrupt handler. While the interrupt handler is running, it will be addressing kernel memory using the upper gigabyte of process A's address space. The kernel will use process A's page table to translate virtual addresses within the kernel region. Once the handler finishes, the CPU will transition back into user mode and will return control to where process A was when it was interrupted.

The goal of the Rootkit is to control the behavior of the operating system. The way that the Rootkit alters the behavior of the operating system is by modifying the data structures that the operating system uses to perform its workload. Since the operating system data structures are located in memory regions that are protected from access by user-mode processes, the Rootkit needs to be installed in the kernel so that when it runs, it will be in kernel mode rather than user mode. Once the Rootkit is installed and running in kernel mode, it has access to all of the objects maintained by the operating system.

While running in kernel mode gives more power of control as well as global control over every process, a Rootkit can also be implemented in user-space as well where it only affects a single process. Most Windows processes map common system Dynamic Link Libraries (DLLs) into their address spaces in order to call functions provided by the Windows Application Programmer Interface (API). Examples of libraries that are mapped into most Windows processes are Kernel32.dll, User32.dll, Gui32.dll, and Advapi.dll. Application programs import the functions contained in the DLL's into their own address space which they can then call to perform system specific services. A Rootkit can be implemented as a user-space DLL and then be remotely "injected" into a victim process. Once the Rootkit DLL is mapped into the victim process, it overwrites the locations of the imported Windows DLL functions with its own "hook" functions. From that point on, whenever the victim process calls one of the imported Windows functions, it will actually be calling the functions installed by the Rootkit which will typically change the results of the original function so that the victim process receives different results.

Based on the type of execution, rootkits may be classified as kernel mode and user mode rootkits.

In order to run within the context of the kernel, the Rootkit is developed as a "kernel module" (also known as a "device driver"). Microsoft provides a Device Driver Toolkit (DDK) which contains a compiler and linker for third party Original Equipment Manufacturers (OEMs) to build drivers for the Windows operating system. A device driver does not have to have an associated piece of hardware installed on the system. A Rootkit can implement the device driver API and then be installed into the operating system.

A device driver can be installed by a program that is running in user mode. The windows API provides functions that user mode programs can call to register a driver and start it. Therefore, a Rootkit can be compiled and linked as a device driver and then installed on the target machine using a program (like a virus) that runs in user-mode.

Device drivers must implement certain "callback" functions that operating system will call when special events happen. The Rootkit simply implements these callback routines which can assume that when they are called, they will be executing in kernel mode and will have access to the kernel address space.

A user space Rootkit runs within the context of a single user process. As opposed to kernel rootkits which are considered 'global' as they modify the behavior of all processes running on the system, a user space Rootkit runs within the context of only one process on the system. In order to "infect" a process, the Rootkit's code needs to be imported (also called "injected") into the victim's address space. Once the code is in the address space of the victim process, the Rootkit needs to have its initialization routine called which will re-direct the imported Windows routines to the Rootkit's detour functions.

User level programs, which have less privilege of access cannot directly access the kernel space. The API request kernel for system resources indirectly by using DLLs.The user space Rootkit is implemented as a Dynamic Link Library (DLL) that is "injected" into a victim process. The Rootkit DLL will implement replacement (also called "hook") functions that it intends to cause the process to be diverted to when calls are made to functions exported by the Windows API system DLLs. The first step is to force the victim process to load the DLL that contains the Rootkit code into its address space; this is called DLL Injection. A popular method is to use a function exported by the Windows API called CreateRemoteThread. The CreateRemoteThread function takes a process identifier (PID) as well as the address of a function to be executed within the target process as the thread's entry point as well as a parameter to pass to the function. The Rootkit will call CreateRemoteThread and pass it the address of another Windows API function called LoadLibrary. The LoadLibrary function takes a DLL name as a parameter and loads the corresponding DLL into the currently running process's address space. To summarize these steps, the Rootkit calls CreateRemoteThread and passes it the PID of the victim process, the address of the LoadLibrary function, and the name of the Rootkit DLL. A thread will then start running within the address space of the victim process whose entry point is the LoadLibrary function. When the thread begins execution within the victim process, LoadLibrary is executed and the Rootkit DLL is loaded into its address space.

Once the Rootkit's code has been successfully injected into the victim process's address space, it needs to be initialized so that the Rootkit can make its necessary modifications to the process's memory bytes. To accomplsh this, the Rootkit takes advantage of the Windows DLL loading mechanism which calls a special callback function whenever a DLL is loaded. The special callback function is called dllmain after the DLL has been mapped into the victim's address space. Therefore, the Rootkit DLL will implement its initialization logic within the DllMain function.

Rootkits use a technique called hooking to intercept the normal function calls of the operating system. They hide their presence by diverting the program flow to the rootkit functions instead of legitimate system functions.

Rootkits employ a technique called "Kernel Hooking" to intercept common function calls and manipulate their behavior. For example, the kernel maintains a special table known as the "System Service Descriptor Table (SSDT). The SSDT contains the addresses of the functions that implement the various services for the system. A user program will call these functions by loading a system call number into the EAX register and issuing an 'int 2e' instruction which will cause the CPU to generate an interrupt. Once the interrupt is generated, the CPU looks into the Interrupt Descriptor Table (IDT) for the Interrupt Service Routine (ISR) that is responsible for handling interrupt number 2e. The ISR that handles interrupt 2e (called KiSystemService) saves the state of the CPU registers, copies the function parameters from the user-mode stack to the kernel stack, finds the system call number in the EAX register and uses it as an index into the SSDT to locate and jump to the corresponding system call handler.

A Rootkit can "hook" the SSDT by defining its own custom system call handler (called the 'hook function') and using it to replace a function already contained in the SSDT. Once a hook function is installed, it will be executed by any process on the system that calls the hooked system call. The Rootkit will often keep a reference to the original system calls that it replaces so that its hook functions can call the original functions and then manipulate the returned results before control is returned to user mode code. For example, a Rootkit may want to hook a system call that finds files in a directory in order to change the results by removing certain files that it is trying to hide. Suppose a virus scanning tool is querying all of the files in a directory and it issues a system call to tell it the names of all of the files in a specific directory. Assuming that the Rootkit has hooked the appropriate function in the SSDT, it's hook function implementation will be called instead of the original function. The Rootkit's hook function will then call the original function, change the results by removing the names of the files it wants to hide, and then return control to user mode. The anti-virus scanner receives the list of file names which contain the names of all the files in the directory except the names of the files that were removed by the Rootkit function. Hence, the anti-virus scanner does not know of the hidden files in the directory and therefore can not detect and remove them.

A Rootkit can also install a hook function in the Interrupt Descriptor Table (IDT) as well which effectively replaces the interrupt handler for a specific interrupt number. For example, a Rootkit could replace the entry at index 2e to install a custom interrupt handler for system calls. The result, whenever a process on the system executes an 'int 2e' the Rootkit's hook function will be called instead of KiSystemService.

Userland hooking can be done by modifying the first few instructions in the function itself so that the execution jumps to a detour function. This is called inline function hooking. When a call is made to a function, the jump instruction in the modified target function takes control to a detour function which is the root kit code. This detour function further calls a trampoline function which executes the instructions that were modified in the initial target function. Once these instructions are executed, the control returns to the point after the overwritten instructions in the original target function. When the function is executed, the results of the function are returned to the detour function. Hence the results can be manipulated by the rootkit to return modified results. So, in this way, the rootkit can hide the processes and other malicious code. This is the technique used by popular rootkits like Hacker Defender to hide ports, registries and services. Hooking is done by injecting the code into the process and this is known as code injection. The process’s memory is modified by using Windows API functions like CreateRemoteThread(), WriteProcessMemory() and SetThreadContext() so that it will carry out the necessary modifications. Once the process injection is done, the rootkit can inject code into other processes and can install hooks to hide the rootkit processes, services and ports from detection.

For example, consider the same example from the Kernel Hooks section of a Rootkit that wants to hide files in a directory from a virus scanning tool that is querying all of the files in a directory. The virus scanner will call the Windows API function FindNextFile (this function performs the system call that we hooked in the Kernel Hook example). The Rootkit will implement its own version of the FindNextFile function (called the Detour) in the injected DLL. When the Rootkit is initialized, it will get the address of the FindNextFile using the GetProcAddress API function. Once the Rootkit knows the memory location of the target function, it installs an Inline Function Hook by overwriting the first 5 bytes of the the FindNextFile code with a JMP instruction to the Detour function. The reason that 5 bytes are overwritten is because the JMP machine instruction requires 1 byte for the JMP opcode plus 4 bytes to encode the address of the Detour function. The Rootkit will save the 5 code bytes before it overwrites them and store them as a Trampoline function which executes the saved 5 bytes and then jumps back to the sixth byte of the target function. In essence, the trampoline simulates calling the original target function with no hook installed. Once the Rootkit has installed the hook, whenever any code within the victim process calls FindNextFile, control will jump to the Detour function. The Detour function will call the Trampoline function which executes the original FindNextFile logic and returns the results back to the Detour. The Detour function will then manipulate the results by removing the files that it is trying to hide and return the modified results back to the original caller.

The Rootkit implementation phase of the project consisted of the creation of two Rootkits; a kernel space Rootkit that hides system process information and a user space Rootkit that hides system files.

The kernel space Rootkit was implemented as a device driver which inserts a function hook into the Kernel's System Service Dispatch Table in place of the kernel function ZwQuerySystemInformation. A third party driver loader was used to register the kernel driver and activate it.

1.) The UserSpaceRootkit.dll containing the Rootkit code which installs an inline function hook into the code bytes of the Windows API FindNextFile function that is imported by all processes from Kernel32.dll.

2.) The DLLInjector.exe program which injects the UserSpaceRootkit.dll into the address space of a target process.

The first Rootkit (kernel space) was implemented as device driver that hides system processes from the Windows Task Manager program. For example, the Windows Task Manager program would normally display a similar process list as follows when no Rootkit is installed:

After the kernel Rootkit is installed and activated, the processes disappear from the task manager's process list as the Rootkit redirects the kernel's control path to the Rootkit's hook function whenever another program issues the system call to retrieve system information. Once the hook function has been called, it goes ahead and calls the intended ZwQuerySystemInformation and removes all of the processes from the returned process list (except for the System Idle Process). The hook function then returns the emptied list back to the caller (the Task Manager in our example) :

Rootkit #1 Workflow Description

Rootkit #1 functions as follows:

· rootkit_driver.sys is loaded into the kernel by a user-space program that calls the Win32 API functions: OpenSCManager, CreateService, and StartService. (See DriverLoader.cpp in Appendix A for an example)

· Upon being loaded, Windows automatically executes the ‘DriverEntry’ function contained in the rootkit_driver.sys module (See rootkit_driver.c Appendix A for complete source code listing).

· The DriverEntry function obtains the address of the kernel function ZwQuerySystemInformation from the System Service Dispatch Table and stores it in global variable so that it can be called later by the Rootkit.

· After saving the address of ZwQuerySystemInformation, the DriverEntry function replaces the SSDT entry with the address of the hook function that is implemented in the rootkit_driver.sys module. The hook function is called ‘NewZwQuerySystemInformation’. From this point on, if any process in the system issues a system call to execute ZwQuerySystemInformation, the NewZwQuerySystemInformation function will be invoked instead.

· When NewZwQuerySystemInformation is invoked, it first calls the original kernel function ZwQuerySystemInformation (using the address that the DriverEntry function saved before overwriting the function pointer).

· After having called the ZwQuerySystemInformation, NewZwQuerySystemInformation then changes the results.

· The ZwQuerySystemInformation function returns a linked list of system processes. The NewZwQuerySystemInformation function removes all of the processes from the list except for the System Idle Process.

· The NewZwQuerySystemInformation then returns the modified results back to the original caller.

The second Rootkit was implemented as dynamic link library (DLL) that is injected into a process to hide system files.

A sample program is implemented called 'FileSearcher.exe' which allows a user to input a directory whose files are to be listed. The program lists the files contained in the specified directory. For example, the contents of the directory C:\temp\rootkit\ might be listed as follows:

The user-space Rootkit is then injected into the FileSearcher.exe process where it inserts a detour into the FindNextFile Windows API function call of the FileSearcher process so that when the program attempts to list the directory contents by calling the imported FindNextFile function, execution control is detoured to the Rootkit where it changes the function parameters to cause the original function to return an invalid result. After modifying the function parameters, the Rootkit transfers control back to the original function where it continues its processing with the modified invalid parameters. The modified parameters cause the function call to return an invalid result and thus, the program believes that the directory contains no files:

· The DLLInjector program causes the target process to map the UserSpaceRootkit.dll into its virtual address space.

· The DLLInjector accomplishes this by calling the Windows API function CreateRemoteThread which causes a thread of execution to be scheduled in the target process. The DLLInjector sets the remote thread’s entry point routine to the Windows API function LoadLibrary that all processes import from Kernel32.dll. The DLLInjector sets the LoadLibrary parameter to the name of the UserSpaceRootkit.dll module.

· The thread in the target process then starts running the LoadLibrary code and the UserSpaceRootkit.dll is loaded into the virtual address space of the target process.

· Whenever a dynamic link library is loaded, the Operating System always calls an entry point function called ‘dllmain’ in the loaded DLL in order to allow it to initialize itself. The UserSpaceRootkit.dll implements the dllmain which gets called after LoadLibrary has mapped the UserSpaceRootkit.dll into the address space of the target process.

· From within the dllmain function, the Rootkit obtains the address of Kernel32.dll’s FindNextFile function by calling another API function called ‘GetProcAddress’.

· Once the Rootkit has the address of the FindNextFile function, it overwrites the first 11 bytes of the FindNextFile’s code with an unconditional JMP instruction whose parameter is the address of the Detour function implemented in UserSpaceRootkit.dll. From this point on, whenever the process calls FindNextFile, control will immediately jump to the detour function.

· The FindNextFile function requires the caller to pass it a parameter called a HANDLE that identifies the search context and allows the Operating System to remember the last file found in between successive calls to FindNextFile.

· The HANDLE parameter is pushed onto the stack by the caller before FindNextFile begins execution.

· The Detour function finds the 4-byte HANDLE on the stack that the caller pushed and overwrites it with the value 0x00000001 (an invalid HANDLE).

· The Detour then executes the overwritten instructions and transfers control back to the FindNextFile code immediately following the inserted JMP instruction.

· The FindNextFile function then proceeds normally, but with the modified search HANDLE. The invalid HANDLE causes FindNextFile to return prematurely and thus, it appears to the caller that no files were found.

Greg Hoglund and James Butler, "Rootkits : Subverting The Windows Kernel", Pearson Education Inc, 2006.

Galen Hunt, Doug Brubacher, "Detours: Binary Interception of Win32 Functions", Microsoft Research, 1999.

Matt Pietrek, "Peering Inside The PE: A Tour of the Win32 Portable Executable File Format", Microsoft Corporation, March 1994.

Mark E. Russinovich and David A. Solomon. "Windows Internals 4th Edition", Microsoft Press, 2005.

Chris Ries, "Inside Windows Rootkits", http://mirror.sweon.net/madchat/vxdevl/library/Inside%20Windows%20Rootkits.pdf, 2006.

Sherri Sparks, Jamie Butler, " Shadow Walker Raising the Bar for Windows Rootkit Detection" , http://www.cs.uiuc.edu/homes/kingst/spring2007/cs598stk/papers/p63-0x08_Raising_The_Bar_For_Windows_Rootkit_Detection.txt

Tom Stanley, "Protected Mode Software Architecture", Mindshare Incorporated, 1996.

Operating System Security:

A Study of Windows Rootkits