Operating System Security :

35
Operating System Security : David Phillips A Study of Windows Rootkits

description

Operating System Security :. A Study of Windows Rootkits. David Phillips. What Is a “Rootkit”?. The term “Root” (super user) originates from the Unix operating system and “kit” is the program that grants an attacker super-user abilities. Different Applications - PowerPoint PPT Presentation

Transcript of Operating System Security :

Page 1: Operating System Security :

Operating System Security :

David Phillips

A Study of Windows Rootkits

Page 2: Operating System Security :

What Is a “Rootkit”?

• The term “Root” (super user) originates from the Unix operating system and “kit” is the program that grants an attacker super-user abilities.

• Different Applications– A Rootkit can be a tool that an attacker installs into a

system in order to deceive the other processes that are running on the system.

– A Rootkit can be a tool that hides in memory and snoops on system activities while not engaging in deception.

Page 3: Operating System Security :

What is NOT a Rootkit?

• A Rootkit is not a computer virus (although a virus might deploy a Rootkit to assist in an attack).

• A computer virus is self-replicating and generally advertises its presence on the system by causing damage and attempting to infect other programs.

• A Rootkit does not attempt to replicate itself nor does it typically engage in damaging the system. Its role is more covert in nature and focused on information snooping or modifying the rules of the system.

Page 4: Operating System Security :

How Does A Rootkit Work?

• A Rootkit “program” is not a typical executable file.

• The Rootkit logic is typically implemented within a kernel module (device driver) or shared library (Dynamic Link Library).

• An executable user-space program called the “loader” program will load the Rootkit module/library into the system using well known Application Programmer Interface (API) system calls.

• Once the Rootkit module is loaded, the module’s entry point initialization routine is called by the operating system. The Rootkit uses the initialization routine to install itself.

Page 5: Operating System Security :

Rootkit Classification

• Rootkits can be classified into two different types: – User Space Rootkits (User Application)– Kernel Space Rootkits (Device Driver)

Page 6: Operating System Security :

User Space Rootkits

• Pros: Easier to write (access to lots of user-space libraries and API’s that cannot be accessed from Kernel space). No kernel module required. Relatively easy to “inject” into other processes.

• Cons: Does not have root privileges on system. Easier for anti-Rootkit software to detect.Affects only the behavior of a single process.No access to kernel data structures.

Page 7: Operating System Security :

Kernel Space Rootkits

• Pros:Affects ALL processes running on the system.

Has root privileges.

Harder for anti-Rootkit software to detect.

Able to access kernel data structures.

• Cons:More difficult to write.

System user must have adequate rights to install.

Page 8: Operating System Security :

Rootkit Implementation Phase

• The Rootkit implementation phase of the project consisted of the creation of two Rootkits:

1.) A kernel space Rootkit that hides system process information by installing a hook

function in the kernel’s System Service Dispatch Table in place of the kernel’s ZwQuerySystemInformation function.

2.) A user space Rootkit that hides system files by installing an inline function hook into

Windows API FindNextFile function. 

Page 9: Operating System Security :

Kernel Space Rootkit Implementation

• Implemented using the Windows Device Driver Kit for Windows XP.

• The Rootkit logic was compiled into a kernel driver/module named “rootkit_driver.sys”.

• All driver modules export a function called ‘DriverEntry’. When the module is loaded into the kernel, the operating system automatically calls this routine.

Page 10: Operating System Security :

Kernel Space Rootkit Driver Entry

This function is executed in Kernel space.

• Device drivers use this function to initialize themselves.

• The Rootkit uses this function to make changes to the kernel structures and install “detours”.

Page 11: Operating System Security :

ZwQuerySystemInformation

• A function implemented in the kernel that constructs a list of processes currently running on the system and returns the list to the caller.

• A user space application (such as the Windows Task Manager) cannot call this function directly because it exists only in the Kernel’s address space.

Page 12: Operating System Security :

Background: Windows Task Manager

• User Space application that displays all of the current processes running on the system.

• Cannot call ZwQuerySystemInformation directly.

• Must perform a System Call instead.

Page 13: Operating System Security :

Windows Task Manager Control Flow

• The Windows Task Manager issues a system call which causes control to jump to Kernel function ‘KiSystemService’.

• The KiSystemService routine locates the ZwQuerySystemInformation in the System Service Dispatch Table using the system call number as an index.

• KiSystemService then calls ZwQuerySystemInformation.

Page 14: Operating System Security :

Kernel Space Rootkit Hook

• The Rootkit module implements its own version of the ZwQuerySystemInformation routine named “NewZwQuerySystemInformation”.

Page 15: Operating System Security :

Kernel Space Rootkit Hook

• During execution of the Rootkit module’s DriverEntry routine, the Rootkit modifies the System Service Dispatch Table by replacing the kernel’s ZwQuerySystemInformation with the Rootkit’s NewZwQuerySystemInformation function.

• From now on, when any process on the system (like the Task Manager) issues a System Call for the process list – the Rootkit’s NewZwQueryInformation routine will be called instead of the kernel’s ZwQuerySystemInformation function.

Page 16: Operating System Security :

NewZwQuerySystemInformation

• The Rootkit’s hook function NewZwQuerySystemInformation calls the kernel’s ZwQuerySystemInformation to masquerade as though the original function were called.

Page 17: Operating System Security :

NewZwQuerySystemInformation

• The original function returns the real list of processes back to the hook function. The hook function then iterates through the returned list and removes all process entries except for the “System Idle Process”.

• The hook function then returns the modified list back to the original caller that invoked the system call.

Page 18: Operating System Security :

Rootkit Filters The Process List

Page 19: Operating System Security :

Task Manager After Kernel Hooked

• The Task Manager receives back the process list that was filtered by the Rootkit’s hook function.

• As a result, no processes show up in the list except for the System Idle Process.

Page 20: Operating System Security :

User Space Rootkit Implementation

• Implemented as a Dynamic Link Library using Visual Studio 2005.

• The Rootkit logic was compiled into a Dynamic Link Library named “UserSpaceRootkit.dll”.

• The Rootkit library is activated by a user-space loader program that “injects” the library into a target process that is already running on the system.

Page 21: Operating System Security :

DLL Injection

• In order to activate the Rootkit, the DLL that contains the Rootkit implementation needs to be loaded into the address space of a target process.

• Forcing another process to load a DLL into its address space is called “DLL Injection”

• This action is performed by the Rootkit loader program.

Page 22: Operating System Security :

How Does a Process Load a DLL?

• A process typically loads a DLL into its own address space by calling the Windows API function “LoadLibrary”.

Page 23: Operating System Security :

How Does a Process Cause Another Process To Load a DLL?

• A process can cause another target process to load a DLL by instructing any thread within the target process to execute the LoadLibrary function.

• The Windows API defines a function named ‘CreateRemoteThread’ that allows a process to create a new thread within a another process.

Page 24: Operating System Security :

How Does The Rootkit Loader Inject The

Rootkit DLL Into the Target Process?• The Rootkit loader creates a remote thread in the target

process. For the thread’s entry point function, the Rootkit passes the address of the LoadLibrary function. It passes the string “UserSpaceRootkit.dll” as a parameter to the LoadLibrary routine.

Page 25: Operating System Security :

Rootkit Loader Continued

• The remote thread begins executing the LoadLibrary function within the context of the target process and the UserSpaceRootkit.dll is loaded into the target process’s address space.

• Once the UserSpaceRootkit.dll is loaded into the target process’s address space; the Operating System automatically calls the dllmain function that is exported by the DLL to allow it to initialize itself.

Page 26: Operating System Security :

User Space Rootkit dllmain

This function is executed in User space in the context of the target process.

• DLL’s use this function to initialize themselves.

• The Rootkit uses this function to make changes to a process’s structures and install “detours”.

Page 27: Operating System Security :

FindNextFile• A Windows API function that is repeatedly called by user space

programs that want to obtain a list of files contained on the file system.

• When a process begins execution, the FindNextFile code is imported from the system library Kernel32.dll and mapped into the process’s address space.

• The process can then call this function directly to iterate objects on the file system.

Page 28: Operating System Security :

User Space Rootkit Hook• During execution of the dllmain function, the Rootkit locates the

FindNextFile function in the target process’s address space.

• After locating the function, the Rootkit changes the first 11 code bytes of the function to an unconditional jump to a “detour” function that is implemented in the UserSpaceRootkit.dll.

Page 29: Operating System Security :

User Space Rootkit DetourOnce the JUMP has been inserted, whenever any thread in the target process calls the FindNextFile function – control will jump to the Rootkit’s detour function which changes the value of the HANDLE function parameter to an invalid value.

Page 30: Operating System Security :

Resume The Original Function• Once the detour has invalidated the function parameter,

it needs to jump back to the original function to let it run “as normal” (but with corrupted parameters).

• The detour function first executes the instructions that were overwritten by the inserted JMP instruction.

Page 31: Operating System Security :

Resume The Original Function• After executing the overwritten instructions, the detour

function then jumps back to the first code byte of FindNextFile that follows the inserted JMP instruction.

• The FindNextFile function then proceeds as normally, but with a corrupted parameter that causes the function to return an invalid result.

Page 32: Operating System Security :

The Windows Command Prompt• User space application that calls FindNextFile to list the

contents of a directory when user types the command ‘dir’.

Page 33: Operating System Security :

Command Prompt Injection

• The UserSpaceRootkit.dll is injected into the command prompt using the DllInjector.exe loader.

Page 34: Operating System Security :

Injected Command Prompt• Once the Rootkit is injected into the command prompt,

whenever the user tries to list the files of a directory – an empty list is returned.

Page 35: Operating System Security :

Thank you.

Questions?