Malwares – Types & Defense

Raghunathan SrinivasanSept 25, 2007CSE 466/598

Computer Systems Security

Malware

How to define malware? Over a broad sense, any malicious

program Types

Viruses Trojans Rootkits Spyware

Virus

A program that can attach itself to another program Can replicate Encrypted

How to prevent them? Anti–virus???

How do they work No – real Answer

Types of Viruses [Evolution?] Parasitic Viruses

Also known as file infectors Date / Logic bomb

Michaelangelo, Sunday, Century Macro Viruses

Infect macro utility feature in word Encrypted Virus

cascade Polymorphic Virus

1260 Stealth Virus

Encrypted Viruses Viruses have

certain patterns present in them Signatures

AV looks for these patterns in files

To avoid detection, the virus encrypts itself

1. Mov

2. Fetch

3. ###$$$

4. &&^^^^

Decryption engine

Encrypted virus body

Encrypted virus

It is not possible to find out what the encrypted text is

So how to find if an encrypted entity is a virus? Look in previous slide

Next step – polymorphic viruses

Polymorphic Can change form from infection to infection There is a mutation engine present in the

virus body During run time – the virus loads the

mutation engine The ME changed the decryption routine The virus changes form on every encryption

Now the virus is difficult to spot

Detection Creating random encryption – decryption

routines is difficult See how many badly designed encryption

algorithms are present CSS

Hence encryption is weak, can be broken Can this be reliable?

No Then what to do?

Detection AV scanners use what is known as

simulation They create a virtual PC in the RAM Load the program in the Virtual PC The program executes, and shows its true

behavior eventually You can read the following paper for further

details Understanding and Managing polymorphic

viruses – Google it, it’s a white paper by Symantec

A new trend in Virus Viruses have become complex Anti-Virus programs are running powerful

engines game of cat and mouse

What further can virus writers do to prevent detection Go stealth

Install rootkits Install portions of program in various other

executables Disable detectors?

Disabling detectors

If you don’t have a defense mechanism, you cant escape infection

Kill all security processes Works, but a smart user can figure

something Patch on the definitions Patch on the program policies

How does that help Disable updates?

Examples of such viruses SpamThru

Locates existing AV in the machine Patches them to prevent updates Installs its own virus scanner

Why? Beast

Kills all existing security services Hooks on to winlogon.exe

What is winlogon.exe

Implications? AV does not function No method to detect the presence of

viruses How to solve this?

Borrow some virus tricks Hide the AV Move the program code Hide files Hide Process name

Other Miscellaneous Malware Worms

Self replicating program Does not require host to replicate It uses the network to send copies of

itself They use the bandwidth and harm the

network Viruses harm the computer (host) Does worm not harm the PC?

Not necessarily Worms for ATM’s

Slammer, Nachi

Trojan USC Trojans? People from the affair of Helen of Troy? NO Program that enters a system disguised as

something else Never trust the gifts from Greeks (lesson learned

from trojan war) Trojan perhaps looks harmless

Or useful Allow installation

Backdoors Rootkits

Rootkits

Term derived from UNIX account ‘root’

Patches on to host kernel libraries, routines

Place hooks on API’s, OS services, Routines, etc

A good rootkit cannot be detected Does the statement sound too strong?

Shadow Walker Designed to deceive in signature

scanners That is how Anti-Virus and most Rootkit

detectors work Hides its presence in the system It hooks on to the page table entries

& the page fault handler It flushes the TLB

No page can be accessed bypassing the page fault handler initially

Shadow Walker ….

So how does that help A scanner attempts to read a page A fault is generated This causes a fetch The rootkit ensures that the scanenr

never gets any access to infected pages

BluePill Rootkit designed for Vista running on AMD

pacifica technology Has special mode for VM executions

Allocate memory for a process More than required What does this do?

Rootkit writes on the paged drivers When the drivers are loaded back, you

have infected drivers in memory Allows Vista to be moved in guest

environment Rootkit becomes a hypervisor

Bluepill .. contd So what happens due to that? The Vista OS becomes the guest, and is

completely under the control of the rootkit. Any scanner working from within the OS can never see the rootkit.

Why? An OS process cannot have access to the layer

below the OS So if we placed something below the OS, the OS

cannot find out about it.

Scenario 1

Hardware

OS Kernel

ApplicationsApplication level malware – easy/slightly difficult to detect

kernel level malware – very difficult to detect

Scenario 2

Hardware

OS Kernel

Applications

VMM layer malwareNot possible to detect from within the OS. Requires Hardware detection

Solutions

VM based rootkit detectors Hardware based rootkit detectors

VMM based detector Type I VMM [XEN] The VMM runs on top

of the hardware. Root of trust

mechanism VMM checks the

privileged VM The PVM checks the

SM SM checks the other

VM’s

VMM The VMM runs 1 Privileged VM(VM0),

and many other guest VM’s The VMM checks the VM0 over

periods of time Ensures the kernel of VM0 is not

tampered with VM0 runs the SM

It contains the integrity values of SM, to detect tampering

VMM detector - contd

The SM can access the states of all applications running on all the Guest VM’s

Guest VM’s run OS’s that run user applications

So what has this achieved? Layered Software

The Trusted VM

What has to be done to penetrate the VMM layer Attack the applications Attack the guest OS Attack the Guest VM Finally attack the VMM

SM detects these before the final step

VMM layer Is a micro kernel

What is a micro kernel Answer: Best left to OS classes

Hence not a general purpose OS Does not execute third party software Due to this, it is secure

Too strong a statement? Ok, has fewer vulnerabilities (due to less code) Has fewer loopholes to exploit Does not suffer from infected third party drivers

What does the VMM do? Isolation between programs in an

Operating System’s is a very difficult process

Many researches on it, fairly inconclusive

VMM provides isolation between the Guest VM’s

VMM also allows us to sandbox an OS and monitor it

VM0

Monitors the SM It can also allow and prevent other

VM’s from accessing certain memory locations

It can protect sections in memory It can prevent other VM’s from

accessing some I/O devices Why is this important?

SM

Checks the VM Provides secure communication to

User Why is this important?

The SM has access to the state of registers, memory and instructions being executed by each Guest Vm This helps to monitor the GVM’s

SM - contd

Checks the integrity values of Guest OS’s during boot Allows detection of boot sector

infections, rootkits Can this help us detect VM based

rootkits? Checks kernel integrity, OS text

section, interrupt vectors, etc

Last step

Can a rootkit impersonate a user Yes, at least it will attempt to do so

So how can this be prevented? The last module Secure I/O device Do you see the answer to a question

regarding I/O device access 3 slides back?

Secure I/O

Provides a trusted mode of communication between user and VMM

It should be a separate device Why? Why cant it be a software channel

Why do we need secure I/O

Are human validations really true What happens if this step is not followed

A viral program can trick the guest OS into sending a message that an update was performed

Allows changing of integrity values The malware gets certified by the SM

Hardware detectors Separate hardware

device Attached to the PCI

slots Can be attached in

other places also Some

implementations involve placing a co-processor on the motherboard

Hardware detectors

This is also a root of trust device The hardware device runs an OS Its resources and state are not

accessible by host CPU/HW It is capable of accessing the host’s

memory It can halt a system if required

Heirachical checking Each level stored

the integrity values of the level above it

The SecCore contains the integrity values of certain critical sections of the kernel

SecCore

The critical sections of the kernel is responsible for checking the rest of the kernel

It is also responsible for checking the applications

The kernel is responsible for maintaining the integrity of the User level programs

Advantages

The Coprocessor does not have to attest the entire OS

Keeps load low It stores information only about a

small space Memory requirements low Most of the checking is offloaded to

the Host CPU

Problems

Many integrity values reside inside the kernel

Can be infected Solution?

Sign them Digital Signatures

Thanks