Detection and Propagation Modeling of Internet Worms

Ph.D. research proposal by:Parbati Kumar Manna

Co-advised by:Dr. Sanjay Ranka and Dr. Shigang

Chen

2

Overview

• Research opportunities in Internet worm

• Contributions towards my dissertation:Detection of text worm

Propagation modeling for Permutation-Scanning worm

Finding the optimal scanning strategy

• Current status and timeline

3

• Computer Security Vs. Network Security

• Malware Computer Viruses

Internet Worms

Trojans

Rootkits

Introduction

4

Internet Worm

• Huge damage potentialInfects hundreds of thousands of computers

Costs millions of dollars in damage

Melissa, ILOVEYOU, Code Red, Nimda, Slammer, SoBig, MyDoom

• Mostly uses Buffer Overflow

• Propagation is automatic

• Characterized by its host-level and network-level behavior

5

Recent Trends

• Worms becoming increasingly evasive and obfuscative

• Arrival of Script Kiddies

• Emergence of Zero-day worms

• Shift in hacker’s mindset

6

Defenses

• PreventionSecure code, Corruption detection, Address

randomization, Non-executable stack

• DetectionPayload signature, NOP sled, CFG, Abstract

Payload Execution, Emulation, Honeypot,PADS

• ContainmentAddress blacklisting, Content filtering, Rate

Limiting, LaBrea tarpit, Failed connection

• Propagation ModelingSI model, SIR, RCS, two-fact model

7

Smart Worm

Evaluation Parameters

Traditional Worm

Worm of Future

Disruption of Service

Maximum Minimal

Network Footprint

Significantly high

Relatively low

Detectability High Low

8

Proposed Work

Worm characterizationand countermeasures

How to detectthe new worms?

What are the effects on the Internet?

How bad can thesituation get?

•Evaluate existing detection systems against advanced worms

•Devise detection strategy for ASCII worm

•Obtain propagation characteristics for Permutation-Scanning worm

•Enhance the potency for Permutation-Scanning worm

•Identify the desired goals of scanning

•Compare the existing scanning methods

•Evaluate if any of the existing propagation strategies are optimal

9

Problem I

Detection of ASCII Worm

10

Motivation

• Presumption of text being benign

• Prevalence of servers expecting text-only input

• Deployment of ASCII filter for bypassing text

• Exponential disassembly cost

• High processing overhead for IDS

11

Buffer Overflow

Overflowing a buffer using an ASCII string:

12

Creation of ASCII Worm

13

Proposed Solution

Malicious Benign

• Lack of opcodes• No negative

displacement • Long decrypter• Long sequence of

valid instructions

• Contains characters that correspond to invalid instructions

• Long sequence of contiguous valid instructions unlikely

No error during execution

14

Proposed Solution

Questions:

• How long is “long”?

• What is the probability of false positive for that threshold?

• Find out the maximum length of valid instruction sequence

• If it is long enough, the stream contains a worm

15

• Toss a coin n times

• What is the probability that the max inter-head distance is ?

Probabilistic Analysis

Head Invalid Instruction

Tail Valid Instruction

T H T T H T T T T T H T T TV I V V I V V V V V I V V V

16

Probabilistic Analysis

n = number of coin tosses p = probability of a head Xi = R.V.s for inter-head

distancesXmax = Max inter-head distanceC.D.F of Xmax = Prob [Xmax ≤ x]

= [1 – p(1-p)x ]n

F.P. rate = 1 - Prob [Xmax ≤ τ] = 1 - [1 – p(1-p)τ ]n

17

Threshold Calculation

n , p, (false positive rate)

(max inter-head distance)

Known

Unknown

)1log(

log))1(1log(1

p

pn

Threshold

18

Threshold Calculation

With increasing n, we must choose a larger to keep the same rate of false positive

19

Determine n

size)n instructio (average

)charactersinput ofnumber (

I

Cn

E[I] = E[Prefix chain length] + E[core instruction length]

Obtained from character frequency of input data

20

1.Privileged instructions

2.Wrong Segment Prefix Selector

3.Un-initialized memory access

Determine p

Invalid Instructions

Only 1. and 2. can be determined on a standalone basis

21

Implementation

22

Experimental Setup

• Benign data setupASCII stream captured from live CISE

network using Ethereal

• Malicious data setupExisting framework used to generate ASCII

worm by converting binary worms

• Promising experimental results for max valid instruction lengthBenign: all max values all below threshold Malicious: values significantly higher than

23

Contributions

• Analyzed the behavior characteristics & constraints of ASCII worms and devised a detection method

• Derived mathematical foundation for generic detection method used in other worm detection strategies

• Deterministic - no “parameter tuning”

24

Problem II

Propagation Modeling

forPermutation-

Scanning Worm

25

Motivation

• Random scanning

Wastes scanning power

• Simple Divide scheme

Not fault tolerant

Unequal load

26

Permutation-Scanning

• Randomizes the real address space into a Permutation Ring

• Each freshly infected host starts scanning from a random location

• Retires upon hitting an already infected host

Real address space

Permutation ring

new host jumps

about to infect

activeactive

retiredGets

infected, jumps

27

Why Model?

• Simulation takes long time16 hrs / run for 400M hosts

• Simulation overhead could be prohibitively highImpossible to scan full IPv6

• Simulation does not always provide mathematical insight

28

• Find # (active hosts) scanning– effectively (X)

– ineffectively (Y)

• Among the scans from the effective hosts (X), calculate how many are hitting uninfected hosts.

• Find how many X and Y hosts hit a pre-infected host (and retire).

Solution Outline

X1 X2

Y

coveredarea

29

Vulnerable Host Classification

30

State Diagram

31

Interaction among Infected Hosts while scanning

32

Final Model for O-jump Permutation Worm

V

ttxtiVtf

V

ttxtitf

ttxtiV

tiVtf

ttxtiV

ttxtf

N

Vdtrf

eff

ineff

new

old

hit

)()()()(

))()(()()(

)()()(

)()(

)()()(

)()()(

Y

X

X

(effective)

(ineffective)

Fraction (covered area)

33

Final Model for O-jump Permutation Worm

0)0()0()0(,)0()0()0(

)()()(

)()()()(

)()()()()(

)()()()()(

)()()()()()(

)()()(

syxai

tdytdxtda

ftytfftxtds

ftytftfftxtdy

fttftfftxtd

tfftxtftfftxtdx

tfftxtdi

hitoldhit

hitineffnewhit

hiteffnewhit

oldhiteffnewhit

newhit

infected

Retired

Active

34

Model Vs. Simulation

N = 223 V = 213 (hitlist size) = 100

35

Extending Model to k-jump Permutation-Scanning Worm

• Instead of retiring, jump another time and restart scanning

• Will retire only after hitting more than k old infections

• Higher infection speed and network footprint

36

State Diagram for k-jump Permutation-Scanning Worm

37

Propagation Model for k-jump Permutation Worm

Similar equations for d(t), dy(t)

38

Propagation Results for k-jump permutation worm

N=223

V=213

=100

39

Contributions

• Obtained propagation model for Permutation-Scanning worms

• Extended modeling for multiple-jump

• Obtained the effect of various worm/network parameters:Bigger hitlist ()Larger V (more vulnerable computers)Bigger N (IPv4 IPv6)Increased k (more jumps allowed)

40

Problem III

Comparative Analysis of Different Scanning Strategies

41

The Next Big One?

• Warhol worms

• Self-stopping worms

High infection speed

Very low network footprint

Modest fault tolerance

42

Motivation

• To find the optimal scanning strategy

• Achieve the most desirable goals of scanning

Infection speed

Stealth

Fault tolerance

43

The Three Proponents

• Random-Constant-Spread wormVery high fault tolerance

• Divide-and-Conquer wormVery low network footprint

• Permutation-Scanning wormHigh infection speed

44

Proposed Work

• Derive propagation curves for all the scanning strategies using same set of notations in order to compare them

• Show equivalence of RCS and Permutation-Scanning worm in terms of infection speed

• Explore the possibility of hybrid scanning strategies

45

Current Status And

Timeline

46

Current Status

• Detecting ASCII Worms Conference paper titled “DAWN: A Novel Strategy for

Detecting ASCII Worms in Networks” submitted to IEEE INFOCOM 2008 and is currently under review

• Modeling Permutation Scanning Conference paper titled “Exact Modeling of Propagation for

Permutation-Scanning Worms” is pending review for IEEE INFOCOM 2008

• Finding Optimal Scanning Strategy Work currently in progress Obtained theoretical equivalence between RCS and

Permutation-Scanning worm In process of modeling Divide-and-Conquer worm

47

Questions

48

Thank you