Packet Anomaly Intrusion Detection PAID

Constantine Manikopoulos

and Zheng Zhang

New Jersey Center for Wireless Networking and Security (NJWINS) at NJIT

George Mason University

September 24-26, 2003

The HIDE/PAID Project

NJWINS – US Army SBIR Phase II Research and Development Effort

• Prototype and Evaluate an Intrusion Detection System for the Tactical Internet of the Digital Battlefield

System Architecture

• Components– Probe– Event

preprocessor– NN classifier– Post processor

Probe Event Preprocessor

Reports from IDAs oflower tiers

NetworkTraffic

Statistical Processor

Neural Networkclassifier

Post ProcessorTo UserInterface

To Higher Tier

System Architecture

BS1 BS2 BS3 BS4

GatewayRouter 1

GatewayRouter 2

Router

Tier 1

Tier 2

Tier 3

Multi-layer Detection

Event BufferReference

Model

EventReport

Event BufferReference

Model

Event BufferReference

ModelLayer-Window M

Layer-Window 2

Layer-Window 1

...

PDF Representation

-4 -2 0 2 4 6 80

0.05

0.1

0.15

0.2

0.25

x

PD

F

binned representationoriginal PDF

Binned PDF Representation

•S be the sample space of a random variable

•events E1­,­E2,…,­Ek a mutually

exclusive partition of S

•Pi­is the expected probability of the

occurrence of the event Ei

•Pi’ be the frequency of the

occurrence of Ei during a given time

interval

Similarity Measuring Algorithms

2-like test.

• Kolmogorov-Smirnov test.

• Anderson-Darling’s statistic.

• Kupier’s statistic.

• Others.

Similarity Measuring Algorithms

pi is the expected probability of event Ei.

Pi’ is the observed probability of event Ei during a time interval.

f(N) is a function that takes into account the total number of occurrences during a time window.

)]().[( '

11

' max ii

k

i

k

iii ppppNfQ

Reference Model Updating

Reference Model Updating Algorithm

pold is the reference model before updating

Pnew is the reference model after updating

is a programmable predefined adaptation rate

s is a learning rate determined by the outputs of the neural network

oldobsnew pspsp )1(

HIDE/PAID: User Interface

Monitoring Parameters Current Pdf Reference Pdf

1. Parameter 1

2. Parameter 2

3. Parameter 3

4. Parameter 4

5. Parameter 5

Most SuspiciousParameters

0 1-1

0 1-1

0 1-1

Aggregate Parameter 1

Aggregate Parameter 2

Aggregate Parameter 3

Parameter 1

Parameter 2

Parameter 3

Two-Dimensional Scatter Plots

Two-dimensional Scatter Plots

Sample Visualization

Normal Attack traffic

Data Description

• DARPA’98 Intrusion Detection Evaluation Data Set– Seven weeks of training data– Two weeks of testing data (not used because

the attack truth is not available)– Categories of the simulated attacks: DOS,

Probe, R2L, U2R

System Configuration

• Only Non-stealthy DOS attacks are tested:– Neptune (SYN flooding), – Pod (Ping-of-Death), – Smurf (ICMP flooding), – Teardrop (Pathetic IP Fragmentation)

• PDF Observation Time Window: 30s.• Classifier: Backpropagation with 4 hidden

neurons

Detection Results on y98w1d3

# of Samples 1970

# of Attacks 2

# of True Positives 2

# of True Negatives 1968

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

dete

ctio

n ra

te

Detection Results on y98w3d4

# of Samples 2520

# of Attacks 104

# of True Positives 104

# of True Negatives 2416

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

dete

ctio

n ra

te

Detection Results on y98w4d2

# of Samples 1769

# of Attacks 15

# of True Positives 14

# of True Negatives 1742

# of False Positives 12

# of False Negatives 1

# of Misclassifications 13

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w4d3

# of Samples 1649

# of Attacks 2

# of True Positives 2

# of True Negatives 1647

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w5d1

# of Samples 926

# of Attacks 64

# of True Positives 64

# of True Negatives 862

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w5d2

# of Samples 2335

# of Attacks 3

# of True Positives 3

# of True Negatives 2332

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w5d4

# of Samples 519

# of Attacks 176

# of True Positives 171

# of True Negatives 343

# of False Positives 0

# of False Negatives 5

# of Misclassifications 5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

dete

ctio

n ra

te

Detection Results on y98w5d5

# of Samples 2315

# of Attacks 108

# of True Positives 108

# of True Negatives 2207

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w6d1

# of Samples 4911

# of Attacks 11

# of True Positives 11

# of True Negatives 4885

# of False Positives 15

# of False Negatives 0

# of Misclassifications 15

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w6d2

# of Samples 2438

# of Attacks 1

# of True Positives 1

# of True Negatives 2437

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w6d3

# of Samples 2504

# of Attacks 107

# of True Positives 107

# of True Negatives 2397

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w6d4

# of Samples 1202

# of Attacks 284

# of True Positives 284

# of True Negatives 912

# of False Positives 6

# of False Negatives 0

# of Misclassifications 6

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w6d5

# of Samples 1297

# of Attacks 54

# of True Positives 53

# of True Negatives 1242

# of False Positives 1

# of False Negatives 0

# of Misclassifications 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w7d2

# of Samples 2438

# of Attacks 1

# of True Positives 1

# of True Negatives 2437

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w7d3

# of Samples 1897

# of Attacks 1

# of True Positives 0

# of True Negatives 1895

# of False Positives 1

# of False Negatives 1

# of Misclassifications 2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w7d4

# of Samples 5154

# of Attacks 4

# of True Positives 4

# of True Negatives 5150

# of False Positives 0

# of False Negatives 0

# of Misclassifications 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Detection Results on y98w7d5

# of Samples 1369

# of Attacks 119

# of True Positives 111

# of True Negatives 1250

# of False Positives 0

# of False Negatives 8

# of Misclassifications 8

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

false alarm rate

de

tec

tio

n r

ate

Summary (1)

Total # of Samples 39015

Total # of Attacks 1060

Total # of Misclassifications 50

Total # of False Positives 35

Total # of False Negatives 15

Misclassification Rate 0.128%

False Positive Rate 0.0898%

False Negative Rate 1.42%

Summary (2)

Attack # of Samples # of False Negatives

False Negative Rate

Neptune 786 13 1.65%

Pod 24 0 0

Smurf 266 0 0

Teardrop 9 2 22.2%