Using Bottleneck Verification to Find Novel New Attacks ... · Using Bottleneck Verification to...

MIT Lincoln Laboratory8 August 98 - 1

Richard Lippmann

Using Bottleneck Verification to Find Novel New Attacks with a Low False Alarm Rate

R. P. Lippmann, D. Wyschogrod, S. E. Webster,

D. J. Weber, A. S. Gorton

MIT Lincoln Laboratory

Presentation at the RAID’98 ConferenceBelgium

Sept 14-15, 1998


Richard Lippmann

Outline

• Baseline Intrusion Detection Evaluation

– Normal Data

– Attacks

– Evaluating Performance

• Approaches to Intrusion Detection

• Bottleneck Verification to Find Illegal Root Transitions

– Sniffer Based

– Audit Based

• Summary and Future Plans


Richard Lippmann

Baseline Sniffer-Based Intrusion Detection System

INTRUSION DETECTOR

HOST HOST ROUTER

INTERNET

SNIFFING

DATA

• Monitors Many Workstations with one Sniffer, Computes Probability of Attack for Every TCP/IP Session

• Attack Probability Based Primarily on Number of Times Keyword Strings Occur in Session Transcripts

• Examples of Keyword Strings – ftp: root, anonymous, failed login (>3)– login: guest, root, incorrect, daemon, passwd, permission

denied

• Similar to Net Ranger, and other NSM Derivative Systems

• Provide Baseline Performance Measure


Richard Lippmann

Sample Transcript Showing Keywords

HP-UX hqdadev A.09.03 D 9000/750 (ttyt1)login: ~tftpPassword:Login incorrectlogin: efsPasswordLogin incorrectlogin: efsPassword:Password:/usr/efs w

3:03pm up 14 days, 21:33, 1 user, load average: 0.00, 0.00User tty login@ idle JCPU PCPU whatefs pty/ttyt1 3:03pm 1 w/usr/efs> cd /

ls -al

total 16336drwxr-xr-x 47 root sys 3072 Sep 23 10:41 .drwxr-xr-x 47 root sys 3072 Sep 23 10:41 ..drwx------ 2 root mail 1024 Nov 28 1994 .elm-rw------- 1 bin bin 8690 Jul 9 15:39 .profile-rw------- 1 root sys 37 Nov 18 1994 .rhostsdr-x------ 3 root other 1024 May 26 1994 .securedrwxr-x--- 3 root sys 1024 Jul 3 14:30 Perlgrep :0: /etc/passwd

root:*:0:3:Beginning of All Things...,,976-HPUX,:/:/bin/ksh


Richard Lippmann

Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



Richard Lippmann

NORMAL BACKGROUND DATAFROM 50 SITES (July - November 1996)

• Pre-Filter Assigns Warning Values to Each Connection• Transcripts Created Only for Conn's With High Warning Values• Many Connections (34.6 M), Fewer Transcripts (375,000)

SNIFFER

HOST HOST ROUTER

INTERNET

PRE-FILTER

CONNECTIONS(34.6 Million)

TRANSCRIPTS(375,000)

1/ 90


Richard Lippmann

Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



Richard Lippmann

Capabilities of AttacksA

TT

AC

K S

TA

RT

ST

AT

E

ATTACK GOAL

LOCAL USER LOCAL ROOT

RE

MO

TE

RO

OT

LO

CA

LN

ET

RO

OT

LO

CA

LU

SE

RNEWS

CRACK

SNIFFIP

SPOOFING

GUESTLOGIN

MIMESENDMAIL

PERL

NEWS +PERL

STACKOVERFLOW

DICTION

REMSH

TELNETSETENV

FTPCOREDUMP


Richard Lippmann

Attack Types

• Old Attacks That Have Been Known For Many Years

– guest login, dictionary attack, internet worm, crack, sniffer, ...

• Recent Attacks That Are Less Than a Year Old

– buffer overflows, ip spoofing, news bug, ...

• 18 Actual Illegal Root Transitions

– perl, loadmodule, hp vhe bug, suid root shell backdoor, telnet environmental variable bug, sniffed passwords

MIT Lincoln Laboratory8 August 98 - 10Richard Lippmann

Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



Measure Performance by Artificially Injecting Intrusions

• Truth Is Known, Assume Normal Traffic Has Few True Unknown Intrusions

• Exploits– OLD– NEW– REAL Attacks Found in Normal Traffic

SNIFFER

BASELINEINTRUSION DETECTION

SYSTEM

WARNING

VALUE

INJECTATTACKS

ATTACK

NORMALTRAFFIC


Evaluating Intrusion Detection Systems

• Vary Threshold to Detect 80% of Attacks, Then Ignore Rejected Transcripts and Examine Accepted Transcripts

ACCEPT

WARNING VALUE > THRESH ?

NO

YES

REJECT

TRANSCRIPT(375,000)

WARNINGVALUEINTRUSION

DETECTION SYSTEM

REJECTED ATTACKSARE MISSES

ACCEPTED NORMALSESSIONS INCREASE

WORKLOAD AND FALSE ALARMS


Workload for Baseline System(Average Transcripts to Examine Per Attack)

• Average Number of Transcripts a System Administrator Must Examine to Detect One Attack

– Set Threshold to Detect 80% of Attacks• Workload Assumes 8 Hour Days, 1 Minute Per Transcript

REAL OLD NEW0

10000

20000

30000

40000

50000

60000

ATTACK TYPE

BASELINE SYSTEM

2,8006 DAYS

15,59032 DAYS

43,00090 DAYS


Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



Current Approaches to Intrusion Detection

FINDS NEW

ATTACKS

ANOMALY DETECTION

BOTTLENECK VERIFICATION

SPECIFICATION-BASED

ONLY FINDS OLD

ATTACKS

COMPUTATION AND MEMORY

REQUIREMENTS

SIGNATUREANALYSIS


Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



BOTTLENECK VERIFICATION

• DETECT IMPORTANT SYSTEM SECURITY STATE CHANGES (e.g. User Obtains Root Privilege, Log Into Remote Machine)

• DETECT USE OF LEGAL STATE CHANGE TOOL THAT ENFORCES A BOTTLENECK (e.g. Valid SU Command)

• FLAG AN ATTACK IF STATE CHANGE OCCURS WITHOUT GOING THROUGH THE BOTTLE NECK

• DETECTS BOTH KNOWN AND NEW ATTACKS

NORMAL SYSTEM STATE

PRIVILEGED STATE

TRANSITION BOTTLENECK


Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



Evaluating Sniffer-Based Bottleneck Verification (50 Air Force Bases - Jul, Aug, Sep,Oct 1996)

• Detect User Obtaining Root Privileges in an Unusual Manner

– Does Not Detect Other Attacks (Crack, Poor Password, ...)

• Found 16 Attacks in 460 Suspicious Sessions

– Marks Line in Transcript Where Attack Occurs

– A Few Hours of Human Examination Found 16 Attacks

– We Hadn’t Seen Most of These Attacks Before

– Missed Two Attacks Where the Transition to Root Was Not in the Transcript (Stolen Password, Telnet Environmental Variable Attack)

TRANSCRIPTS(86,015)

LINCOLN BOTTLENECKVERIFICATION

SUSPICIOUS SESSIONS

(460)

HUMAN ANALYST

16 ILLEGAL ROOT

TRANSITIONS


Reduction in Workload

• Attacks Include All Illegal Root Incidents (18) Found in Four Months (86,015 Transcripts) of Data

• Workload Reduced from 6 Days Per Attack to 30 Minutes– Assumes 1 Minute Per Transcript, 8 Hour Days

BASELINE BOTTLENECKVERIFICATION

0

1000

2000

3000

4000

2,800 TRANSCRIPTS6 DAYS

29 TRANSCRIPTS30 MINUTES

REAL ATTACKS


PERL SUID ROOT ATTACK DISCOVERED BY BOTTLENECK VERIFICATION

[USA@traffic .xfm]$ more magic

#!/usr/bin/perl

$ENV{PATH}="/bin:/usr/bin";

$>=0;$<=0;

exec("/bin/bash");

[USA@traffic .xfm]$ chmod 4700 magic

[USA@traffic .xfm]$ chmod 4744 magic

[USA@traffic .xfm]$ pperl magic

[root@traffic .xfm]

[root@traffic .xfm]# card /var/logs

bash: /var/logs: No such file or directory

[root@traffic .xfm]# w

EXAMINE PERL ATTACK SCRIPT

MAKE SET-ID SCRIPT

RUN SCRIPT

USER IS NOW ROOT!

TRY TO COVER TRACKS

WHO’S WATCHING?


Why Bottleneck Verification Performance Works so Well

1) Parses User Interaction and Expoits Local Context

– Login Sequence, Banner, Shell Prompt, Command, Command Options, and Command Outputs Are Processed Separately

– Most Signature Analysis Systems Inappropriately Look for Attack Strings in Mail Messages, Man Pages, Ls Output, Database Programs, Web Traffic

2) Minimize False Alarm Rates

– Detect Only Breakin Actions

– Most Signature Analysis Systems Do Not Select Strings to Minimize False Alarms

3) Process Only Telnet Shell Sessions

– Many Air Force Sessions Are Special Applications, Not Shell Sessions


Outline


– Normal Data

– Attacks




– Sniffer Based

– Audit Based



Bottleneck Verification Based on Audit Data

• Host Based Intrusion Detection System• Uses Audit Data From Sun’s Basic Security Module (BSM)• Looks for a Privileged Shell Launched From a Non-Privileged

Shell Without an Su• Low-Complexity Real-Time Implementation• Successfully Finds Stealthy Buffer Overflow Illegal Root Transitions

Legal IllegalLegaluser shell

process

user shell

user shell

su

root shell

user shell

attack

root shell


Summary

• A Simple Sniffer-Based Baseline Intrusion Detection System Requires High Estimated Workloads

• Sniffer-Based Bottleneck Verification

– Reduces Estimated Human Workload by Two Orders of Magnitude and Successfully Finds Unknown Attacks

• Audit-Based Bottleneck Verification

– Allows Real-Time Low-Complexity Operation and Finds Stealthy Attacks that Would be Missed by Sniffer

• Planning Further Evaluations and Extensions

– Improve Sniffer Script Parsing

– Detect More Complex Attacks with Audit Data

– Evaluate Using DARPA Intrusion Detection Evaluation Training/Test Data Being Created at Lincoln Laboratory

Using Bottleneck Verification to Find Novel New Attacks ... · Using Bottleneck Verification to...

Documents

Transcript of Using Bottleneck Verification to Find Novel New Attacks ... · Using Bottleneck Verification to...