1 Intruders and Intrusion Detection Behzad Akbari Fall 2009 In the Name of the Most High These...

1

Intruders and Intrusion Detection

Behzad AkbariFall 2009

In the Name of the Most High

These slides are based in parts on slides by Vitaly Shmatikov at Texas University and S. Jane Fritz at SJC

Outline

Intruders Intrusion Detection Techniques

Statistical Anomaly Rule Based Detection

Password management Password Protection Password Selection Strategies

Recommended Reading and WEB Sites

Intruders

Three classes of intruders (hackers or crackers): Masquerader- unauthorized individual who exploits

legitimate user’s account (outsider) Misfeasor- legitimate user, who misuses his or her

privileges (insider) Clandestine user- individual who seizes supervisory

control and uses it to evade auditing or access controls (insider or outsider)

Intruders

Clearly a growing publicized problem from “Wily Hacker” in 1986/87 to clearly escalating CERT stats

May seem benign, but still cost resources May use compromised system to launch

other attacks

Intrusion Detection

If intrusion is detected quickly, intruder can be identified and ejected from system before damage is done

An effective detection system acts as a deterrent, to prevent intrusions

Detection enables the collection of information to strengthen the system.

Intrusion Detection

The goal of an intrusion detection system (IDS) is to detect that bad things are happening… …just as they start happening (hope so) How is this different from a firewall?

Successful attack is usually (but not always) associated with an access control violation A buffer overflow has been exploited, and now attack code is being

executed inside a legitimate program Outsider gained access to a protected resource A program or file has been modified System is not behaving “as it should”

Intrusion Detection Techniques Objective of intruder is to gain access to the

system or to increase range of privlileges System maintains a file that associates a

password with each authorized user. Password file can be protected with:

One-way encryption Access Control

Intrusion Techniques

Aim to increase privileges on system Basic attack methodology

target acquisition and information gathering initial access privilege escalation covering tracks

Key goal often is to acquire passwords Then exercise access rights

Intrusion Techniques

Techniques for guessing passwords:• Try default passwords.• Try all short words, 1 to 3 characters long.• Try all the words in an electronic dictionary(60,000).• Collect information about the user’s hobbies, family

names, birthday, etc.• Try user’s phone number, social security number,

street address, etc.• Try all license plate numbers (MUP103).• Use a Trojan horse• Tap the line between a remote user and the host

system.

Prevention: Enforce good password selection (Ij4Gf4Se%f#)

Profiles of Behavior of Intruders and Authorized Users

Intrusion Detection

Based on assumption that behavior of intruder differs from legitimate user

Statistical anomaly detection threshold detection (frequency of events) profile based (change in activity of user)

Rule based detection anomaly detection (rules detect deviation in behavior

pattern) penetration identification(searches for suspicious

behavior)

Intrusion Detection

Statistical anomaly attempts to define normal or expected behavior and are effective against masqueraders.

Rule based approaches attempt to define proper behavior and are effective against misfeasors

Combination of both are generally used

Misuse Detection(Signature or Rule Based) Set of rules defining a behavioral signature likely to be associated

with attack of a certain type Example: buffer overflow

A setuid program spawns a shell with certain arguments A network packet has lots of NOPs in it Very long argument to a string function

Example: SYN flooding (denial of service) Large number of SYN packets without ACKs coming back …or is this simply a poor network connection?

Attack signatures are usually very specific and may miss variants of known attacks Why not make signatures more general?

Tools for Intrusion Detection

Statistical Anomaly Detection primarily through the analysis of audit records using the following metrics: Counter: eg. number of login attempts, password failures Gauge: value of user connections,applications,

messages Interval timer: length of time between events (eg. logins) Resource Utilization: quantity of resources consumed

(eg. pages printed, time consumed by program execution)

Tools for Intrusion Detection

Audit Records - software that collects information on user activity: Subject: action initiators (user or process) Action: operation performed Object: action receptors ( files, programs, etc.) Exception Condition Resource usage Time stamp

Other Measures Used for Intrusion Detection Login frequency by day and time. Frequency of login at different locations. Time since last login. Password failures at login. Execution frequency. Execution denials. Read, write, create, delete frequency. Failure count for read, write, create and delete.

Statistical Tests

Mean and Standard Deviation- average behavior and its variability

Multivariate Model- correlation between two or more variables

Markov Process- establishes transition probabilities between two or more states

Time Series - focuses on time intervals Operational Model- judgement of what is

abnormal ( See table p. 303)

Extracting Misuse Signatures Use invariant characteristics of known attacks

Bodies of known viruses and worms, port numbers of applications with known buffer overflows, RET addresses of overflow exploits

Hard to handle mutations (e.g., metamorphic viruses)

Big research challenge: fast, automatic extraction of signatures of new attacks

Honeypots are useful for signature extraction Try to attract malicious activity, be an early target

Ross Anderson’s example: dummy hospital records with celebrity names to catch snooping employees

Rule Based Detection

Observe events and apply set of rules as to whether or not they are suspicious, for example: Users should not read other user’s files Users must not write other’s files Users who login after hours usually use files they used

earlier in the day Users generally use system commands to open devices Users should not be logged in more than once to same

system Users should not make copies of system programs

Anomaly Detection

Define a profile describing “normal” behavior Works best for “small”, well-defined systems (single program rather

than huge multi-user OS) Profile may be statistical

Build it manually (this is hard) Use machine learning and data mining techniques

Log system activities for a while, then “train” IDS to recognize normal and abnormal patterns

Risk:attacker trains IDS to accept his activity as normal Daily low-volume port scan may train IDS to accept port

scans IDS flags deviations from the “normal” profile

Intrusion Detection Errors

False negatives: attack is not detected Big problem in rule-based misuse detection

False positives: harmless behavior is classified as an attack Big problem in statistical anomaly detection

Both types of IDS suffer from both error types Which is a bigger problem?

Attacks are fairly rare events IDS often suffer from base-rate fallacy

Conditional Probability

Suppose two events A and B occur with probability Pr(A) and Pr(B), respectively

Let Pr(AB) be probability that both A and B occur What is the conditional probability that A occurs assuming

B has occurred?

Pr(AB)Pr(A | B) = Pr(B)

Bayes’ Theorem

Suppose mutually exclusive events E1, … ,En together cover the entire set of possibilities

Then probability of any event A occurring is

Pr(A) = 1in Pr(A | Ei) Pr(Ei)

Intuition: since E1, … ,En cover entire probability space,

whenever A occurs, some event Ei must have occurred

Can rewrite this formula as Pr(A | Ei) Pr(Ei) Pr(Ei | A) = Pr(A)

Base-Rate Fallacy

1% of traffic is SYN floods; IDS accuracy is 90% IDS classifies a SYN flood as attack with probability

of 90%, classifies a valid connection as attack with probability of 10%

What is the probability that a valid connection is erroneously flagged as a SYN flood by the IDS?

The Stages of a Network Intrusion1. Scan the network to:

• locate which IP addresses are in use, • what operating system is in use, • what TCP or UDP ports are “open” (being listened to by Servers).

2. Run “Exploit” scripts against open ports3. Get access to Shell program which is “suid” (has “root”

privileges).4. Download from Hacker Web site special versions of

systems files that will let Cracker have free access in the future without his cpu time or disk storage space being noticed by auditing programs.

5. Use IRC (Internet Relay Chat) to invite friends to the feast.

26

Where are IDS employed?

Host-based intrusion detection Monitor activity on a single host Advantage: better visibility into behavior of individual

applications running on the host Network-based intrusion detection (NIDS)

Often placed on a router or firewall Monitor traffic, examine packet headers and payloads Advantage: single NIDS can protect many hosts and look

for global patterns

Distributed Intrusion Detection Major issues in design:

Need to deal with different audit record formats One of more nodes in network will serve as

collection points for data, which must then be transmitted securely

Centralized or decentralized architecture can be used

Requires coordination and cooperation

Distributed Intrusion Detection System

Developed at Univ. of California-Davis: Host Agent module: collects data on security

events in hosts and transmits to central manager

LAN monitor agent module: analyzes LAN traffic and reports to central manager

Central manager module: receives reports and correlates them to detect intrusion

Distributed Intrusion Detection

Developed at University of California at Davis

Distributed Intrusion Detection

HONEYPOTS

Relatively recent intrusion detection technology Honeypots are decoy systems designed to lure a

potential attacker away from critical systems: divert attacker from accessing critical systems collect information about attacker’s activities encourages the attacker to stay on the system long enough

for administrators to respond Filled with fabricated information designed to appear

valuable, but which user wouldn’t access

Intrusion Detection Exchange Format

Standards are needed to facilitate IDS that can function across a wide range of platforms and environments

Currently being developed by IETF (Internet Engineering Taskforce) working group

(See http://www.ietf.org )

Host Based IDS

Use OS auditing and monitoring mechanisms to find applications taken over by attacker Log all system events (e.g., file accesses) Monitor shell commands and system calls executed by

user applications and system programs Pay a price in performance if every system call is filtered

Killer application: detect rootkits Con: need an IDS for every machine Con: if attacker takes over machine, can tamper

with IDS binaries and modify audit logs Con: only local view of the attack

RootKit Rootkit is a set of Trojan system binaries

Emerged in 1994, evolved since then

Typical infection path: Use stolen password or dictionary attack to log in Use buffer overflow in rdist, sendmail, loadmodule, rpc.ypupdated, lpr, or passwd to

gain root access Download Rootkit by FTP, unpack, compile and install

Includes a sniffer (to record users’ passwords) Hides its own presence!

Installs hacked binaries for netstat, ps, ls, du, login Modified binaries have same checksum as originalsCan’t detect attacker’s processes,

files or network connections by running standard UNIX commands!

Detecting Rootkit Presence

Sad way to find out Run out of physical disk space because of sniffer logs Logs are invisible because du and ls have been hacked!

Manual confirmation Reinstall clean ps and see what processes are running

Automatic detection Rootkit does not alter the data structures normally used by

netstat, ps, ls, du, ifconfig Host-based intrusion detection can find Rootkit files

…assuming an updated version of Rootkit did not disable your intrusion detection system!

Tripwire File integrity checker

Records hashes of critical files and binaries Recorded hashes must be in read-only memory (why?)

Periodically checks that files have not been modified, verifies sizes, dates, permission

Good for detecting rootkits Can be subverted by a clever rootkit

Install backdoor inside a continuously running system process

(no changes on disk!) Modify database of file attributes Copy old files back into place before Tripwire runs

Network Based IDS

Inspect network traffic For example, use tcpdump to sniff packets on a router Passive (unlike packet-filtering firewalls) Default action: let traffic pass (unlike firewalls)

Watch for protocol violations, unusual connection patterns, attack strings in packet payloads Check packets against rule sets

Con: can’t inspect encrypted traffic (IPSec, VPNs) Con: not all attacks arrive from the network Con: record and process huge amount of traffic

Popular IDS

Snort Most popular open-source tool Large rule sets for known vulnerabilities

Date: 2005-04-05 Synopsis: the Sourcefire Vulnerability Research Team (VRT) has learned of serious vulnerabilities affecting various implementations of Telnet […] Programming errors in the telnet client code from various vendors may present an attacker with the opportunity to overflow a fixed length buffer […] Rules to detect attacks against this vulnerability are included in this rule pack

Bro (www.bro-ids.org) Developed by Vern Paxson at Lawrence Berkeley Labs Separates data collection and security decisions

Event Engine distills the packet stream into high-level events describing what’s happening on the network

Policy Script Interpeter uses a script defining the network’s security policy to decide what to do in response

Detecting Backdoors with NIDS

Look for telltale signs of sniffer and rootkit activity Entrap sniffers into revealing themselves

Use bogus IP addresses and username/password pairs; open bogus TCP connections, then measure ping times Sniffer may try a reverse DNS query on the planted address; rootkit

may try to log in with the planted username If sniffer is active, latency will increase

Clever sniffer can use these to detect NIDS presence! Detect attacker returning to his backdoor

Small packets with large inter-arrival times Simply search for root shell prompt “# ” (!!)

Attacks on Network-Based IDS

Overload NIDS with huge data streams, then attempt the intrusion Bro solution: watchdog timer

Check that all packets are processed by Bro within T seconds; if not, terminate Bro, use tcpdump to log all subsequent traffic

Hide malicious data, split into multiple packets NIDS does not have full TCP state and does not always

understand every command of receiving application Simple example: send “ROB<DEL><BS><BS>OT”,

receiving application may reassemble to “ROOT”

Detecting Attack Strings

Want to detect “USER root” in packet stream Scanning for it in every packet is not enough

Attacker can split attack string into several packets; this will defeat stateless NIDS

Recording previous packet’s text is not enough Attacker can send packets out of order

Full reassembly of TCP state is not enough Attacker can use TCP tricks so that certain packets

are seen by NIDS but dropped by the receiving application Manipulate checksums, TTL (time-to-live), fragmentation

E

TCP Attacks on NIDS

Insertion attack

NIDS

U S R r X o o t

Insert packet with

bogus checksum

EU S R r

X

o o t

Dropped

E

TTL attack

NIDS

U S R r

X

o o t

EU S R r

X

o o t

10 hops 8 hops

TTL=20

TTL=12

Short TTL to ensure this packet

doesn’t reach destination

TTL=20Dropped (TTL

expired)

Intrusion Detection Summary

No bullet-proof solutions, constant arms race Increasing diversity of traffic = challenge for NIDS

Lots of anomalous, but benign junk Vern Paxson on stuff they’ve seen on a DMZ:

Storms of 10,000+ FIN or RST packets due to TCP bugs Horrible fragmentation TCPs that acknowledge data that was never sent TCPs that retransmit different data from what was sent

False alarms are THE problem for IDS “The Boy Who Cried Wolf” (base-rate fallacy) Can’t flag every anomaly as an attack

Storing UNIX Passwords

UNIX passwords were kept in in a publicly readable file, etc/passwords.

Now they are kept in a “shadow” directory and only visible by “root”.

Password Management

Frontline of defense against intruders is the password system:

User ID - determines if user is authorized to gain access, and determines the privileges accorded to user

Password authenticates the ID of the individual

Managing Passwords Need policies and good user education Ensure every account has a default password Ensure users change the default passwords to

something they can remember Protect password file from general access Set technical policies to enforce good passwords

minimum length (>6) require a mix of upper & lower case letters, numbers,

punctuation block know dictionary words

Managing Passwords

May reactively run password guessing tools note that good dictionaries exist for almost any

language/interest group May enforce periodic changing of passwords Have system monitor failed login attempts, &

lockout account if see too many in a short period Need to educate users and get support Balance requirements with user acceptance Be aware of social engineering attacks

Proactive Password Checking

Most promising approach to improving password security

Allow users to select own password But have system verify it is acceptable

simple rule enforcement (see previous slide) compare against dictionary of bad passwords use algorithmic (markov model or bloom filter) to

detect poor choices

UNIX Password System

User selects password ( 5-8 characters) Converted to 56 bit value used as key to encryption

routine- crypt 3- based on DES Modified using a 12 bit “salt” value- related to time

at which password was assigned prevents duplicates from being visible in password file increases length of password prevents use of hardware implementation of DES

Output encrypted 25 times more

UNIX Password Scheme

Loading a new password

UNIX Password Scheme

Verifying a password file

Threats to Password Files

Encryption scheme is designed to discourage guessing but users can gain access on a machine using a guest

account and run a password guessing program or “cracker”

if opponent can gain access to password file, cracker can be run on another machine

Cracker programs are getting better and hardware executes them faster…. ( See p. 312)

Password Capture

Another attack involves password capture watching over shoulder as password is entered using a trojan horse program to collect monitoring an insecure network login (eg. telnet, FTP,

web, email) extracting recorded info after successful login (web

history/cache, last number dialed etc) Using valid login/password can impersonate user Users need to be educated to use suitable

precautions/countermeasures

Password Guessing One of the most common attacks Attacker knows a login (from email/web page etc) Then attempts to guess password for it

try default passwords shipped with systems try all short passwords then try by searching dictionaries of common words intelligent searches try passwords associated with the user

(variations on names, birthday, phone, common words/interests) before exhaustively searching all possible passwords

Check by login attempt or against stolen password file Success depends on password chosen by user Surveys show many users choose poorly

Password Selecting Strategies User education -

must be long and complex enough many choose password <= 3 characters not easily guessable

Computer-generated passwords - hard to remember

Reactive password checking- randomly checks, cancels ones that are guessed

Proactive password checking- check at time of creation, reject any too simple

Proactive Password Checkers

Two techniques for rejecting words on a list show promise:

Markov Model- based on structure of passwords

Spafford- based on use of a Bloom Filter

Markov Model

Example shows a language consisting of 3 characters (as a finite state machine) and the probability of the letters in a sequence.

The state at any time is the most recent letter The value on the transition is the probability that

one letter follows another Markov model is a quadruple [m, A,T,k], where m

is number of states, A is state space, T is matrix of transitional probabilities and k is the order of the model.

Markov Model

Transition Matrix

For a second order model- determine the frequency matrix f, where f(i,j,k) is the number of occurrences of the trigram consisting of the ith, jth and kth character.

For each bigram ij, calculate f(i,j, ) as the total number of trigrams beginning with ij.

Compute the entries of T as follows:

),,(),,(

),,( jifkjif

kjiT

Markov Model

Resulting model reflects the structure of the words in a dictionary. “Is this a bad password?” becomes “Can this string be generated by the Markov Model?” Statistical test can be done to see if the password is

likely and, if so, it is rejected.

Bloom Filter

A Bloom Filter of order k consists of a set of k independent hash functions, H1(x), H2(x)… Hk(x), where each function maps a password into a hash value in the ranger 0 to N-1.

Spafford (Bloom Filter)

where

10;1;1)( NyDjkiyXH ji

dictionarypasswordinwordofnumberD

dictionarypasswordinwordjthXj

The following procedure is then applied to the dictionary:

1. A hash table of N bits is defined, with all bits initially set to 0.

2. For each password, its k hash values are calculated, and the corresponding bits in the hash table are set to 1

3. When a password is presented, its hash values are calculated and if all bits =1 it is rejected.

Spafford (Bloom Filter)

Design the hash scheme to minimize false positive. Probability of false positive:

)()(,/

)1ln(

,,

)1()1(

/1

//

wordssizedictionarytobitssizetablehashofratioDNR

dictionaryinwordsofnumberD

tablehashinbitsofnumberN

functionhashofnumberk

where

P

kR

lyequivalentor

eeP

k

kRkkNkD

Performance of Bloom Filter

Graph plots the probability P as a function of R ( ratio of the Hash table size to dictionary size.

Can achieve a compression of a factor of about 7 ( hash table storage- 1.2MB for an 8MB dictionary)

Password checking involves computing 6 hash functions, independent of size of dictionary and without substantial searching

Performance of Bloom Filter

Reading Assignment

Appendix 9A in Stallings Explains the base-rate fallacy

Optional: “Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection” by Ptacek and Newsham Reference

Recommended Reading and WEB Sites

Denning, P. Computers Under Attack: Intruders, Worms, and Viruses. Addison-Wesley, 1990

CERT Coordination Center (WEB Site) AntiVirus Online (IBM’s site)

1 Intruders and Intrusion Detection Behzad Akbari Fall 2009 In the Name of the Most High These...

Documents

Transcript of 1 Intruders and Intrusion Detection Behzad Akbari Fall 2009 In the Name of the Most High These...