An Introduction of Botnet Detection – Part 2

Guofei Gu, Wenke Lee (Georiga Tech)

2009/5/26 Speaker: Li-Ming Chen 2

Reference

Guofei Gu, Wenke Lee, et al. BotHunter: Detecting Malware Infection through IDS-driven Dial

og Correlation USENIX Security 2007

BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic ACM NDSS 2008

BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-independent Botnet Detection USENIX Security 2008

Moheeb Abu Rajab, et al. A Multifaceted Approach to Understanding the Botnet Phenomen

on ACM IMC 2006

2009/5/26 Speaker: Li-Ming Chen 3

Lifecycle of a Typical Botnet Infection

Why Botnet is hard to detect?• involving multiple steps• flexible design of C&C

channels

6. Malicious activities (e.g., DDoS)(borrow infection strategies from traditional malicious attacks)

(optional)

authentication

2009/5/26 Speaker: Li-Ming Chen 4

C&C (Command and Control) Channels

Centralized C&C channel

P2P C&C channel

Message Response Crowd

Activity Response Crowd

2009/5/26 Speaker: Li-Ming Chen 5

Comparison of the 3 ApproachesBotHunter BotSniffer BotMiner

Detection Target

Bot Botnet Botnet

Description Detect the lifecycle of a bot, including infection and command execution

Detect group of hosts with spatial-temporal similarity in C&C communication

BotSniffer extension.

Support various C&C comm. framework.

Assumptions Predefine bot infection lifecycle

Focus on centralized C&C communication

Bots will perform tasks and response

Insight Vertical correlation of IDS alerts

Horizontal correlation of similar behaviors

Cluster hosts with similar traffic patterns

Approach detect individual events identify parts of the lifecycle

group hosts connect to the same C&C server detect similar activity or message response behaviors

cluster similar C&C comm. cluster similar malicious traffic. cross clustering

2009/5/26 Speaker: Li-Ming Chen 6

BotHunter

Utilize Snort to detect sign of local infection

Signs matchthe predefinedevidences (dialogtransitions)

A Bot could be:• E2 AND E3-E5• At least two distinct

signs of E3-E5

Predefined Lifecycle

2009/5/26 Speaker: Li-Ming Chen 7

BotHunter (cont’d)

• Current bots are multi-vector• Design two modules (inbound/outbound) for scan detection• Assign high weight to ports often used by malware (predefined)• Observe outbound scan rate, outbound connection failure rate, and address dispersion

• Anomaly-based payload exploit detection• Learn normal profile (using 2-gram PAYL)• Check deviation distance of a test payload from the normal profile

• Use bot-specific heuristics to build signatures (rules)

2009/5/26 Speaker: Li-Ming Chen 8

BotHunter:Evaluation Results (1/2) Experiments in a virtual network

To test FN rate (by examining 10 different bots)# of generated dialog warnings

# involving the victim

2009/5/26 Speaker: Li-Ming Chen 9

BotHunter:Evaluation Results (2/2) Honeynet-based experiments

Use SRI honeynet to capture real-world bot infection Use BotHunter to analysis these traces 95.1% TP rate (1920/2019 in 3 weeks) FN is due to:

Infection failure, honeynet setup and policy failure, data corruption failure.

Experiments in a campus network 98 profiles were generated in 4 months (no FP)

Experiments in SRI laboratory network Generate 1 bot profile and it is FP (a 1.6 GB multifile FTP transfer

matchs “E2 & E3”)

2009/5/26 Speaker: Li-Ming Chen 10

BotHunter:Pros and Cons Pros:

Real-time detection of bot infections Evidence trail gathering for investigation of putative inf

ections Cons:

Use heuristic (2 conditions) to decide a bot infection Less flexible

2009/5/26 Speaker: Li-Ming Chen 11

BotSniffer

Response crowd:• Density check• Homogeneity check

(data reduction)

Port-independent,payload inspection

2009/5/26 Speaker: Li-Ming Chen 12

BotSniffer:Evaluation Methodology Use normal traffic traces to test the FP rate and

use botnet traces (mix normal traffic) to test the detection performance

Normal traces: Capture 8 IRC traces (port 6667) and 5 complete trace

s from campus network Botnet traces:

Collect 3 real-world IRC-based botnet traces Generate 3 botnet traffic by modifying source codes of

3 common botnets Implement 2 http-based botnet

2009/5/26 Speaker: Li-Ming Chen 13

BotSniffer:Evaluation Results (1/2)All FP are generated due to

single client incoming messageresponse analysis.

(Apply both activity response and message response group analysis)

2009/5/26 Speaker: Li-Ming Chen 14

BotSniffer:Evaluation Results (2/2)

honeynet

IRC logs

(both messageand activity)

(periodically connect to server)

(random delay)

(the randomization of connection periods did not cause a problem, becausethere were still several clients performing activity responses at the time window)

2009/5/26 Speaker: Li-Ming Chen 15

BotSniffer:Pros and Cons Pros

Successfully detect all botnets (low FP rate) Efficient alert reduction More robust than other botnet detection system

Cons Focus on centralized C&C communication Configure time window for group analysis Possible evasions (e.g., misusing whitelist, encryption,

protocol matcher, long response delay, obfuscation)

2009/5/26 Speaker: Li-Ming Chen 16

BotMiner (similar to BotSniffer)

Focus on flow statistics, not message response!

log

log

• Combine results and make final decision

(more straightforward)

(more complex)

2009/5/26 Speaker: Li-Ming Chen 17

BotMiner: Evaluation Methodology (same) use normal traffic traces to test the FP ra

te and use botnet traces (mix normal traffic) to test the detection performance

Normal traces: Capture 10 days traffic record at the campus network

Botnet traces: 4 IRC, 2 HTTP and 2 P2P botnets

2 IRC and 2 HTTP are also used for BotSniffer P2P: 2 real-world traces (Nugache and Storm)

TCP, encrypted UDP

2009/5/26 Speaker: Li-Ming Chen 18

BotMiner: Evaluation Results (1/3) (C-plan data reduction)

Most useful,Only record internal toexternal flows. Remove

helf-openTCP flows Whitelist

2009/5/26 Speaker: Li-Ming Chen 19

BotMiner: Evaluation Results (2/3)

4 features:• temporal – fph, bps• spatial – ppf, bpp

Cluster by using themean and varianceof the features

Further cluster by separatingeach feature as a vector of13 elements according to their distribution

Ignore clusters only contain 1 host

Most FP clusters containonly 2 hosts

2009/5/26 Speaker: Li-Ming Chen 20

BotMiner: Evaluation Results (3/3)

FN

2009/5/26 Speaker: Li-Ming Chen 21

BotMiner:Pros and Cons Pros:

Anomaly-based botnet detection system (independent of the protocol and structure used by botnets)

Low FN and FP rate Cons:

Stealthy: botmaster can commond the bots to perform extremely delayed task (evade cross clustering)

2009/5/26 Speaker: Li-Ming Chen 22

Summary

Bothunter: Vertical Correlation Correlation on the behaviors of single host

Botsniffer: Horizontal Correlation Focus on centralized C&C botnets

Botminer: Extension on Botsniffer No limitations on the C&C types.