Studying Spamming Botnets Using Botlab

John P. John, Alexander Moshchuk, Steven D. Gribble, Arvind Krishnamurthy

[John 2009] John, John P., Alexander Moshchuk, Steven D. Gribble, and Arvind Krishnamurthy. "Studying Spamming Botnets Using Botlab." In NSDI,

vol. 9, pp. 291-306. 2009.

Presented by Sharan Dhanala

2

Background on the Botnet Threat

• A botnet is a large-scale, coordinated network of computers, each of which executes specific bot software.

• Botnet operators recruit new nodes by taking control of the victim hosts and secretly installing bot code onto them.

• The resulting army of “zombie” computers is typically controlled by one or more command-and-control (C&C) servers.

• Botnets have become more sophisticated and complex in how they recruit new victims and mask their presence from detection systems 1. Propagation

2. Customized C&C protocols 3. Rapid evolution

3

Botlab architecture

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/architecture.png]

4

The Botlab Monitoring platform

• Botlab’s design was motivated by four requirements:o Attributiono Adaptationo Immediacyo Safety

• Incoming Spam o On an average, UW receives 2.5 million e-mail messages each day, over 90% of

which is classified as spam.

• Malware Collectiono Botlab crawls URLs found in its incoming spam feed.o Botlab periodically crawls binaries or URLs contained in public malware repositories

or collected by MWCollect Alliance honeypots.

5

• Botlab executes spamming bots within sandboxes to monitor botnet behavior.

• Network fingerprintingo Each flow record <protocol, IP address, DNS address, port>

o Similarity coefficient of two binaries B1 and B2

o If similarity coefficient of two binaries is high then the binaries are to be behavioural duplicates.

The Botlab Monitoring platform

Identifying spamming bots

6

The Botlab Monitoring platformIdentifying spamming bots

• Safely generating fingerprints o Tight rope between safety & effectiveness.

o Human operator with tools that act as safety net.

o Redirect traffic to spamhole.

• Experience classifying bots o Bots that detect VM & bare-metal.

o Bots checking domain name- required modifying spamhole.

o Bots perform comprehensive SMTP verification.

7

The Botlab Monitoring platformExecution Engine

• Seven spamming bots:o Grum, Kraken, MegaD, Pushdo, Rustock, Srizbi, and Storm.

• Avoiding blacklisting o anonymizing “Tor” (The Onion router) network

• Multiple C&C serverso C&C redundancy mechanism

Image source: http://www.hotforsecurity.com

/images/zombie_network.jpg

8

The Botlab Monitoring platformCorrelation analyzer

• Correlate incoming spam with outgoing spam and perform attribution; identify IPs for a given botnet.

• For spam that cannot be directly attributed, cluster based on source IPs and merge with an attributed set if there is overlap.

9

Analysis• Examine the actions of the bots being run in Botlab – Outgoing

Spam.

• Analyse the incoming spam feed.

• Analysis obtained out of studying both the outgoing and incoming spam feeds.

10

Behavioural characteristics

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/six_botnets_summary.png]

Analysis

11

AnalysisAnalysis of outgoing spam

• Outgoing spam feeds

o Size of mailing lists

• Using the outgoing spam feeds to estimate the size of the botnets’ recipient lists.o A bot periodically obtains a new chunk of recipients from the master and

sends spam to this recipient list.

oOn each such request, the chunk of recipients is selected uniformly at random from the spam list.

o The chunk of recipients received by a bot is much smaller than the spam list size.

12

AnalysisAnalysis of outgoing spam

• Outgoing spam feedso Overlap in mailing lists

• They also examined whether botnets systematically share parts of their spam lists.

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_pahn/john_html/figs/spamlist_overlap.png]

13

AnalysisAnalysis of outgoing spam

• Outgoing spam feedso Spam subjects

• Between any two spam botnets, there is no overlap in subjects sent within a given day, and an average overlap of 0.3% during the length of their study.

• Subject-based classification.

14

AnalysisAnalysis of Incoming Spam

• Analysis of Incoming Spam

o Analysed 46 million spam messages obtained from a 50 day trace.o University of Washington’s filtering systems :o 89.2% of incoming mail as spamo 0.5% of spam contain viruses as attachments.o 95% of the spam messages contain HTTP links.o 1% contain links to executables.

15

• Spam campaigns and Web hosting

o They cluster spam based on the following attributes• The domain names appearing in the URLs found

in spam.

• The content of Web pages linked to by the URLs.

• The resolved IP addresses of the machines hosting this content.

AnalysisAnalysis of Incoming Spam

Imagesource:[https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/num_distinct_hosts.png]

Imagesource:https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/num_messages_per_cluster.png]

16

• Spam classification

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/spam_classification.png]

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/spam_breakdown_per_botnet_over_time.png]

AnalysisCorrelation analysis

17

AnalysisCorrelation analysis

• Spam campaigns

Image source:[https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/classification_by_campaign_nobold.png]

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/hosting_infrastructure_overlap.png]

18

AnalysisCorrelation analysis

• Recruiting campaigns

Image source: [https://www.usenix.org/legacy/event/nsdi09/tech/full_papers/john/john_html/figs/propagation_campaigns.png]

19

Applications enabled by Botlab

• Safer web browsingo They have found 40K malicious URLs propagated by Srizbio None of them were in malware DBs (Google, etc.)o Further Gmail’s spam filtering rate was only 21% for Srizbi.o BotLab can generate malware list in real-time; they have developed a Firefox

plugin to check against this

• Spam filteringo Developed a Thunderbird extension that compares an incoming email with the

list of spam subjects and list of URLs being propagated by captive botso Preliminary results are promising

• Availability of Botlab Data- http://botlab.org/

20

Critics about the paper• "Relying on anti-virus software is also impractical, as these tools do not detect many new

malware variants." was mentioned in the paper yet they used anti-virus tools to validate their duplicate binaries elimination procedure.

• Botnets are continuous evolving and it is going to be quite hard to conduct safe experiments.

• More ways of monitoring can be done on the application layer. In addition to monitoring attachments and message headers, monitoring the text content of email can also be facilitated for spam monitoring.

• There are some entities in Botlab that needs human operators. This doesn't completely eradicate the human interference. Would be exciting to see a fully automated tool.

• Paper can be considered as a basis for building a more powerful tool for spam filtering.

21

Conclusion• Described Botlab, a real-time botnet monitoring system.

• Behaviour and classification of botnets.

• My critics on the paper.

22

Thank you