Using Traffic Shaping to Combat Spam

David Cawley, Senior Engineer

December 12th, 2007

Overview

1. Evolution of E-mail & Spam

2. Spamonomics

3. SMTP Multiplexing

4. Traffic Shaping

5. Asynchronous IO

6. Passive OS Fingerprinting

The Dawn of E-mail

• 1965 MIT shared mainframe

• 1971 The @ symbol

• 1976 Queen of England sends an e-mail

• 1982 IETF RFC821/822

• 1989 Lotus Notes released (35k copies sold)

• 1996 Microsoft Internet Mail 1.0

• 2001 IETF RFC2821/2822

Attempts to secure...

• SMTP is inherently insecure

• SMTP-Auth/TLS

• SPF

• Sender-ID

• Why it didn't stop spam

The Evolution of Spam

• 1978 The first spam

• 1988 Usenet cross-posting

• 1993 “spam” coined as a name

• 1997 Open Relays abused

• 2000 Birth of Nigerian spam

• 2001 Formail exploit

• 2003 Sobig virus sends spam

The Evolution of Spam

• 2003 CAN-SPAM act

• 2004 Bill gates prediction & botnets

• 2005 Image spam, Ascii art

• 2006 Animated images, flash, pdf

• 2007 mp3, excel, p2p botnets

The escalating spam problem

Source: spamnation.info/stats

The good old days.

Spammer Economics

• 0.02% people click and buy [source: NY Times]

• Average filter effectiveness is 90%

– 1/10 of spam messages get through

• Improve effectiveness to 95%

– 1/20 of spam messages get through

• Spammer Solution?

– Double spam volume

– Same profit

Traditional Filtering

• MD5's, Fuzzy Signatures, Bayesian

• Header Regex, RBL's, URL Lists, Grey Listing

• Problems

– Obfuscation Techniques

– Formats – html, image, pdf, doc, xls, ole, mp3..

– Zombies, Botnets

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 300

100000

200000

300000

400000

500000

600000

700000

800000

How often do we see a unique Botnet IP?

The Number of Unique IP's versus the number of times reported

# Times Reported

# U

niq

ue

Bo

tne

t IP

's

SMTP Multiplexing

• Transparent SMTP Proxy

• Connection Pooling

• Insulates the MTA

• Avoids delay of legitimate mail

• High Concurrency

– Up to 10,000 simultaneous connections

12

Traffic Shaping

• What can we do?

• Provide a Quality of Service

• Reputation Network

• Throttle unknown senders

• Fast track legitimate senders

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 50 100 150 200 250 300 350 400 450

Perc

enta

ge o

f Con

necti

ons S

till C

onne

cted

Time (Seconds)

Spammers are Less Patient than Legitimate Senders

Spammers

Legitimate Senders

16

Does Sendmail Throttle?

ratecontrol

ConnectionRateThrottle

conncontrol

Asynchronous IO

• Non-Blocking front end

• Blocking Back-end

• Event driven

• Finite State Machine

• Management of Resources

Passive OS Fingerprinting

1.Look at IP packet data

2.Determine the Operating System

3.Decision to Throttle

OS Comparison

Delivered

Windows

Linux

FreeBSD

Solaris

Novell

HP

NetCache

Not delivered

Windows

Linux

FreeBSD

Solaris

Novell

HP

NetCache

Conclusions

1.Spamming is driven by economics

2.Botnet operators need to make money

3.Slowing down spam makes it go away

questions@mailchannels.com

Nick Shelness, Former CTO, Lotus:“I am able to report that I have been running an instance of

TrafficControl in my own network for four months, and that it has reduced the volume of spam hitting my boundary MTAs on most days

by approximately 95%.”

+1-778-785-6143

www.mailchannels.com