Antispam activities @ GARR Michele Michelotto Hepix Karlsruhe, 11 May 2005.

Antispam activities Antispam activities @ GARR@ GARR

Michele MichelottoMichele Michelotto

HepixHepix

Karlsruhe, 11 May 2005Karlsruhe, 11 May 2005

HEPIX, Karlsruhe 11/5/05 2Antispam activities at GARR

WG sec mailWG sec mail Enrico Ardizzoni (Università di Ferrara)Enrico Ardizzoni (Università di Ferrara) Alberto D’Ambrosio (INFN, Torino)Alberto D’Ambrosio (INFN, Torino) Roberto Cecchini (INFN, Firenze)Roberto Cecchini (INFN, Firenze) Fulvia Costa (INFN, Padova)Fulvia Costa (INFN, Padova) Giacomo Fazio (INAF, Palermo)Giacomo Fazio (INAF, Palermo) Antonio Forte (INFN, Roma 1)Antonio Forte (INFN, Roma 1) Matteo Genghini (IASF, Bologna)Matteo Genghini (IASF, Bologna) Michele Michelotto (INFN, Padova)Michele Michelotto (INFN, Padova) Ombretta Pinazza (INFN, Bologna)Ombretta Pinazza (INFN, Bologna) Alessandro Spanu (INFN, Roma 1)Alessandro Spanu (INFN, Roma 1) Alfonso Sparano (Università di Salerno)Alfonso Sparano (Università di Salerno)


GoalsGoals anti-spam and anti-virusanti-spam and anti-virus

Stop them or at least reduce to a Stop them or at least reduce to a reasonable levelreasonable level

““best practices”best practices” mail services configuration and mail mail services configuration and mail

server protectionserver protection Sender authenticationSender authentication

SPF, domain keysSPF, domain keys DisseminationDissemination

http://www.garr.it/WG/sec-mailhttp://www.garr.it/WG/sec-mail mailto:<[email protected]>mailto:<[email protected]>


anti-spamanti-spam SpamAssassin (SA) analysis and SpamAssassin (SA) analysis and

efficiency improvement:efficiency improvement: Monitoring;Monitoring; Bayesian filter;Bayesian filter; Real Time Block List (RBL);Real Time Block List (RBL); Network distributed “cooperative” Network distributed “cooperative”

systems.systems.


anti-spamanti-spam Alternative tools tests:Alternative tools tests:

Bogofilter: Bogofilter: http://bogofilter.sourceforge.net/http://bogofilter.sourceforge.net/

DSPAM:DSPAM:http://www.nuclearelephant.com/projects/http://www.nuclearelephant.com/projects/dspamdspam


SpamAssassinSpamAssassin

Rule basedRule based Each rule adds a score (positive Each rule adds a score (positive

or negative)or negative) Mail over threshold can be Mail over threshold can be

deleted, marked, moved to a deleted, marked, moved to a quarantine folderquarantine folder

Choice of threshold is difficultChoice of threshold is difficult Some spam have a score lower Some spam have a score lower

than legitimate mail (ham)than legitimate mail (ham)

HEPIX, Karlsruhe 11/5/05 7Antispam activities at GARRDove metto la soglia?Threshold too high – Many FALSE NEGATIVES

Two weeks

275417 e-mails

208436 spams (75.7%)

HEPIX, Karlsruhe 11/5/05 8Antispam activities at GARRDove metto la soglia?

Threshold too low – Some FALSE POSITIVES (Dangerous)

Two weeks

275417 e-mails

208436 spams (75.7%)


Indipendent methodsIndipendent methods Improve the spam/ham Improve the spam/ham

identification identification I can’t move the thresholdI can’t move the threshold

If I lower it I get too many False If I lower it I get too many False NegativesNegatives

If I raises is even worse because I can If I raises is even worse because I can get some False Positivesget some False Positives

Look for “indipendent methods”Look for “indipendent methods” Bayesian FiltersBayesian Filters Cooperative methodsCooperative methods RBLRBL


Bayesian FiltersBayesian Filters Based on Bayesian statisticsBased on Bayesian statistics The filters “learn” which words (actually The filters “learn” which words (actually

tokens) are more probable in ham and tokens) are more probable in ham and spamspam

Bayesian filters ageingBayesian filters ageing Learning by manually submitting ham Learning by manually submitting ham

spam sample is time consumingspam sample is time consuming Auto Learning is dangerous. Spammers Auto Learning is dangerous. Spammers

send mail designed to “poison” the filterssend mail designed to “poison” the filters Best performance with frequents update Best performance with frequents update

submitted by the userssubmitted by the users Even better: different databases for each Even better: different databases for each

useruser


Bayesian FiltersBayesian Filters

Filters “ageing”: must keep them Filters “ageing”: must keep them up to date.up to date.

Manual update is time expensiveManual update is time expensive Frequents update from selected Frequents update from selected

samples chosen by users, best with samples chosen by users, best with individual db for each user.individual db for each user.

Automatic update is dangerousAutomatic update is dangerous Some mail sent only for bayesing Some mail sent only for bayesing

filter “poisoning”.filter “poisoning”.


ageingageing

0

10

20

30

40

50

60

70

80

90

100

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

% Spam

% + Bayes

% Bayes

Primario

AGEING

NEW TRAINING


Real-Time Block ListReal-Time Block List For each e-mail a DNS query is issued For each e-mail a DNS query is issued

to see if the sender is present in a list to see if the sender is present in a list of known spammerof known spammer

Good method to add scoreGood method to add score Don’t use to reject mail Don’t use to reject mail

Spoofing of senderSpoofing of sender Some RBL not very accurate in checking if Some RBL not very accurate in checking if

sender is a real spammer or in removing sender is a real spammer or in removing those who fixed the problemthose who fixed the problem

URIRBL: Very good because the check URIRBL: Very good because the check is done against the URL in the mail is done against the URL in the mail bodybody The spammer will not spoof the URL in the The spammer will not spoof the URL in the

body !!!body !!!


Cooperative methodsCooperative methods UBE: Unsolicited Bulk EmailUBE: Unsolicited Bulk Email Based on the Mass Diffusion of spamBased on the Mass Diffusion of spam

Razor:Razor: Users submit spam to a network of Razor Users submit spam to a network of Razor

server.server. Mail with many submission tagged as spamMail with many submission tagged as spam Users ratingUsers rating Closed protocol and closed server networkClosed protocol and closed server network

Pyzor:Pyzor: Similar to Razor but protocol and sw is open Similar to Razor but protocol and sw is open

source and you can became a serversource and you can became a server


DCCDCC Mail with similar signature are Mail with similar signature are

counted in several sitescounted in several sites If a mail is seen by many DCC If a mail is seen by many DCC

server is tagged as suspectserver is tagged as suspect Open NetworkOpen Network Our group now has 3 DCC Our group now has 3 DCC

ServersServers Each server can provide Each server can provide

anonymous access or high anonymous access or high priority access to registered userpriority access to registered user


Dcc statsDcc stats


DCC: our statsDCC: our stats A tipical day at the A tipical day at the

DCC server at IASF DCC server at IASF in Palermoin Palermo 800k checksum 800k checksum

request (70k request (70k from registered from registered clients)clients)

1.2M report 1.2M report from 25000 from 25000 clientsclients

Average response Average response time 5mstime 5ms


Spam in September 04Spam in September 04

SpamScore

0

50

100

150

200

250

-1 4 9 14 19 24 29 34 39 44 49

Score

Co

un

t

Series1

5000 spam received in my mailbox during the CHEP week

12% False Negatives


Spam in September 04Spam in September 04

SpamScore

0.00%

1.00%

2.00%

3.00%

4.00%

5.00%

6.00%

7.00%

8.00%

9.00%

-1.0 4.0 9.0 14.0 19.0 24.0 29.0 34.0 39.0 44.0 49.0

Score

Co

un

t Series1

Series5

From 12% at the end of September to 1.7% False Negatives at end of November


Monitoring trendMonitoring trend


Top pluginTop plugin


Sender AuthenticationSender Authentication

Sender Policy Framework (SPF):Sender Policy Framework (SPF): Each DSN server should publish a Each DSN server should publish a

“reverse MX record” DNS listing the “reverse MX record” DNS listing the smtp server smtp server autorizedautorized to send email to send email for that domainfor that domain

The receiver can use this information The receiver can use this information to reject mail or to increase SA scoreto reject mail or to increase SA score

This means that the roaming users This means that the roaming users should always use his own SMTP should always use his own SMTP server (after authentication)server (after authentication)


SPF testsSPF tests

Salerno UniversitySalerno University One monthOne month 650 650 · · 101033 mail mail 32% from SPF compliant domain32% from SPF compliant domain

12% esternal12% esternal 20% internal (useful to cut all the spam 20% internal (useful to cut all the spam

with faked internal sender, mostly virus with faked internal sender, mostly virus or phishing)or phishing)


Best practicesBest practices Open port 25 only to your site email Open port 25 only to your site email

serverserver Open ports 587 and 468 for external Open ports 587 and 468 for external

authenticated usersauthenticated users Force external users authentication Force external users authentication

(necessary to implement SPF)(necessary to implement SPF) Antivirus configuration to avoid Antivirus configuration to avoid

sender notification (since is almost sender notification (since is almost always spoofed)always spoofed)

““greet pause” on greet pause” on sendmailsendmail (≥ 8.13) (≥ 8.13)


Open itemOpen item ““unofficial” plugin testunofficial” plugin test Sender AuthenticationSender Authentication Bogofilter and dspam testsBogofilter and dspam tests More DCC or Pyzor server?More DCC or Pyzor server? Online filter (spam rejection)?Online filter (spam rejection)? Close group and buy commercial Close group and buy commercial

“turnkey” sw ?“turnkey” sw ? Like we do with A/V Like we do with A/V (e.g. Sophos PureMessage)(e.g. Sophos PureMessage)


Questions?Questions?

Antispam activities @ GARR Michele Michelotto Hepix Karlsruhe, 11 May 2005.

Documents

Transcript of Antispam activities @ GARR Michele Michelotto Hepix Karlsruhe, 11 May 2005.