Stop Spam by using SpamAssassin and Procmail ! Mark Kushinsky – MDS Computer Solutions...

Stop SpamStop Spamby using SpamAssassin and Procmail !by using SpamAssassin and Procmail !

Mark Kushinsky – MDS Computer SolutionsMark Kushinsky – MDS Computer [email protected]@mdspc.com

Mark Kushinsky - mdspc.com - [email protected] 2

What is Spam?What is Spam?

According to dictionary.com Spam is : Unsolicited e-mail, often of a commercial nature, sent

indiscriminately to multiple mailing lists, individuals, or newsgroups; junk e-mail.

A trademark used for a canned meat product consisting primarily of chopped pork pressed into a loaf.

To crash a program by overrunning a fixed-size buffer with excessively large input data.


Why Bother to filter Spam ?Why Bother to filter Spam ?

Depending on who you ask Spam is about 30% to 60% of ALL email and growing! (January 2004 - 60%, December 2003 - 58%, November 2003 – 56%, brightmail.com)

Users are forced to waste time clearing out inboxes, which cost everyone money.

Legal “hostile work environment” ramifications due to the offensive nature of most spam.


What is SpamAssassin ?What is SpamAssassin ?(http://www.spamassassin.org/full/2.6x/dist/README)(http://www.spamassassin.org/full/2.6x/dist/README)

SpamAssassin is a mail filter which attempts to identify spam using text analysis and several Internet based real time blacklists.

Using its rule base, it uses a wide range of tests on mail headers and body text to identify "spam", also known as unsolicited commercial email. Once identified, the mail can then be optionally tagged as spam for later filtering using the user's own mail user-agent application.

SpamAssassin typically differentiates successfully between spam and non-spam in between 95% and 99% of cases, depending on what kind of mail you get.

SpamAssassin also includes support for reporting spam messages automatically, and/or manually, to collaborative filtering databases such as Vipul's Razor(http://razor.sourceforge.net/) .

Latest version 2.63 now includes a daemonized version, which runs persistently. This allows an MTA to process large volumes of mail through SpamAssassin without having to fork/exec a perl interpreter for each one. IOW low CPU utilization for high volume mail servers.

SpamAssassin lives at http://spamassassin.org/ or in CPAN.


What do I need to run What do I need to run SpamAssassin? SpamAssassin?

Linux (DOH !). OK Windows also but don't ask me how !

Procmail Perl - 5.6.x is recommended.

File::Spec, Pod::Usage, HTML::Parser, DB_File, Net::DNS perl modules. See install documentation !

A local mail agent (fetchmail, qmail, sendmail, etc.)

root access to install for system wide usage. Normal access to run.


How do I install ?How do I install ?(http://www.spamassassin.org/full/2.6x/dist/INSTALL)(http://www.spamassassin.org/full/2.6x/dist/INSTALL)

Installing or Upgrading SpamAssassin:

The easiest way to do this is using CPAN.pm :

perl -MCPAN -e shell [as root]

o conf prerequisites_policy ask

install Mail::SpamAssassin

quit

Debian, you can apt-get it from unstable, thanks to Duncan Findlay.

Alternatively download the tarfile, zipfile or Red Hat RPM from http://spamassassin.org/ , and install that, like so:

[unzip/untar the archive]

cd Mail-SpamAssassin-*

perl Makefile.PL

[option: add -DSPAMC_SSL to $CFLAGS to build an SSL-enabled spamc]

make

make install [as root]


How do I install ? (Cont)How do I install ? (Cont) Create a non privileged user “spamd” or similar

Download this start up script, modify the start section and put it as “spamd” in /etc/init.d - http://www.peregrinehw.com/downloads/SpamAssassin/spamassassin

Start: “daemon spamd -d -a -u spamd” - (-d daemon, -a auto-whitelist, -u <user>)

Don't forget to make it executable !

modify /etc/procmailrc to include :

PMDIR=$HOME/procmail – sets local procmail directory

LOGFILE=$PMDIR/log – sets up local logging – don't forget to rotate log files with cron

VERBOSE=no

LOG="

"

MAILDIR=$HOME/msgs – sets location of local mail files

INCLUDERC=$PMDIR/rc.spam – this tells procmail to run local procmail recipes, spamc will be called from here. We can make it run on ALL mail by calling it from within /etc/procmailrc.


How do I install ? (Cont)How do I install ? (Cont)

rc.spam – local procmail file that catches bad attachments and calls spamassassin.

VERBOSE=yes – set verbose logging here.

:0 BH # contains a virus or other suspicious attachment ?

* .*\/(Content-(Type|Description|Disposition):.*\.(hta|vbs|exe|scr|pif|lnk|bat|ocx|cmd|zip)|\

.*\/name=.*\.(hta|vbs|exe|scr|pif|lnk|bat|ocx|cmd|tst|zip))

{ :0c:

IN-attach

:0h

| /bin/mail -s "CHECK msgs-IN-attach" mark }

:0fw: spamassassin.lock # here we call spamassassin

* < 256000 # is it smaller that 256K ?

| spamc

:0: # does the Spam-Status header contain the word “Yes”. If so put it in special folder called IN-REALSPAM

* ^X-Spam-Status: Yes

IN-REALSPAM

set up a .spamassassin folder and give spamd access rights so that it can manage the auto whitelist and Bayesian databases


How do I customize ?How do I customize ?

local.cf is the file that controls what SpamAssassin does. It is usually located here :

/etc/mail/spamassassin/local.cf

http://www.yrex.com/spam/spamconfig.php - SpamAssassin local.cf Configuration Generator

Local.cf documentation lives here :

perldoc Mail::SpamAssassin::Conf

typical local.cf below :

rewrite_subject 1

subject_tag **SA**

report_safe 2

required_hits 5.0

ok_languages en he

use_bayes 1

use_terse_report 0

score FORGED_HOTMAIL_RCVD2 3.5

blacklist_from [email protected]

whitelist_from [email protected]


What does the SpamAssassin report look like for a typical piece of spam?What does the SpamAssassin report look like for a typical piece of spam?

X-Spam-Flag: YES

X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on linux1.bgaddict.com

X-Spam-Status: Yes, hits=18.4 required=5.0 tests=BAYES_99,HTML_70_80,

HTML_LINK_PUSH_HERE,HTML_MESSAGE,HTML_TITLE_UNTITLED,

MIME_HEADER_CTYPE_ONLY,MIME_HTML_ONLY,RCVD_IN_DSBL,RCVD_IN_NJABL,

RCVD_IN_NJABL_PROXY,RCVD_IN_SORBS,RCVD_IN_SORBS_SOCKS,SORTED_RECIPS,

SUSPICIOUS_RECIPS autolearn=spam version=2.63

X-Spam-Level: ******************

Content preview: Untitled Document Order Rx Meds From Home Valium -

Xanax - Vicodin ES - Hydrocodone - Viagra Weight Loss, Sexual Health,

Pain Relief [...]

Content analysis details: (18.4 points, 5.0 required)

pts rule name description

- 0.1 HTML_70_80 BODY: Message is 70% to 80% HTML

5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100%

[score: 1.0000]

1.0 HTML_LINK_PUSH_HERE BODY: HTML link text says "push here" or similar

0.3 MIME_HTML_ONLY BODY: Message only has text/html MIME parts

0.1 HTML_MESSAGE BODY: HTML included in message

0.4 HTML_TITLE_UNTITLED BODY: HTML title contains "Untitled"

3.5 SUSPICIOUS_RECIPS Similar addresses in recipient list

2.7 SORTED_RECIPS Recipient list is sorted by address

etc....


ConsiderationsConsiderations

Bayesian filtering does NOT kick in until SpamAssassin has learned at least 200 spam messages.

Use sa-learn to teach SpamAssassin about “spam” and “ham” sa-learn --spam --no-rebuild --showdots --mbox ~/msgs/IN-REALSPAM

sa-learn --ham --no-rebuild --mbox --showdots $MAIL

sa-learn --rebuild

sa-learn --dump magic

By default local configuration files, called user_prefs, located in $HOME/.spamassassin are ignored unless specially allowed in local.cf. “allow_user_rules = 1”


Resource PageResource Page http://spamassassin.org/ - SpamAssassin lives here !

http://news.gmane.org/gmane.mail.spam.spamassassin.general - SpamAssassin list archive

http://wiki.spamassassin.org/ - SpamAssassin FAQ in Wiki format

http://www.yrex.com/spam/spamconfig.php - SpamAssassin Configuration Generator

http://www.peregrinehw.com/downloads/SpamAssassin/spamassassin - SpamAssassin start up script

http://www.spambouncer.org/ – SpamAssassin alternative http://www.ii.com/internet/robots/procmail/qs/ - Procmail quick start

http://pegasus.rutgers.edu/~elflord/unix/procmail.html - Procmail Tutorial

comp.mail.sendmail -- and – others.


Questions ?Questions ?

If you got em, ask em !

Stop Spam by using SpamAssassin and Procmail ! Mark Kushinsky – MDS Computer Solutions...

Documents

Transcript of Stop Spam by using SpamAssassin and Procmail ! Mark Kushinsky – MDS Computer Solutions...