Improving Spam Detection Based on Structural Similarity
By Luiz H. Gomes, Fernando D. O. Castro, Rodrigo B. Almeida,Luis M. A. Bettencourt, Virgílio A. F. Almeida, Jussara M. Almeida
Presented at Steps to Reducing Unwanted Traffic on the Internet Workshop, 2005
Presented by Jared Bott
2
Outline
Overview
Concepts
Detecting Spam
Experimental Results
Analysis of Paper
3
Overview
New algorithm to detect spam messagesUses email information that is harder to
changeWorks in conjunction with another spam
classifier I.e. SpamAssassin
Less false positives than compared methods
4
Spam Detection Problem
Spam detection algorithms use some part of emails to determine if a message is spam Spammers change messages so that they do
not meet detection criteria for spam
Very easy to change spam messages, usernames, domains, subjects, etc.
5
Key Idea
The lists that spammers and legitimate users send messages to and from can be used as the identifiers of classes of email traffic. The lists of addresses spammers send to are
unlikely to be similar to those of legitimate users.
Lists don’t change that often
6
Using Lists
A user is not just an email address. It can be a domain, etc.
Represent email user as a vector in multi-dimensional conceptual space created with all possible contacts Each sender and each recipient has their own
vectorModel relationship between senders and
recipients
7
Constructing Vectors
If there is at least one email sent from sender si to recipient rn, then the value in si’s vector’s nth dimension is 1. Otherwise, that value is 0.
If there is at least one email received by recipient ri from sender sn, the value in ri’s vector’s nth dimension is 1. Otherwise it is 0.
8
Example Vectors
User 1
User 2
User 3
S[0,1,1]R[0,1,0]
S[1,0,1]R[1,0,0]
S[0,0,0]R[1,1,0]
9
Similarity Between Senders
Similarity between senders si and sk is the cosine of the angle between their vectors cos(si, sk) 0 means no shared contact 1 means identical contact lists
In legitimate email, a 1 means that the senders operate in the same social group.
In spammers, a 1 means that the senders use the same list or are the same person.
10
Grouping Users Into Clusters
Group users with similar vectors Users with similar vectors are likely to have
related roles, i.e. spammer or legitimate user
Each cluster is represented by a vector This vector is the sum of all its component
users’ vectors
11
Similarity Between a User and a Cluster
Similarity is derived from user to user similarity equation If sender si is a member of cluster sck, then the
similarity is cos(sck – si, si).
If sender si is not a member of cluster sck, then the similarity is cos(sck, si).
Similarity between a user and a cluster will change over time Remove the user’s vector from the cluster’s vector when
computing similarity and reclassifying a user
12
Detecting Spam
Two probabilities to compute Ps(m) – Probability of an email m being sent by
a spammer
Pr(m) – Probability of an email m being addressed to users that receive spam
13
Detecting Spam
When an email arrives, classify it using some other method
Find the cluster (sc) the email’s sender belongs in If many users in the cluster send messages that are
classified as spam by auxiliary method, the probability of all the users in that cluster sending spam is high
Update the sc’s spam probability Ps(m) ← sc’s spam probability
14
Detecting Spam
For all recipients of the email, find the cluster (rc) each one belongs to
Update the spam probability for each cluster
Pr(m) ← Pr(m) + spam probability of each rc
Pr(m) ← Pr(m)/number of recipients
15
Detecting Spam
Compute a spam rank for the email based upon Pr(m) and Ps(m)
If the spam rank is above some threshold (ω), label it as spam
If the spam rank is below 1- ω, label it is legitimate
Otherwise label the email as the auxiliary method’s classification
16
17
Experimental Results
Tested on a log of eight days of email from a large Brazilian university
Tested on a 2.8 GHz Pentium 4 with 512 MB RAM Able to classify 20 messages per second Faster than the average message arrival peak
rate
18
Results
Measure Non-Spam Spam Aggregate
# of emails 191,417 173,584 365,001
Size of emails 11.3 GB 1.2 GB 12.5 GB
# of distinct senders
12,338 19,567 27,734
# of distinct recipients
22,762 27,926 38,875
19
Results
Manually checked false positives to see if they were spam or not Auxiliary algorithm had more false positives
Algorithm % of Misclassifications
Original Classification 60.33%
Their approach 39.67%
20
Strengths
Less false positives than SpamAssassin
Low-cost
Works with message information that doesn’t change that much
21
Weaknesses
Needs an additional message classifier, i.e. SpamAssassin
Manual tuning of algorithm
22
Improvements
Time correlation of similar addresses
Collaborative filtering based upon user feedback
Top Related