PPM based Spam Filtering in SEWM2008

12
PPM based Spam Filtering in SEWM2008 Liu JuXin, Xu Congfu, Peng Peng, Lu Guanzhong [email protected],[email protected] ,[email protected] [email protected] College of Computer Science, Zhejiang University April 10, 2008

TAGS:

description

PPM based Spam Filtering in SEWM2008. Liu JuXin, Xu Congfu, Peng Peng, Lu Guanzhong [email protected],[email protected] ,[email protected] [email protected] College of Computer Science, Zhejiang University April 10, 2008. Outline. PPM( prediction by partial matching ) - PowerPoint PPT Presentation

Transcript of PPM based Spam Filtering in SEWM2008

Page 1: PPM based Spam Filtering in SEWM2008

PPM based Spam Filtering

in SEWM2008Liu JuXin, Xu Congfu, Peng Peng, Lu

Guanzhong

[email protected],[email protected],[email protected] [email protected]

College of Computer Science, Zhejiang UniversityApril 10, 2008

Page 2: PPM based Spam Filtering in SEWM2008

Outline

PPM( prediction by partial matching ) Email Pre-processing Train PPM Model Model Classification

Page 3: PPM based Spam Filtering in SEWM2008

PPM

Data Compression

Page 4: PPM based Spam Filtering in SEWM2008

PPM Framework

Page 5: PPM based Spam Filtering in SEWM2008

Email Pre-processing

Source alphabet Merge continuous spaces Truncate long messages

Page 6: PPM based Spam Filtering in SEWM2008

Email Pre-processing

Raw DataAbcd_= - Af?/[]=+ safj =ab fe addfe

Sample:Alphabet : {a,b,c,d,e,f,_,=, }Replace char: ?Truncate length: 20

After Replaceabcd_= ? Af????=? ?af? =ab fe addfe

After Merge Blankabcd_= ? Af????=? ?af? =ab fe addfe

After Truncateabcd_= ? Af????=? ?a

Page 7: PPM based Spam Filtering in SEWM2008

Train PPM Model

Use order-6 PPM* model Use Method D Escape estimation Train Two PPM model HAM Model SPAM Model

Page 8: PPM based Spam Filtering in SEWM2008

Model Classification

MCE( Minimum Cross-entropy ) MDL( Minimum Description Length ) Spam Score

Page 9: PPM based Spam Filtering in SEWM2008

Advantage

Simple pre-processing No decode ( avoid obfuscate ) Highly self-adaptive Low false positive

Page 10: PPM based Spam Filtering in SEWM2008

Reference

《 Spam Filtering Using Statistical Data Compression Models 》

《 Unbounded Length Contexts for PPM 》

Page 11: PPM based Spam Filtering in SEWM2008

Question

Delay Index ham, Ham and HAM Active learning 10000

Deliver the filter

Page 12: PPM based Spam Filtering in SEWM2008

Thanks for your attention!Q&A


Privacy-Aware Collaborative Spam Filtering document

Privacy-Aware Collaborative Spam Filtering document

Spam Filters - plg.uwaterloo.caplg.uwaterloo.ca/~gvcormac/csc.pdf/csctalk.pdfCormack Spam Filters 15 February, 2006 Why Standardized Evaluation? To answer questions! Is spam filtering

Spam Filters - plg.uwaterloo.caplg.uwaterloo.ca/~gvcormac/csc.pdf/csctalk.pdfCormack Spam Filters 15 February, 2006 Why Standardized Evaluation? To answer questions! Is spam filtering

Logistic Regression for Spam Filtering

Logistic Regression for Spam Filtering

SPAM And SpamAssassin - haifux.orghaifux.org/hebrew/lectures/155/spam.pdf · The SPAM arms race SPAM filtering is a unique problem in AI. Moving target – SPAM keeps changing. –

SPAM And SpamAssassin - haifux.orghaifux.org/hebrew/lectures/155/spam.pdf · The SPAM arms race SPAM filtering is a unique problem in AI. Moving target – SPAM keeps changing. –

Spam Filtering at CERN Emmanuel Ormancey - 23 October 2002

Spam Filtering at CERN Emmanuel Ormancey - 23 October 2002

Anomaly-based Spam Filtering - SECRYPT 2011

Anomaly-based Spam Filtering - SECRYPT 2011

Deep belief networks for spam filtering

Deep belief networks for spam filtering

Spam Filtering with Naive Bayes – Which Naive Bayes?

Spam Filtering with Naive Bayes – Which Naive Bayes?

ANALISIS SPAM FILTERING PADA MAIL SERVER DENGAN …eprints.uns.ac.id/13903/1/321622310201309541.pdf · perpustakaan.uns.ac.id digilib.uns.ac.id commit to user i ANALISIS SPAM FILTERING

ANALISIS SPAM FILTERING PADA MAIL SERVER DENGAN …eprints.uns.ac.id/13903/1/321622310201309541.pdf · perpustakaan.uns.ac.id digilib.uns.ac.id commit to user i ANALISIS SPAM FILTERING

Twitter Content-based Spam Filtering - CISIS 2013

Twitter Content-based Spam Filtering - CISIS 2013

Bayesian Learning Application to Text Classification Example: spam filtering

Bayesian Learning Application to Text Classification Example: spam filtering

RNT Spam Filtering Michael Del Valle

RNT Spam Filtering Michael Del Valle

Filtering E-mail for Spam: PC - CSNFiltering for Spam Filtering for spam can be a challenge since spam messages usually come from many different senders, including senders who use

Filtering E-mail for Spam: PC - CSNFiltering for Spam Filtering for spam can be a challenge since spam messages usually come from many different senders, including senders who use

Spam Filtering with Naive Bayes – Which Naive Bayes? · Spam Filtering with Naive Bayes – Which Naive Bayes? Vangelis Metsis1,2, Ion Androutsopoulos1 and Georgios Paliouras2 1Department

Spam Filtering with Naive Bayes – Which Naive Bayes? · Spam Filtering with Naive Bayes – Which Naive Bayes? Vangelis Metsis1,2, Ion Androutsopoulos1 and Georgios Paliouras2 1Department

Network-Based Spam Filtering Anirudh Ramachandran Nick Feamster Georgia Tech.

Network-Based Spam Filtering Anirudh Ramachandran Nick Feamster Georgia Tech.

A Buyers Guide to Spam Filtering - IT Consulting Services ... Buyers... · A Buyers Guide to Spam Filtering Introduction ‘Spam’ is an irreverent term for a very serious problem.

A Buyers Guide to Spam Filtering - IT Consulting Services ... Buyers... · A Buyers Guide to Spam Filtering Introduction ‘Spam’ is an irreverent term for a very serious problem.

Spam Filtering Based On The Analysis Of Text Information ...jmlr.csail.mit.edu/papers/volume7/fumera06a/fumera06a.pdf · Spam Filtering Based On The Analysis Of Text Information Embedded

Spam Filtering Based On The Analysis Of Text Information ...jmlr.csail.mit.edu/papers/volume7/fumera06a/fumera06a.pdf · Spam Filtering Based On The Analysis Of Text Information Embedded

6/1/2015 Email Spam Filtering - Muthiyalu Jothir 1 Email Spam Filtering Computer Security Seminar N.Muthiyalu Jothir – 271120 Media Informatics.

6/1/2015 Email Spam Filtering - Muthiyalu Jothir 1 Email Spam Filtering Computer Security Seminar N.Muthiyalu Jothir – 271120 Media Informatics.

EMAIL SPAM FILTERING USING DECISION TREE …...EMAIL SPAM FILTERING USING DECISION TREE ALGORITHM Divesh Palival, Kevin Printer, Ramchandra Devre, Asst.Prof. Nikita Lemos Abstract—

EMAIL SPAM FILTERING USING DECISION TREE …...EMAIL SPAM FILTERING USING DECISION TREE ALGORITHM Divesh Palival, Kevin Printer, Ramchandra Devre, Asst.Prof. Nikita Lemos Abstract—

Filtering spam at the ISP - AFNOG · 2008-02-06 · Filtering spam at the ISP Patrick J Okui Ayitey Bulley (Liberal Borrowing from Brian Candler) Sources of junk email Spam Unsolicited,

Filtering spam at the ISP - AFNOG · 2008-02-06 · Filtering spam at the ISP Patrick J Okui Ayitey Bulley (Liberal Borrowing from Brian Candler) Sources of junk email Spam Unsolicited,