MAHOUT classifier tour

Mahout

1Wednesday, March 16, 2011

MahoutScalable Data Mining for Everybody


What is Mahout

• Recommendations (people who x this also x that)

• Clustering (segment data into groups of)

• Classification (learn decision making from examples)

• Stuff (LDA, SVD, frequent item-set, math)


What is Mahout?

• Recommendations (people who x this also x that)

• Clustering (segment data into groups of)

• Classification (learn decision making from examples)

• Stuff (LDA, SVM, frequent item-set, math)


Classification in Detail

• Naive Bayes Family

• Hadoop based training

• Decision Forests


• Logistic Regression (aka SGD)

• fast on-line (sequential) training


Classification in Detail

• Naive Bayes Family


• Decision Forests


• Logistic Regression (aka SGD)

• fast on-line (sequential) training


So What?

Online training has low overhead for small and moderate size data-sets


So What?

Online training has low overhead for small and moderate size data-sets

big starts here


An Example


And Another

From: Dr. Paul AcquahDear Sir,Re: Proposal for over-invoice Contract Benevolence

Based on information gathered from the India hospital directory, I am pleased to propose a confidential business deal for our mutual benefit. I have in my possession, instruments (documentation) to transfer the sum of 33,100,000.00 eur thirty-three million one hundred thousand euros, only) into a foreign company's bank account for our favor....


And Another

Date: Thu, May 20, 2010 at 10:51 AMFrom: George <[email protected]>

Hi Ted, was a pleasure talking to you last night at the Hadoop User Group. I liked the idea of going for lunch together. Are you available tomorrow (Friday) at noon?


mailto:[email protected]

mailto:[email protected]

And Another

Date: Thu, May 20, 2010 at 10:51 AMFrom: George <[email protected]>

Hi Ted, was a pleasure talking to you last night at the Hadoop User Group. I liked the idea of going for lunch together. Are you available tomorrow (Friday) at noon?


Mahout’s SGD

• Learns on-line per example

• O(1) memory

• O(1) time per training example

• Sequential implementation

• fast, but not parallel


Special Features

• Hashed feature encoding

• Per-term annealing

• learn the boring stuff once

• Auto-magical learning knob turning

• learns correct learning rate, learns correct learning rate for learning learning rate, ...


Feature Encoding


Hashed Encoding


Feature Collisions


Learning Rate AnnealingLe

arni

ng R

ate

# training examples seen


Per-term AnnealingLe

arni

ng R

ate




arni

ng R

ate


Common Feature



arni

ng R

ate


Rare Feature


General Structure

• OnlineLogisticRegression

• Traditional logistic regression

• Stochastic Gradient Descent

• Per term annealing

• Too fast (for the disk + encoder)


Next Level

• CrossFoldLearner

• contains multiple primitive learners

• online cross validation

• 5x more work


And again

• AdaptiveLogisticRegression

• 20 x CrossFoldLearner

• evolves good learning and regularization rates

• 100 x more work than basic learner

• still faster than disk + encoding


A comparison

• Traditional view

• 400 x (read + OLR)

• Revised Mahout view

• 1 x (read + mu x 100 x OLR) x eta

• mu = efficiency from killing losers early

• eta = efficiency from stopping early


Deployment

• Training

• ModelSerializer.writeBinary(..., model)

• Deployment

• m = ModelSerializer.readBinary(...)

• r = m.classifyScalar(featureVector)


The Upshot

• One machine can go fast

• SITM trains in 2 billion examples in 3 hours

• Deployability pays off big

• simple sample server farm


MAHOUT classifier tour

Technology

Transcript of MAHOUT classifier tour