CSSE463: Image Recognition CSSE463: Image Recognition Day 11Day 11 Due: Due:

Written assignment 1 tomorrow, 4:00 pmWritten assignment 1 tomorrow, 4:00 pm Start thinking about term project ideas.Start thinking about term project ideas.

Lab 4 (shape) tomorrow: feel free to start in Lab 4 (shape) tomorrow: feel free to start in advanceadvance

Questions?Questions?

Next 1.5 weeks: Pattern recognitionNext 1.5 weeks: Pattern recognition Concepts, error types (today)Concepts, error types (today) Basic theory and how to use classifiers in Matlab:Basic theory and how to use classifiers in Matlab:

Neural networks Neural networks Support vector machines (SVM).Support vector machines (SVM).

Pattern recognitionPattern recognition Making a decision from Making a decision from

datadata A classification problem: A classification problem:

assign a single class label assign a single class label to a datum pointto a datum point

Can include a special class, Can include a special class, rejectreject, ,

if a sample (a single datum if a sample (a single datum point) appears not to belong to point) appears not to belong to any known classany known class

If it is on the boundary between If it is on the boundary between classesclasses

Else Else forced classificationforced classification Boundaries between Boundaries between

classes-how?classes-how? There’s tons of theory, can There’s tons of theory, can

be applied to many areas. be applied to many areas. We focus on small subset We focus on small subset of those used for visionof those used for vision

Q1

Baseline: Hand-tuned decision Baseline: Hand-tuned decision boundariesboundaries

You did this based on observations for fruit You did this based on observations for fruit classificationclassification

You’ll do the same thing in Lab 4 for You’ll do the same thing in Lab 4 for shapesshapes

But what if the features were much more But what if the features were much more complex?complex?We now discuss classifiers that learn class We now discuss classifiers that learn class

boundaries based on boundaries based on exemplarsexemplars (e.g., labeled (e.g., labeled training examples)training examples)

Ex: Nearest neighbor classifierEx: Nearest neighbor classifier Assumes we have a feature Assumes we have a feature

vector for each imagevector for each image Calculate distance from Calculate distance from new new

test sampletest sample to to each labeled each labeled training sampletraining sample. .

Assign label as closest training Assign label as closest training samplesample

Generalize by assigning same Generalize by assigning same label as the majority of the k label as the majority of the k nearest neighbors. No majority?nearest neighbors. No majority?

d

i

ipipppdDIn

ypypxpxpppDIn

1

22121

221

22121

))()((,

))()(())()((,2

Nearest class meanNearest class mean Find class means and Find class means and

calculate distance to calculate distance to each meaneach mean Pro?Pro? Con?Con?

Partial solution: Partial solution: clusteringclustering Learning vector Learning vector

quantization (LVQ): quantization (LVQ): finds optimal clustersfinds optimal clusters

LVQ

Test point

Q2

Common model of learning Common model of learning machinesmachines

StatisticalLearning

Labeled TrainingImages

Extract Features(color, texture)

TestImage

Summary

LabelClassifierExtract Features(color, texture)

Focus on testingFocus on testingLet m = the number of possible class labelsLet m = the number of possible class labelsConsider m==2. Consider m==2. Example: Calculate distance to cluster Example: Calculate distance to cluster

means for 2 classes.means for 2 classes.

Dist/prob1

Dist/prob2

TestImage

Extract Features(color, texture)

Decide Class:1, 2

Multiclass problems Multiclass problems

Consider m>2. Consider m>2. Example: Calculate distance to cluster Example: Calculate distance to cluster

means for 10 classes.means for 10 classes.

Dist/prob1

Dist/prob2

Dist/probN

TestImage

Extract Features(color, texture)

Decide Class:1, 2,…N

How good is your classifier?How good is your classifier? Example from medicine: Example from medicine:

Disease detectionDisease detection Consider costs of false Consider costs of false

neg. vs. false pos.neg. vs. false pos. Lots Lots of different error of different error

measuresmeasures Accuracy = 10500/10800 = Accuracy = 10500/10800 =

97%. Is 97% accuracy OK?97%. Is 97% accuracy OK? Recall (or true positive Recall (or true positive

rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200

= 2%= 2%

DetectedDetected

TrueTrue

YesYes NoNo

YesYes 500500(true (true pos.)pos.)

100100(false (false neg.)neg.)

NoNo 200200(false (false pos.)pos.)

1000010000(true (true neg.)neg.)

10200Total actualnegative

600Total actualpositive

700Total det. as pos.

10100Total det. as neg.

How good is your classifier?How good is your classifier? Write out definitions of Write out definitions of

each measure noweach measure now

ExamplesExamples Accuracy = 10500/10800 = Accuracy = 10500/10800 =

97%. 97%. Recall (or true positive Recall (or true positive

rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200

= 2%= 2%

Detected:Detected:

Has:Has:

YesYes NoNo

YesYes 500500(true (true pos.)pos.)

100100(false (false neg.)neg.)

NoNo 200200(false (false pos.)pos.)

1000010000(true (true neg.)neg.)

Q3

What if I have a tunable What if I have a tunable threshold, t?threshold, t?

If t == 0: TPR = ___, FPR = ___If t == 0: TPR = ___, FPR = ___9/129/12 2/82/8

If t == 1: TPR = ___, FPR = ___If t == 1: TPR = ___, FPR = ___

Repeat for many values of tRepeat for many values of t

-3 -2 -1 0 1 2 3

NN N P N P N N P N PP N PP PP PPP

Simple example: single real-valued output.Simple example: single real-valued output.Thresholding: label = output > t ? P : NThresholding: label = output > t ? P : N

False Pos Rate

True

Pos

Rat

e

True class:

ROC curve ROC curve Receiver-operating Receiver-operating

characteristiccharacteristic Useful when you Useful when you

can change a can change a threshold to get threshold to get different true and different true and false positive ratesfalse positive rates

Consider extremesConsider extremes Much more Much more

information information recorded here!recorded here!

Q3

Confusion matrices for m>2 Confusion matrices for m>2 (outdoor image example)(outdoor image example)

Beach recall: 169/(169+0+2+3+12+14)=84.5%Beach recall: 169/(169+0+2+3+12+14)=84.5% Note confusion between mountain and urban Note confusion between mountain and urban

classes due to featuresclasses due to features Similar colors and spatial layoutSimilar colors and spatial layout

Bch Sun FF Fld Mtn UrbBch 169 0 2 3 12 14Sun 2 183 5 0 5 5FF 3 6 176 6 4 5Fld 15 0 1 173 11 0Mtn 11 0 2 21 142 24Urb 16 4 8 5 27 140

Detected

True

Q4

Why do we need separate training Why do we need separate training and test sets?and test sets?

Exam analogyExam analogy

But working on practice questions is helpful…get But working on practice questions is helpful…get the analogy? the analogy? We hope our ability to do well on We hope our ability to do well on practice questions helps us on the actual exampractice questions helps us on the actual exam

Application to nearest-neighbor classifiersApplication to nearest-neighbor classifiers

Often reserve a 3Often reserve a 3rdrd set for validation as well (to set for validation as well (to tune parameters of training set)tune parameters of training set)

Q5-8