Veggie Vision by IBM
description
Transcript of Veggie Vision by IBM
CSE 803 Fall 2015 1
Veggie Vision by IBM
Ideas about a practical system to make more efficient the selling and
inventory of produce in a grocery store.
CSE 803 Fall 2015 2
Problem is recognizing produce
• properly charge customer • do inventory • save customer and checker time
CSE 803 Fall 2015 3
15+ years of R&D now This information was shared by IBM researchers. Since that time, the system has been tested in small markets and has been modified according to that experience.
CSE 803 Fall 2015 4
Up to 400 produce types
CSE 803 Fall 2015 5
Practical problems of application environment
CSE 803 Fall 2015 6
Engineering the solution
CSE 803 Fall 2015 7
System to operate inside the usual checkout station
• together with bar code scanner • together with scale • together with accounting • together with inventory • together with employee • within typical store environment * figure shows system asking for help from the cashier in making final decision on touch screen
CSE 803 Fall 2015 8
Modifying the scale
CSE 803 Fall 2015 9
Need careful lighting engineering
CSE 803 Fall 2015 10
Need to segment product from background, even through plastic
CSE 803 Fall 2015 11
Previously published thresholding decision
CSE 803 Fall 2015 12
Quality segmented image obtained
CSE 803 Fall 2015 13
Design of pattern recognition paradigm (from 1997)
FEATURES are: color, texture, shape, and size all represented uniformly by HISTOGRAMS Histograms capture statistical properties of regions – any number of regions.
CSE 803 Fall 2015 14
Matching procedure n Sample product represented by concatenated
histograms: about 400 D n 350 produce items x 10 samples = 3500
feature vectors of 400D each n Have about 2 seconds to compare an
unknown sample to 3500 stored samples (3500 dot products)
n Analyze the k nearest: if closest 2 are from one class, recognize that class (sure)
CSE 803 Fall 2015 15
HSI for pixel color: 6 bits for hue, 5 for saturation and intensity
For each pixel quantify H HIST[H]++ same for S&I
CSE 803 Fall 2015 16
Histograms of 2 limes versus 3 lemons
Distribution or population concept adds robustness: • to size of objects • to number of objects • to small variations of color (texture, shape, size)
CSE 803 Fall 2015 17
Texture: histogram results of LOG filter[s] on produce pixels
Leafy produce A
Leafy produce B
CSE 803 Fall 2015 18
Shape: histogram of curvature of boundary of produce
CSE 803 Fall 2015 19
Banana versus lemon or cucumber versus lime
Large range of curvatures indicates complex object
Small range of curvatures indicates roundish object
CSE 803 Fall 2015 22
Learning and adaptation n System “easy” to train: show it produce
samples and tell it the labels. n During service: age out oldest sample;
replace last used sample with newly identified one.
n When multiple labeled samples match the unknown, system asks cashier to select from the possible choices.
CSE 803 Fall 2015 23
Extension of Veggie Vision n http://www.internetnews.com/xSP/article.php/
3642386 n System uses almost all color features n Installed in few places: many stores have self-
checkout, putting work on customer. n IBM has a “shopping research” unit
http://www.usatoday.com/tech/news/techinnovations/2003-09-26-future-grocery-shop_x.htm
n Customers will tolerate a higher human error than a machine error