Assessing product image quality for online shopping
-
Upload
anjan-goswami -
Category
Data & Analytics
-
view
63 -
download
1
Transcript of Assessing product image quality for online shopping
![Page 1: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/1.jpg)
eBay Confidential
Assessing product image quality for online shopping
Anjan Goswami, Sung H. Chung, Naren Chittar and Atiq Islam
![Page 2: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/2.jpg)
eBay Confidential
Outline
• Motivation• Definition • Features• Experiments• Image quality models.• What’s next?
![Page 3: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/3.jpg)
eBay Confidential
Motivation
![Page 4: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/4.jpg)
eBay Confidential
Motivation
• Better Aesthetics.• Better customer experience?• Better conversion?• Better branding?• Better looking merchandising pages.
![Page 5: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/5.jpg)
eBay Confidential
Applications
• Ranking.• Improve browsing experience.• Guidelines for sellers.• Merchandising.• Many more.
![Page 6: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/6.jpg)
eBay Confidential
What is product image quality?• We care mainly about product images.• Product images have specific characteristics.
![Page 7: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/7.jpg)
eBay Confidential
Computing image quality
• Machine learning problem.• Factors from images to construct feature
vectors.• Label the data points as one of the classes.• Build a classifier to get the class probabilities.• Alternatively, make a regression model from
human judgment data.
![Page 8: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/8.jpg)
eBay Confidential
What factors? How do we compute?
• Size factors such as area, aspect ratio.• Image attributes such as brightness,
saturation, colorfulness, contrast, dynamic range.
• Factors based on background and foreground segmentation.
![Page 9: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/9.jpg)
eBay Confidential
BrightnessAverage grayscale intensity of the image.
0.299*red+0.587*green+0.114*blue; 0.18
0.58
![Page 10: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/10.jpg)
eBay Confidential
Colorfulness
• Difference between a color against gray. • Many empirical notions.• We use an empirical expression designed in
Natural Color Space. (NCS)• This space has a concept of rg and yb
coordinates.• stdev (rg,yb)+ 0.3 mean (rg,yb)• We compute this globally and for foreground.
![Page 11: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/11.jpg)
eBay Confidential
ExampleGlobal FG20 8
60 64
![Page 12: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/12.jpg)
eBay Confidential
Saturation
• Many notions and empirical formula.• Average of (max – min) in RGB space. (Used
mainly in photography)
![Page 13: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/13.jpg)
eBay Confidential
Example of SaturationScore86
255
![Page 14: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/14.jpg)
eBay Confidential
Dynamic range
• Variants of expressions in photography literature and in computer vision.
• We are using a simpler definition used in photography based on range of gray scale intensity.
![Page 15: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/15.jpg)
eBay Confidential
Example of Dynamic RangeScore32
100
![Page 16: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/16.jpg)
eBay Confidential
Contrast
• Michelson contrast : Range /(max + min) in a color space.
• RMS contrast : stdev of spatial intensities.
![Page 17: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/17.jpg)
eBay Confidential
Example of Michelson ContrastScore25
99
![Page 18: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/18.jpg)
eBay Confidential
Segmentation
• Grabcut with a heuristic for automated segmentation.
![Page 19: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/19.jpg)
eBay Confidential
Segmentation
![Page 20: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/20.jpg)
eBay Confidential
Background and foreground area ratio
• Use segmented image.• An approximation is used by using ratio of
pixels in the foreground and in the background.
![Page 21: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/21.jpg)
eBay Confidential
ExampleScore12
83
![Page 22: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/22.jpg)
eBay Confidential
background and foreground differences.
• Brightness L1 distance.• Contrast L2 distance.
![Page 23: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/23.jpg)
eBay Confidential
Properties of background
• stdev of lightness (distance from white in RGB.)
• Mean of lightness. (RGB)
• A score on uniformity of background intensity that approximates texture properties.
![Page 24: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/24.jpg)
eBay Confidential
An example
Image Colorfulness
RMS Contrast
Brightness diff
Area ratio
Stdev of lightness of background
Uniformity Whiteness of background
Colorfulness of foreground
38.85
0.30 142.33
0.75 4.22 2.63 100 35.76
44.77
0.086 28.6808
0.66 27.27 16.37 0 39.73
![Page 25: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/25.jpg)
eBay Confidential
Data Collection
• Crowdsourcing (52K images)• Professional judgments. (6K images)
![Page 26: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/26.jpg)
eBay Confidential
Crowdsourcing
• Has its own challenges.• Require thoughts for framing questions.• Require thoughts for conducting the
experiment.• Cheap labelers can attempt cheating.• Classifier result can be different based on
voting techniques used to find the label.• More judgments are better.
![Page 27: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/27.jpg)
eBay Confidential
Image Quality Classes
• Good• Fair• Poor
![Page 28: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/28.jpg)
eBay Confidential
Professional Quality Images
• Mostly white, light or uniform background.• Image is free from compression artifact such
as blurring.• Professionally photographed in proper lighting
condition .• Subject has a reasonable size and is in focus.• Example of such images can be seen in
branded retail websites.
![Page 29: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/29.jpg)
eBay Confidential
Professional Quality
![Page 30: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/30.jpg)
eBay Confidential
Poor images
• Poor or dark background• Can have incomprehensible texture.• Subject small.• Subject unclear.• Bad aspect ratio.• Poor resolution and photography.
![Page 31: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/31.jpg)
eBay Confidential
Poor Images
![Page 32: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/32.jpg)
eBay Confidential
Fair images
• These are images that are not poor. However, they are not as clean as professional looking photos. (Add examples)
![Page 33: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/33.jpg)
eBay Confidential
How do we develop the model
• Multi-class classification with two data sources.
• Direct Regression with crowd-sourced data.• Used Gradient boosted tree.
![Page 34: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/34.jpg)
eBay Confidential
Factor importance in Quality Classifier (Classifier)
• Background lightness • Brightness • Aspect ratio • Dynamic range • Background foreground area ratio• Michelson contrast
![Page 35: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/35.jpg)
eBay Confidential
Error Rates
• Poor misclassification 10%• Fair misclassification 50% [BAD]• Good misclassification 7%
• However, our training data is so far not perfect.
![Page 36: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/36.jpg)
eBay Confidential
Quality Score for Classification
• The quality score is the expected average of the class weights using the class probabilities.
• Currently class weights are simple linear function that maps poor, fair and good to 1,2,3.
![Page 37: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/37.jpg)
eBay Confidential
Quality Score201
244
102
119
![Page 38: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/38.jpg)
eBay Confidential
Quality Score 177
137
153
![Page 39: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/39.jpg)
eBay Confidential
Quality Score122
102
105
![Page 40: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/40.jpg)
eBay Confidential
Factor Importance in Regression
• Background lightness• Saturation• Aspect ratio• RMS contrast• Colorfulness
![Page 41: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/41.jpg)
eBay Confidential
Comparison of ModelsImage Classifier Regression
98 86
100 57
231 235
133 96
![Page 42: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/42.jpg)
eBay Confidential
What’s Next (Semantics)
• Object recognition/relevance.
• Text/watermark detection.
• Human model detection.
>
![Page 43: Assessing product image quality for online shopping](https://reader034.fdocuments.net/reader034/viewer/2022052603/55cfcf25bb61eb0e708b46f5/html5/thumbnails/43.jpg)
eBay Confidential
Q & A
Thanks!Ideas & questions?