Contact Sheets: Searching Untagged Images on Smartphones · Contact Sheets: Searching Untagged...
1
Contact Sheets: Searching Untagged Images on Smartphones Problem Wolfgang Richter CMU, Kiryong Ha CMU, Alok Shankar CMU, Ardalan Amiri Sani Rice University, Jan Harkes CMU, Lin Zhong Rice University, Mahadev Satyanarayanan CMU PDLR11 Solution Strategy We focus on a cloud-based approach to opportunistic, near real-time search of untagged images on smartphones that is sensitive to bandwidth and energy constraints. • Fidelity Reduced virtual contact sheets Evaluation • Vision and image processing black boxes • Test performance on fidelity reduced inputs • Dataset size of 12,963 images • Varied • Size in resolution • JPEG Quality Factor How do we do • real time • mobile device • image search while conserving: • energy • bandwidth Thief caught in background of family photo [CNN] 0 50 100 150 200 250 300 350 400 Full 640x480 320x240 160x120 80x60 40x30 20x20 0 0.2 0.4 0.6 0.8 1 Bandwidth (KB) Accuracy Fidelity Brightness Accuracy Color Histogram Accuracy Face Detection Accuracy Bandwidth 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 Full 640x480 320x240 160x120 80x60 40x30 20x20 0 0.2 0.4 0.6 0.8 1 Energy Consumption (mJ) Accuracy Fidelity Brightness Accuracy Color Histogram Accuracy Face Detection Accuracy No Batching Batching Average JPEG Size Full 374.06K Resoluon 100 90 75 50 25 5 640x480 220.53K 113.83K 47.03K 30.88K 19.36K 5.67K 320x240 60.61K 32.89K 14.34K 9.61K 6.18K 1.94K 160x120 17.20K 9.92K 4.59K 3.17K 2.13K 826.16b 80x60 5.17K 3.23K 1.63K 1.19K 891.75b 459.93b 40x30 1.70K 1.18K 725.13b 587.58b 483.87b 344.76b 20x20 1000.91b 727.07b 501.09b 435.51b 385.46b 316.62b 0 0.2 0.4 0.6 0.8 1 640x480 320x240 160x120 80x60 40x30 20x20 Filter Accuracy Fidelity 0.991 0.899 100 95 75 50 25 0 0.2 0.4 0.6 0.8 1 640x480 320x240 160x120 80x60 40x30 Filter False Negative Rate Fidelity 0.124 0.042 100 95 75 50 25 0 0.2 0.4 0.6 0.8 1 640x480 320x240 160x120 80x60 Filter False Negative Rate Fidelity 0.575 0.281 100 95 75 50 25 0 0.2 0.4 0.6 0.8 1 Filter Accuracy Fidelity 0.650 0.424 100 95 75 50 25 0 0.2 0.4 0.6 0.8 1 Filter False Negative Rate Fidelity 0.040 100 95 75 50 25 640x480 320x240 160x120 80x60 40x30 20x20 0 0.2 0.4 0.6 0.8 1 640x480 320x240 160x120 80x60 40x30 Filter Accuracy Fidelity 0.907 0.803 100 95 75 50 25 640x480 320x240 160x120 80x60
Transcript of Contact Sheets: Searching Untagged Images on Smartphones · Contact Sheets: Searching Untagged...
Contact Sheets: Searching Untagged Images on Smartphones
Problem
Wolfgang Richter CMU, Kiryong Ha CMU, Alok Shankar CMU, Ardalan Amiri Sani Rice University, Jan Harkes CMU, Lin Zhong Rice University, Mahadev Satyanarayanan CMU
PDLR11
Solution StrategyWe focus on a cloud-based approach to opportunistic, near real-time search of untagged images on smartphones that is sensitive to bandwidth and energy constraints.• Fidelity Reduced virtual contact sheets
Evaluation• Vision and image processing black boxes• Test performance on fidelity reduced inputs• Dataset size of 12,963 images• Varied• Size in resolution• JPEG Quality Factor
How do we do• real time• mobile device• image search
while conserving:• energy• bandwidth Thief caught in background of
family photo [CNN]
0
50
100
150
200
250
300
350
400
Full 640x480 320x240 160x120 80x60 40x30 20x20 0
0.2
0.4
0.6
0.8
1
Ban
dwid
th (K
B)
Acc
urac
y
FidelityBrightness Accuracy
Color Histogram AccuracyFace Detection Accuracy
Bandwidth
0 2000 4000 6000 8000
10000 12000 14000 16000 18000
Full 640x480320x240160x120 80x60 40x30 20x20 0
0.2
0.4
0.6
0.8
1
Ene
rgy
Con
sum
ptio
n (m
J)
Acc
urac
y
FidelityBrightness Accuracy
Color Histogram AccuracyFace Detection Accuracy
No BatchingBatching
Average JPEG SizeFull 374.06KResolution 100 90 75 50 25 5640x480 220.53K 113.83K 47.03K 30.88K 19.36K 5.67K320x240 60.61K 32.89K 14.34K 9.61K 6.18K 1.94K160x120 17.20K 9.92K 4.59K 3.17K 2.13K 826.16b80x60 5.17K 3.23K 1.63K 1.19K 891.75b 459.93b40x30 1.70K 1.18K 725.13b 587.58b 483.87b 344.76b20x20 1000.91b 727.07b 501.09b 435.51b 385.46b 316.62b
0
0.2
0.4
0.6
0.8
1
640x480 320x240 160x120 80x60 40x30 20x20
Filte
r Acc
urac
y
Fidelity
0.9910.899
10095755025
0
0.2
0.4
0.6
0.8
1
640x480 320x240 160x120 80x60 40x30
Filte
r Fal
se N
egat
ive
Rat
e
Fidelity
0.1240.042
10095755025
0
0.2
0.4
0.6
0.8
1
640x480 320x240 160x120 80x60
Filte
r Fal
se N
egat
ive
Rat
e
Fidelity
0.575
0.281
10095755025
0
0.2
0.4
0.6
0.8
1Fi
lter A
ccur
acy
Fidelity
0.650
0.424
10095755025
0
0.2
0.4
0.6
0.8
1
Filte
r Fal
se N
egat
ive
Rat
e
Fidelity
0.040
10095755025
640x480 320x240 160x120 80x60 40x30 20x20
0
0.2
0.4
0.6
0.8
1
640x480 320x240 160x120 80x60 40x30
Filte
r Acc
urac
y
Fidelity
0.9070.803
10095755025
640x480 320x240 160x120 80x60
