Themes in Computer Vision Carlo Tomasi. Applications autonomous cars, planes, missiles, robots,......
-
Upload
shanon-hunter -
Category
Documents
-
view
218 -
download
1
Transcript of Themes in Computer Vision Carlo Tomasi. Applications autonomous cars, planes, missiles, robots,......
Themes in Computer Vision
Carlo Tomasi
Applications
• autonomous cars, planes, missiles, robots, ...• space exploration• aid to the blind, ASL recognition• manufacturing,
quality control• surveillance, security• image retrieval• medical imaging• ...• perceptual input for
cognition
(CMU NavLab ‘90)
Vision is Effortless to Us
• driving a car
• threading a needle
• recognizing a distant, occluded object
• understanding (flat!) pictures
• perceive the mood of a painting
Technical Difficulties
• 512x512x3x30 ≈ 23.5MB/s was a problem 10 years ago
• technology just gotgood enough
• great opportunity!
Fundamental Challenges I
• 3D2D implies information loss
• sensitivity to errors
• need for models
graphics
vision
Reconstruction and Geometry
must use redundancy toaddress sensitivity to noise
Reconstruction Example
(Tomasi & Kanade ‘91)
Fundamental Challenges II
• Appearance changes with viewpoint, i.e., the same thing looks different• Geometric changes: surface slant depends on
viewpoint• Photometric changes: surface brightness and
color depend on viewpoint• Occlusions: what is hidden depends on
viewpoint
• Ambiguity: different things look similar• Correspondence is hard
Photometric and Geometric Change
Occlusion
?
Technicality: Motion Blur
Wrong Correspondence
Simple Images are Harder
(Birchfield and Tomasi ‘01)
Models
• must be insensitive to• viewing position
changes• lighting changes• object configuration
changes• occlusion• clutter
• must be sensitive to• object changes!
Low-Level Models are General
Model: surfaces are smooth, connected
(Marr and Poggio ‘80)
Higher-Level Models Work Better…
•… when they are right• (and much worse when they are wrong)
(Lin and Tomasi ‘01)
State of the Artle
ft in
put i
mag
eground truth disparity
our
resu
ltdisparity error
(Lin and Tomasi, 01)
Fundamental Challenges III• An old problem in the
new context of recognition:• Variation of appearance:
Objects change over time, with context, viewpoint, lighting, pose, expression,…
• Similarity: Different objects look similar
• [BTW, objects do not always appear in isolation…]
(US Army FERET Database)
Modeling Images as Points12
n
...
...
1
2
n
principal componentsform an approximate basisfor all the images in the set
... ... ... ... ... ... ... ...
Example: Eigenfaces
(Turk, Pentland ‘91; Murase-Nayar ‘93; many others)
........................
...
=
the projection of a new imageonto the eigenbasis isa compressed representationof that image
can use this to recognize faces,synthesize new images, ...
Fundamental Challenges IV:
“read my lips”
“run”• Variation, self-occlusion,occlusion, clutter, …
Motions can be complex
Simple Models Are Fast
(Birchfield ‘98)
a head is an ellipse with two colors,surrounded by strong intensity gradients
(Bregler ‘93)
2D Articulated Models for Tracking
3D Models are More Accurate…
•… when they are right• [BTW, why is she wearing a black shirt?]
(Isard & Blake ‘99)
Probabilistic Models Handle Uncertainty
• world state , observation (image) • prior P()
• colors change moderately (?)• arms move with limited acceleration (boxing?)• the height of a head can only change so much (dancing?)• contours are smooth and change smoothly• balls follow the laws of gravity• …
• sensor model P(|)• image motion can be measured only so well• motion blurs the image• noise corrupts pixel values• ...
Bayesian Tracking
• Bayes’ rule: P(|) P(|) P()
• what is the world state likely to be, given that we observed the image ?
(Isard & Blake ‘99)
Even Higher Models May Be Needed
[MY COMPUTER CAN UNDERSTAND SIGN] computer No(1(HandsIpsi 1 1 0 S Out Down, NeutralIpsi 0 0 0 S Out Down)( ,-) 0(" " 0 -1 " " ", " " " " " " ") (",-) 0(" " -1 0 " " ", " " " " " " ") (",-) 0(" " 0 1 " " ", " " " " " " ") (",-) 1(" " 1 0 " " ", " " " " " " ")) understand No(1(HandIn 0 0 0 X Out Contra,NeutralOut 0 0 0 D Up Contra)(-,-) "(" 1 " " " " ", " " " " " " "))signs No(1( 0 0 0 B Up Out, - - - - - - -) (-,-) "(" 1 0 0 " " ", - - - - - - -))can No(1(HandUp 0 0 0 Out Contra,NeutralOut 0 0 -1 B Out Up) (-,-) "(" " " " " " ", " " " 1 " " "))
(Richards & Tomasi ‘02)
Fundamental Challenge V:Images are Diverse
Previous Work in Image Retrieval
Hulton Deutsch
Color and Texture Models
orientation
scal
e
text
ure
Image Distances
(Rubner & Tomasi ‘97)
(Rubner & Tomasi ‘97)
Retrieval by Refinement - 1
(Rubner & Tomasi ‘97)
Retrieval by Refinement - 2
(Rubner & Tomasi ‘97)
Vision is AI Complete
• Vision is an inverse problem
• Strong models of the world are required
• Vision implies reasoning about the world
• Vision is AI