Computational Vision Jitendra Malik University of California at Berkeley Jitendra Malik University...
-
date post
21-Dec-2015 -
Category
Documents
-
view
222 -
download
1
Transcript of Computational Vision Jitendra Malik University of California at Berkeley Jitendra Malik University...
Computational VisionComputational Vision
Jitendra Malik
University of California at Berkeley
Jitendra Malik
University of California at Berkeley
Taxonomy of Vision ProblemsTaxonomy of Vision Problems
• Reconstruction:– estimate parameters of external 3D world.
• Visual Control:– visually guided locomotion and manipulation.
• Segmentation:– partition I(x,y,t) into subsets of separate objects.
• Recognition:– classes: face vs. non-face,
– activities: gesture, expression.
• Reconstruction:– estimate parameters of external 3D world.
• Visual Control:– visually guided locomotion and manipulation.
• Segmentation:– partition I(x,y,t) into subsets of separate objects.
• Recognition:– classes: face vs. non-face,
– activities: gesture, expression.
ReconstructionReconstruction
• Computer graphics is the forward problem: given scene geometry, reflectances and lighting, synthesize an image.
• Computer vision must address the inverse problem: given an image/multiple images, reconstruct the scene geometry, reflectacnes and illumination.
• Computer graphics is the forward problem: given scene geometry, reflectances and lighting, synthesize an image.
• Computer vision must address the inverse problem: given an image/multiple images, reconstruct the scene geometry, reflectacnes and illumination.
Recovering geometryRecovering geometry
• Historical roots in photogrammetry and analysis of 3D cues in human vision
• Single images adequate given knowledge of object class
• Multiple images make the problem easier, but not trivial as corresponding points must be identified.
• Historical roots in photogrammetry and analysis of 3D cues in human vision
• Single images adequate given knowledge of object class
• Multiple images make the problem easier, but not trivial as corresponding points must be identified.
Taj MahalTaj Mahalmodeled frommodeled from
one photographone photographby G. Borshukovby G. Borshukov
Recovered Campus ModelRecovered Campus Model
Campanile + 40 Buildings (Debevec et al)Campanile + 40 Buildings (Debevec et al)
Inverse Global Illumination (Yu et al)Inverse Global Illumination (Yu et al)
Reflectance Properties
Radiance Maps
Geometry Light Sources
Challenges in ReconstructionChallenges in Reconstruction
• Finding correspondences automatically
• Optimal estimation of structure from n views under perspective projection
• Models of reflectance and texture for natural materials and objects
• Finding correspondences automatically
• Optimal estimation of structure from n views under perspective projection
• Models of reflectance and texture for natural materials and objects
ControlControl
• Visual feedback signal for control of manipulation tasks such as grasping, moving and assembly
• Visual feedback for guiding locomotion– Obstacle avoidance for a moving robot– Lateral and longitudinal control of driving
• Visual feedback signal for control of manipulation tasks such as grasping, moving and assembly
• Visual feedback for guiding locomotion– Obstacle avoidance for a moving robot– Lateral and longitudinal control of driving
Challenges in controlChallenges in control
• Delay in feedback loop due to visual processing
• Hierarchies in sensory motor control– Open loop or closed loop– Discrete planning or continuous control
• Delay in feedback loop due to visual processing
• Hierarchies in sensory motor control– Open loop or closed loop– Discrete planning or continuous control
Boundaries of image regions defined by a number of attributes
Boundaries of image regions defined by a number of attributes
– Brightness/color
– Texture
– Motion
– Stereoscopic depth
– Familiar configuration
– Brightness/color
– Texture
– Motion
– Stereoscopic depth
– Familiar configuration
Approaches Approaches
• Fitting a piecewise smooth surface to the image e.g. Mumford and Shah
• Probabilistic Inference using Markov Random Field model of image e.g. Geman and Geman
• Graph partitioning using spectral techniques e.g. Shi and Malik
• Fitting a piecewise smooth surface to the image e.g. Mumford and Shah
• Probabilistic Inference using Markov Random Field model of image e.g. Geman and Geman
• Graph partitioning using spectral techniques e.g. Shi and Malik
Image Segmentation as Graph PartitioningImage Segmentation as Graph PartitioningBuild a weighted graph G=(V,E) from image
V: image pixels
E: connections between pairs of nearby pixels
region
same the tobelong
j& iy that probabilit :ijW
Partition graph so that similarity within group is large and similarity between groups is small -- Normalized Cuts [Shi&Malik 97]
Challenges in SegmentationChallenges in Segmentation
• Interaction of multiple cues
• Local measurements to global percepts
• Interplay of image-driven and object model driven processing
• Interaction of multiple cues
• Local measurements to global percepts
• Interplay of image-driven and object model driven processing
RecognitionRecognition
• Possible for both instances or object classes (Mona Lisa vs. faces or Beetle vs. cars)
• Tolerant to changes in pose and illumination, and occlusion
Challenges in recognitionChallenges in recognition
• Unified framework for segmentation and recognition
• Representing shape variability in a category
• Interplay of discriminative vs generative models
• Unified framework for segmentation and recognition
• Representing shape variability in a category
• Interplay of discriminative vs generative models
Core disciplinesCore disciplines
• Geometry – Differential geometry– Projective geometry
• Probability and Statistics– Reconstruction = estimation– Control = decision theory– Segmentation = clustering– Recognition = classification
• Geometry – Differential geometry– Projective geometry
• Probability and Statistics– Reconstruction = estimation– Control = decision theory– Segmentation = clustering– Recognition = classification