CSE 527: Intro. to Computer Vision - Welcome to alumni...
6
CSE 527: Intro. to Computer Vision www.cs.sunysb.edu/~cse527 Instructor: Prof. M. Alex O. Vasilescu Email: [email protected] Phone: 631 632-8457 Office: 1421 Vision What does it mean, to see? “to know what is where by looking”. How to discover from images what is present in the world, where things are, what actions are taking place. from Marr, 1982 Vision Problems Recognize objects people we know things we own Locate objects in space to pick them up Track objects in motion catching a baseball avoiding collisions with cars on the road – auto navigation Recognize actions walking, running, pushing Why study Computer Vision? Images and movies are everywhere Fast-growing collection of useful applications building representations of the 3D world from pictures automated surveillance (who’s doing what) face finding movie post-processing HCI Various deep and attractive scientific mysteries how does object recognition work? Greater understanding of human vision Why study Computer Vision? Images and movies are everywhere Fast-growing collection of useful applications building representations of the 3D world from pictures automated surveillance (who’s doing what) face finding movie post-processing HCI Various deep and attractive scientific mysteries how does object recognition work? Greater understanding of human vision Structure from Motion (Tomasi and Kanade 1992) Video Features 3D Reconstruction
Transcript of CSE 527: Intro. to Computer Vision - Welcome to alumni...
1
CSE 527: Intro. to Computer Vision
www.cs.sunysb.edu/~cse527
Instructor: Prof. M. Alex O. VasilescuEmail: [email protected]: 631 632-8457Office: 1421
VisionWhat does it mean, to see?
“to know what is where by looking”.
How to discover from images what is present in the world, where things are, what actions are taking place.
from Marr, 1982
Vision ProblemsRecognize objects
people we knowthings we own
Locate objects in spaceto pick them up
Track objects in motioncatching a baseballavoiding collisions with cars on the road – auto navigation
Recognize actionswalking, running, pushing
Why study Computer Vision?Images and movies are everywhereFast-growing collection of useful applications
building representations of the 3D world from picturesautomated surveillance (who’s doing what)
face findingmovie post-processingHCI
Various deep and attractive scientific mysterieshow does object recognition work?
Greater understanding of human vision
Why study Computer Vision?Images and movies are everywhereFast-growing collection of useful applications
building representations of the 3D world from picturesautomated surveillance (who’s doing what)
face findingmovie post-processingHCI
Various deep and attractive scientific mysterieshow does object recognition work?
Greater understanding of human vision
Structure from Motion(Tomasi and Kanade 1992)
Video Features
3D Reconstruction
2
Photo Collections Panoramic imaging
Image and video registration
Why study Computer Vision?Images and movies are everywhereFast-growing collection of useful applications
building representations of the 3D world from picturesautomated surveillance (who’s doing what)
face findingmovie post-processingHCI
Various deep and attractive scientific mysterieshow does object recognition work?
Greater understanding of human vision
Tracking
Why study Computer Vision?Images and movies are everywhereFast-growing collection of useful applications
building representations of the 3D world from picturesautomated surveillance (who’s doing what)
face finding
movie post-processingHCI
Various deep and attractive scientific mysterieshow does object recognition work?
Greater understanding of human vision
http://www.ri.cmu.edu/projects/project_271.html
3
http://www.ri.cmu.edu/projects/project_320.html
Several million sold (most of any digital camera). Imaging chip is Mitsubishi Electric’s “Artificial Retina” CMOS detector.
Nintendo Game Boy Camera
Why study Computer Vision?Images and movies are everywhereFast-growing collection of useful applications
building representations of the 3D world from picturesautomated surveillance (who’s doing what)
face findingmovie post-processingHCI
Various deep and attractive scientific mysterieshow does object recognition work?
Greater understanding of human vision
Detect ground plane in video andintroduce pictures on them
Insert new objects
Video example: http://break.com/index/ufo-lands-on-guys-desk.html
Video Summary
4
Black or WhiteFace DetectionFace LocalizationSegmentationFace TrackingFacial features localizationFacial features trackingMorphing
www.youtube.com/watch?v=ZI9OYMRwN1Q
Why study Computer Vision?Images and movies are everywhereFast-growing collection of useful applications
building representations of the 3D world from picturesautomated surveillance (who’s doing what)face findingmovie post-processingHCI
Various deep and attractive scientific mysterieshow does object recognition work?
Greater understanding of human vision
Game: Decathlete Optical-flow-based Decathlete figure motion analysis
Decathlete javelin throw Decathlete javelin throw
5
Decathlete 100m hurdles Course Outline
Introduction and Math ReviewWhat is Computer Vision?Tutorial on Linear Algebra and Matlab
PART I: 2D VisionImage FormationAppearance-Based MethodsFeature Extraction2D Shape Models
PART II: 3D Vision3D Shape Estimation from Shading, from MotionSurface Reconstruction
PART I: 2D VisionImage Formation
Cameras, Lenses, and SensorsColor and Image Statistics
PART I: 2D VisionAppearance-Based Methods
Statistical Linear Models: PCA, ICA, FLDNon-negative Matrix Factorization, Sparse Matrix FactorizationStatistical Tensor Models: Multilinear PCA, Multilinear ICAPerson and Activity Recognition
PART I: 2D Vision
Feature Extraction:Linear filters and edges Feature extraction (corners and blobs)Representations: Gaussian Pyramids, Laplacian Pyramids, Steerable PyramidsApplication: face detection
Image
Oriented, multi-scale representation
PART I: 2D Vision2D Shape Models
Physically Based Models:Mass-Spring SystemsActive Contours (Snakes) - energy minimization, regularizationwww.youtube.com/watch?v=5se69vcbqxA
Statistical Shape ModelsActive Shape ModelsActive Appearance ModelsKalman FiltersParticle FiltersMean Shift
6
Estimation of 3D Geometry:Camera calibration, Epipolar GeometryStereo, Multi-View GeometryShape from ShadingStructure from Motion, Optical Flow Surface Reconstruction – energy minimization, regularization
PART II: 3D Vision General CommentsPrerequisites:
Linear Algebra!!!Some image processing, signal processing is useful, but not required
Emphasis on programming projects!Building something from scratch (Matlab!)
Textbooks and Reading material:Computer Vision: A Modern Approach, David Forsyth and Jean Ponce., Prentice Hall, 2003.Robot Vision, Berthold HornSelected journal articles
Grading
30%Final Project:An original implementation of a new or published ideaA detailed empirical evaluation of an existing implementation
of one or more methods5-10 page report
Project proposal not longer than two pages must be submitted and approved before the end of March.
10%Class Participation
20%One take-home exam. (Take-home exams may not be discussed.)
40%Problem Sets (~6) with lab exercises in Matlab.Problem sets may be discussed, but all written work and coding must be done individually.
30%
10%
0%
60%
Administrative StuffLate Policy
Seven late days total, to be spent wisely
CheatingLet’s not embarrass ourselvesAll resources must be acknowledged
SoftwareMATLAB!!!
Internet ResourcesMatlab:
University of Colorado Matlab TutorialsA decent collection of Matlab tutorials, including one focusing on image processing.
Matlab Image Processing TutorialA short introduction to the manipulation of images in Matlab, including an introduction to principal components analysis via eigenfaces.
Computer Vision: Computer Vision HomepageFace Recognition HomepageFace Detection Homepage
Introductions
Name, year, supervisorWhy do you want to take this class?What are you hoping to learn?
