Lecture 1 - University of Maryland Institute for Advanced Computer
Transcript of Lecture 1 - University of Maryland Institute for Advanced Computer
![Page 1: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/1.jpg)
Lecture 1
CMSC426Computer Vision
![Page 2: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/2.jpg)
Course
• Mondays and Wednesdays– 3:30-4:45, CSIC 3120
• Grading– mid-term 25%– Final 35%– Homework, 30% (every other week)– Quizzes, 10% (surprise, 2 )
![Page 3: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/3.jpg)
Instructor
• Ramani Duraiswami– ([email protected]) 301 405 6710
• Office hours– Thursday 11:00 – 12:30– By appointment
• No official TA• If you have questions meet me
– E-mail support is deprecated
![Page 4: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/4.jpg)
Campus Policies• Absences: Campus policy
– Students claiming a excused absence must apply in writing and furnish documentary support (such as from a health care professional who treated the student) for any assertion that the absence qualifies as an excused absence. The support should explicitly indicate the dates or times the student was incapacitated due to illness. Self-documentation of illness is not itself sufficient support to excuse the absence. An instructor is not under obligation to offer a substitute assignment or to give a student a make-up assessment unless the failure to perform was due to an excused absence. An excused absence for an individual typically does not translate into an extension for team deliverables on a project.
![Page 5: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/5.jpg)
Policies
• Collaborative work– Collaboration is encouraged on understanding
assignments and formulating strategies for solution– Similarly web and other research is encouraged– However, the actual writing of solutions must be done
independently, without access to any material from colleagues, or other sources.
– Exams must be done independently– Any other collaboration is academic dishonesty, and
subject to “academic integrity action.”
![Page 6: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/6.jpg)
Textbook
• Computer Vision: A Modern Approach
• Readings from the text• Recommended strongly
that you get access to the book
• Homework problems may be assigned fromit
![Page 7: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/7.jpg)
Other TextsUNDERGRADUATE1. A Guided Tour of Computer Vision, by V. S. Nalwa,
Addison-Wesley, 1993.2. Introductory Techniques for 3-D Computer Vision, by
Emanuele Trucco, Alessandro Verri, Prentice-Hall, 1998
GRADUATE/ Specialized Reference1. Multiple View Geometry, by Richard Hartley, Andrew
Zisserman, Cambridge University Press, 2000.2. Numerical Recipes in C, by William Press et al.,
Cambridge Univ Press, 1992.3. Pattern Classification and Scene Analysis, by Richard
O. Duda, Peter E. Hart, John Wiley & Sons, 1973.
![Page 8: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/8.jpg)
Online Resources• Computer Vision home page
– http://www.cs.cmu.edu/afs/cs/project/cil/ftp/html/vision.html
• CV Online texts– http://www.dai.ed.ac.uk/CVonline/
• Many others … any good ones please provide them to me
• Course home pagehttp://www.umiacs.umd.edu/~ramani/cmsc426/index.html• – Lecture Slides• – Homework• – Solutions (?)
![Page 9: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/9.jpg)
Why study vision?• Intellectually interesting
– How do we figure out what objects are where?• With cheap digital imaging many applications are
now possible– Multi megapixel cameras are available everywhere– Chips are less than $2 --- every cellphone will have a
camera• Computers and software
packages/libraries/architectures make it possible for people to build useful applications with relatively modest investment of time, effort and resources
• Large commercial interest– Movies, television, live sports broadcasts, advertisements– Post processing of video– Surveillance
![Page 10: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/10.jpg)
Vision
• ``to know what is where, by looking.’’ (Marr).
• Where?• What?
![Page 11: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/11.jpg)
Why is Vision Interesting?
• Psychology– ~ 50% of cerebral cortex is for vision.– Vision is how we experience the world.
• Engineering– Want machines to interact with world.– Digital images are everywhere.
![Page 12: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/12.jpg)
Vision is interdisciplinary
• Geometry• Physics• Prior Knowledge: The nature of objects in
the world (This is the hardest part)• Computer Science: to process, store and
solve problems on images• Numerical Methods/Scientific computing:
to do these efficiently• Probability/Statistics to deal with noise
![Page 13: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/13.jpg)
What do we use vision for?• Recognize objects
– people we know– things we own
• Locate objects in space– to pick them up
• Track objects in motion– catching a baseball– avoiding collisions with cars on the road
• Recognize actions– walking, running, pushing– Emotions, gestures
![Page 14: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/14.jpg)
Vision is
– Deceivingly easy– Deceptive– Computationally demanding– Critical to many applications
![Page 15: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/15.jpg)
Vision is deceivingly easy• We see effortlessly
– seeing seems simpler than “thinking”– we can all “see” but only select gifted people can
solve “hard” problems like chess– we use half of our brains for visual perception!
• All “creatures” see– frogs “see”– birds “see”– snakes “see”
but they do not see alike
![Page 16: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/16.jpg)
Vision is deceivingly easy• The M.I.T. summer vision
program– summer of 1965– point TV camera at stack of
blocks– locate individual blocks
• recognize them from small database of blocks
– describe physical structure of the scene
• support relationships
• Formally ended in 1985
![Page 17: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/17.jpg)
Vision is deceptive
• Vision is an exceptionally strong sensation– vision is immediate– we perceive the visual world as external to ourselves,– but it is a reconstruction within our brains
• we regard how we see as reflecting the world “as it is;” but human vision is– subject to illusions– quantitatively imprecise– limited to a narrow range of frequencies of radiation– passive
![Page 18: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/18.jpg)
![Page 19: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/19.jpg)
![Page 20: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/20.jpg)
Vision is inferential: Light
(http://www-bcs.mit.edu/people/adelson/checkershadow_illusion.html)
![Page 21: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/21.jpg)
Inference
• Humans have to make assumptions about illumination: bump (left) is perceived as hole (right) when upside down
Illumination direction is unknown. It is assumed to come from above
![Page 22: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/22.jpg)
Spectral limitations of human vision• We “see” only a small part of the energy
spectrum of sunlight– we don’t see ultraviolet or lower frequencies
of light– we don’t see infrared or higher frequencies of
light– we see less than .1% of the energy that
reaches our eyes• But objects in the world reflect and emit
energy in these and other parts of the spectrum
![Page 23: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/23.jpg)
Non-human vision• Infrared vision• Polarization vision
– navigation for birds• Ultrasound vision• X-ray vision!• RADAR vision
![Page 24: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/24.jpg)
Computationally Demanding
• Millions of pixels• Hours of video• Thousands of hours of film• Millions of cameras
![Page 25: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/25.jpg)
What is Computer Vision?
• Image Understanding (AI, behavior)• Computer emulation of human vision• A sensor modality for robotics• Inverse of Computer Graphics
Computervision
World model
Computergraphics
World model
http://www.nips.cc/Conferences/2004/Tutorials/slides/szeliskiSlides.pdf
![Page 26: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/26.jpg)
Computer Vision: [Trucco&Verri’98]
![Page 27: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/27.jpg)
Tom Hanks in Forrest Gump
![Page 28: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/28.jpg)
Images from: http://www.symah-vision.fr/
![Page 29: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/29.jpg)
Application: Football first-down line
Requires (1) accurate camera registration; (2) a colour model fordistinguishing foreground from background
www.sportvision.com
![Page 30: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/30.jpg)
Image Enhancement
• High dynamic range photography[Debevec et al.’97; Mitsunaga & Nayar’99]– combine several different exposures
together
From: http://www.nips.cc/Conferences/2004/Tutorials/slides/szeliskiSlides.pdf
![Page 31: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/31.jpg)
Tracking
![Page 32: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/32.jpg)
Panoramic Mosaics
• + + … + =
Slide from: Richard Szeliski’s tutorial on Computer Vision at NIPS, 2004
![Page 33: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/33.jpg)
Medicine/Genetics• Diagnosis
– radiology - read X rays, CAT scans– pathology - read biopsies
• Remote and tele-medicine• Virtual reality surgical assistance
– project images onto head during brain surgery• Genetics: microarray analysis
![Page 34: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/34.jpg)
Transportation• Traffic safety and control
– detection and ticketing of speeding vehicles
– vehicle counting for flow control’• Robot drivers
– convoys• Advanced automobiles
– autonomous parallel parking– road sign detectors and driver alerts– collision avoidance– smart cruise control
Pittsburgh to San Diego!
![Page 35: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/35.jpg)
Agriculture• Safety and quality inspection
– sorting by size - peaches– sorting by shape - potatoes– identifying defects - blemishes on fruit, rot in
potatoes– disease monitoring - chickens
• Robotic farming equipment– robotic harvesters - apple pickers, orange
pickers
![Page 36: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/36.jpg)
• So vision is interesting and useful• We are going to study
![Page 37: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/37.jpg)
Related Fields
• Graphics. “Vision is inverse graphics”.• Visual perception.• Neuroscience.• AI• Learning• Math: eg., geometry, stochastic processes.• Optimization.
![Page 38: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/38.jpg)
Tools Needed for Course
• Math– Calculus– Linear Algebra .
• Computer Science– Algorithms– Programming, will use Matlab.
• Signal Processing (necessar parts will be covered).
![Page 39: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/39.jpg)
SyllabusIntroduction to computer vision. Cameras
Projection and Photometry Matlab introduction & Linear Algebra Refresher Intensities – 2.1D vision.
Boundary detection (edges) i. Linear filtering ii. Discontinuities (edge detection)iii. SNAKEs (finding the bounding contours of objects)Perceptual groupingBoundary detection (Regions). Lightness constancy Color Texture Features (corners, lines). Perceptual Grouping
Mid term Exam
![Page 40: Lecture 1 - University of Maryland Institute for Advanced Computer](https://reader035.fdocuments.net/reader035/viewer/2022070222/613d179f736caf36b7593892/html5/thumbnails/40.jpg)
Syllabus1. 3D Vision
1. Stereo 2. Structure-from-Motion 3. i. Flow 4. ii. 2 frame perspective structure-from-motion5. iii. Multi-frame affine structure-from-motion
2. Application: Image-based Rendering 3. Shading
1. Photometric stereo for Lambertian2. Recognition as linear combinations of intensities.
4. Recognition 1. Template matching 2. Pose determination 3. Invariance
5. Classification 6. Approaches to Vision 7. FINAL EXAM