Lecture 05: Vision
-
Upload
university-of-colorado-at-boulder -
Category
Documents
-
view
405 -
download
3
description
Transcript of Lecture 05: Vision
Introduction to RoboticsPerception II
CSCI 4830/7000February 15, 2010
Nikolaus Correll
Review: Sensing
• Important: sensors report data in their own coordinate frame
• Examples from the exercise– Accelerometer of Nao– Laser scanner
• Treat like forward kinematics
Today
• Perception using vision• Practical angle:– Why is vision hard– Basic image processing– How to combine image processing primitives into
object recognition• OpenCV / SwisTrack
Why is Vision Hard?The difference between seeing and perception.
Gary Bradski, 2009 4
What to do? Maybe we should try to find edges ….
Gary Bradski, 2005
5
• Depth discontinuity• Surface orientation
discontinuity• Reflectance
discontinuity (i.e., change in surface material properties)
• Illumination discontinuity (e.g., shadow)
Slide credit: Christopher Rasmussen
But, What’s an Edge?
To Deal With the Confusion, Your Brain has Rules...
That can be wrong
We even see invisible edges…
And surfaces …
We need to deal with 3D Geometry
9
Perception is ambiguous … depending on your point of view!
Graphic by Gary Bradski
And Lighting in 3D
Which square is darker?
Lighting is Ill-posed …Perception of surfaces depends on lighting assumptions
11Gary Bradski (c) 2008 11
Contrast
12
Which one is male and which one is female?
Illusion by: Richard Russell, Harvard University
Russell, R. (2009) A sex difference in facial pigmentation and its exaggeration by cosmetics. Perception, (38)1211-1219
Frequency
Color
http://briantobin.info/2009/06/lost-and-found-visual-illusion.html
Pin-hole Model
Pin-Hole Camera
A. Efros
Aperture
Increasing Aperture: Lens
Thin Lens
Objects need to have the right distance to be in focus -> Depth-from-Focus method
Thresholds
2020
Screen shots by Gary Bradski, 2005
http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm
Canny Edge Detector
21Gary Bradski (c) 2008 21
Morphological Operations Examples• Morphology - applying Min-Max. Filters and its combinations
Opening IoB= (IB)BDilatation IBErosion IBImage I
Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - IGrad(I)= (IB)-(IB)
Stereo Calibration
Gary Bradski (c) 2008 2323
Screen shots and charts by Gary Bradski, 2005
3D Stereo Vision• Find Epipolar
lines:
• Triangulate points:
• Align images:
• Depth:
Example: Tomato-Picking Robot
• Challenges– Foliage– Reflections– Varying size and shape– Varying color– Partly covered fruits
http://swistrack.sourceforge.net
N. Correll, N. Arechiga, A. Bolger, M. Bollini, B. Charrow, A. Clayton, F. Dominguez, K. Donahue, S. Dyar, L. Johnson, H. Liu, A. Patrikalakis, T. Robertson, J. Smith, D. Soltero, M. Tanner, L. White, D. Rus. Building a Distributed Robot Garden. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1509-1516, St. Louis, MO.
Filter-based object recognition
• Filter image– Sobel– Hough transform– Color– Spectral
highlights– Size and shape
• Weighted sum of filters highlights object location
Sobel Hough Color SpectralHighlights
Group exercise
• Object recognition– Goal– Players– Ball– Field
Homework
• Read sections 4.2-5 (pages 145-180)• Questionnaire on CU Learn