Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire...
Transcript of Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire...
![Page 1: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/1.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 1 Digital Image Processing: Bernd Girod, © 2013-2014 Stanford University -- Introduction 1
Digital Image Processing EE368/CS232
Prof. Gordon Wetzstein (previously taught by Prof. Bernd Girod) Department of Electrical Engineering Stanford University
![Page 2: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/2.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 2
Imaging
[Albrecht Dürer, 1525]
![Page 3: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/3.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 3
Imaging
■ Most images are defined over a rectangle
■ Continuous in amplitude and space
X
X
y
y
■ Image: a visual representation in form of a function f(x,y)
where f is related to the brightness (or color) at point (x,y)
[Albrecht Dürer, 1525]
![Page 4: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/4.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 4
Imaging
■ Most images are defined over a rectangle
■ Continuous in amplitude and space
■ Image: a visual representation in form of a function f(x,y)
where f is related to the brightness (or color) at point (x,y)
Dark chamber with lenses [Kircher 1646]
![Page 5: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/5.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 5
Digital Images and Pixels
■ Digital image: discrete samples f [x,y] representing continuous image f (x,y)
■ Each element of the 2-d array f [x,y] is called a pixel or pel (from “picture element“)
200x200 100x100 50x50 25x25
![Page 6: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/6.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 6
Color Components
Red R[x,y] Green G[x,y] Blue B[x,y]
Monochrome image
R[x,y] = G[x,y] = B[x,y] 20 μm
![Page 7: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/7.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 7
Why do we process images?
■ Acquire an image – Correct aperture and color balance
– Reconstruct image from projections
■ Prepare for display or printing – Adjust image size
– Color mapping, gamma-correction, halftoning
■ Facilitate picture storage and transmission – Efficiently store an image in a digital camera
– Send an image from space
■ Enhance and restore images – Touch up personal photos
– Color enhancement for security screening
■ Extract information from images – Read 2-d bar codes
– Character recognition
■ Many more ... image processing is ubiquitous
![Page 8: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/8.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 8
Image Processing Examples
source: M. Borgmann, L. Meunier, EE368 class project, spring 2000.
Mosaic from 21 source images
Mosaic from 33 source images
![Page 9: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/9.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 9
Image Processing Examples
Face morphing
Source: Yi-Wen Liu and Yu-Li Hsueh, EE368 class project, spring 2000.
![Page 10: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/10.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 10
Image Processing Examples
Face Detection
source: Henry Chang, Ulises Robles, EE368 class project, spring 2000.
![Page 11: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/11.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 11
Image Processing Examples
source: Michael Bax, Chunlei Liu, and Ping Li, EE368 class project, spring 2003.
Face Detection
![Page 12: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/12.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 12
Image Processing Examples
Face Detection
Face Blurring for Privacy Protection
![Page 13: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/13.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 13
Image Processing Examples
http://cs.stanford.edu/group/roadrunner/stanley.html
![Page 14: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/14.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 14
EE368 Spring 2006 Project:
Visual Code Marker Recognition
![Page 15: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/15.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 15
EE368 Spring 2007 Project:
Painting Recognition
2 1 3 4
6 5 7 8
9 10
![Page 16: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/16.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 16
EE368 Spring 2007 Project:
Painting Recognition
![Page 17: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/17.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 17
Painting Recognition for Augmented Reality
Right-eye LCD
Camera
Android controller
Left-eye LCD
![Page 18: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/18.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 18
EE368 Spring 2008 Project:
CD Cover Recognition
![Page 19: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/19.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 19
CD Cover Recognition on Cameraphone
![Page 20: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/20.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 20
EE368/CS232 Topics
■ Point operations/combining images/histograms
■ Color science
■ Image thresholding/segmentation
■ Morphological image processing
■ Image filtering, deconvolution, template matching
■ Eigenimages, Fisherimages
■ Edge detection, keypoint detection
■ Scale-space image processing
■ Image matching, image registration
![Page 21: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/21.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 21
Image Processing and Related Fields
Artificial Intelligence
Robotics, Inspection,
Photogrammetry
Imaging
Machine learning
M-d Signal
Processing Image coding
Optical Engineering
Computer Vision
Machine Vision
Computer Graphics
Statistics, Information
Theory
Visual
Perception
Display
Technology
Computational Photography
Image
Processing
![Page 22: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/22.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 22
Other Courses ■ How does the course relate to computer vision classes?
CS131 – Foundations CS231M - Mobile
CS231A - Introduction
CS231B – Cutting Edge
CS231n – Conv Net
CS431 – High-level
CS331 – Reading
EE368 / CS232
![Page 23: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/23.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 23
Other Courses ■ How does the course relate to computer vision classes?
■ CS 148: Introduction to Computer Graphics and Imaging
■ PSYCH 221: Applied Vision and Image Systems Engineering
■ EE 367 / CS 448I: Computational Imaging and Display
■ EE 257: Virtual Reality and Sensing
■ CS 178: Digital Photography
■ CS 448A: Computational Photography
CS131 – Foundations CS231M - Mobile
CS231A - Introduction
CS231B – Cutting Edge
CS231n – Conv Net
CS431 – High-level
CS331 – Reading
EE368 / CS232
not actively taught, but archived!
![Page 24: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/24.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 24
EE368/CS232 Organisation
■ Lectures ● MWF 1:30 pm – 2:50 pm in Gates B03 for 7 weeks
● Attendance highly recommended.
● Lecture videos (recordings from previous years) on OpenEdX: view after class, or before, or not at all.
■ Problem session: Fr 4:30 – 5:20 pm in Gates B03 for 7 weeks
■ Office hours ● Gordon Wetzstein: Wed 3 pm – 4 pm (after class), Packard 236
● Kushagr Gupta (TA): Mo 5-7 pm, Packard 277
■ Class Piazza page: https://piazza.com/stanford/fall2015/ee368
![Page 25: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/25.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 25
EE368/CS232 Organisation
■ Class website: ● http://web.stanford.edu/class/ee368/
![Page 26: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/26.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 26
EE368/CS232 Weekly Assignments
■ Weekly problem assignments ● Handed out Mondays, correspond to the lectures of that particular week
● About 8-12 hours of work, requires computer + Matlab
● Discussions among students encouraged, however, individual solution must be submitted.
● Due 9 days later (Wednesday 1 pm).
■ Homework submission: ● Electronic online submission via Gradescope, entry code M3PZ29
■ Weekly lecture review and online quizzes ● Multiple choice questions covering the lectures on OpenEdX (https://suclass.stanford.edu)
● Review the corresponding module, if you are uncertain about your answer
● Graded, solve individually, due at the same time as corresponding problem assigments
■ First assignment handed out on September 21 (first day of class)
![Page 27: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/27.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 27
EE368/CS232 Midterm
■ 24-hour take-home exam
■ Problems similar to weekly assignments
■ Typically requires 5-6 hours of work
■ 3 slots one week after the last lecture, November 11-13, 2015
![Page 28: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/28.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 28
EE368/CS232 Final Project
■ Individual or group project, plan for about 50-60 hours per person
■ Develop, implement and test/demonstrate an image processing algorithm
■ Project proposal due: October 16, 11:59 p.m.
■ Project presentation: Poster session, December 2, 2015, 4-6:30 p.m.
■ Remote SCPD students can alternatively submit a narrated video presentation
■ Submission of written report and source code: December 4, 2015, 11:59 p.m.
![Page 29: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/29.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 29
EE368/CS232 Grading
■ Online quizzes: 10%
■ Homework problems: 20%
■ Midterm: 30%
■ Final project: 40%
■ No final exam.
![Page 30: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/30.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 30
In-class Discussions and iClickers
■ Brief in-class quizzes integrated into the lectures
■ iClickers allow you to share your answers instantaneously and anonymously.
■ It’s o.k. to make mistakes; you will not be graded.
■ Take an iClicker before each class and return afterwards.
Power
Selection button
![Page 31: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/31.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 31
SCIEN Laboratory
■ SCIEN = Stanford Center for Image Systems Engineering (http://scien.stanford.edu)
■ Exclusively a teaching laboratory
■ Location: Packard room 021
■ 20 Linux PCs, scanners, printers etc. ● Matlab with Image Processing Toolbox
● Android development environment
■ Access: ● Door combination for lab entry will be provided by TA
● Account on SCIEN machines will be provided to all enrolled in class
![Page 32: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/32.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 32
Mobile image processing (optional)
■ 40 Motorola DROID cameraphones available for class projects (must be returned after, sorry)
■ Lectures on Android image processing online
■ Android development environment on your own computer or in SCIEN lab
■ Programming in Java (C++ for OpenCV)
![Page 33: Digital Image Processingweb.stanford.edu/class/ee368/Handouts/Lectures//2015_Autumn/1... · Acquire an image –Correct aperture and color balance –Reconstruct image from projections](https://reader036.fdocuments.net/reader036/viewer/2022070113/605f2251eafa5b58dc35da72/html5/thumbnails/33.jpg)
Digital Image Processing: Bernd Girod, © 2013-2015 Stanford University -- Introduction 33
Reading ■ Slides available as pdf files on the class website (click on for source code and data)
http://www.stanford.edu/class/ee368/handouts.html
■ Popular text books ● William K. Pratt, „Introduction to Digital Image Processing,“ CRC Press, 2013. ● R. C. Gonzalez, R. E. Woods, „Digital Image Processing,“ 3rd edition, Prentice-Hall, 2008. ● A. K. Jain, „Fundamentals of Digital Image Processing,“ Prentice-Hall, Addison-Wesley, 1989.
■ Software-centric books ● R. C. Gonzalez, R. E. Woods, S. L. Eddins, „Digital Image Processing using Matlab,“
2nd edition, Pearson-Prentice-Hall, 2009. ● G. Bradski, A. Kaehler, „Learning OpenCV,“ O‘Reilly Media, 2008.
■ Comprehensive state-of-the-art ● Al Bovik (ed.), „The Essential Guide to Image Processing,“ Academic Press, 2009.
■ Journals/Conference Proceedings ● IEEE Transactions on Image Processing ● IEEE International Conference on Image Processing (ICIP) ● IEEE Computer Vision and Pattern Recognition (CVPR) ● ....