• http://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/index.html

• Check web page often• T,R 12:30-1:50pm PSCB (Phy Sci Class Blg) 140• Course intro handout

Computational Photography and Vision (CS116)

Deva Ramanandramanan@ics.uci.edu

Slides adapted from Alyosha Efros, Rick Szeliski and Steve Seitz

Agenda• Intros• Computational photography/vision overview• Course overview• Image processing – let’s dive in

Readings (due next class)• Online Book (link from course website): 

Richard Szeliski, Computer Vision: Algorithms and Applications – Intro: Ch 1.0 & 2.1

Ungraded HW (due next class)• MATLAB tutorial (link from course website)

About me

Deva Ramanan

Relatively new faculty (2cnd class ever!)

My research focus is on computer vision

Part of new Computational Vision Lab

http://vision.ics.uci.edu/

What is computational photography?Convergence of image processing, computer

vision, computer graphics and photography

Digital photography:• Simply replaces traditional sensors and recording by digital

technology• Involves only simple image processing

Computational photography• More elaborate image manipulation, more computation• New types of media (panorama, 3D, etc.)• Camera design that take computation into account

GRAPHICS

What is computer graphics? (3D->2D)

3D geometry

physics

Simulation

projection

What is computer vision? (2D->3D)

3D geometry

physics

Estimation

What is computer vision?

Terminator 2

Every picture tells a story

Goal of computer vision is to write computer programs that can interpret images

Can computers match (or beat) human vision?

Yes and no (but mostly no!)• humans are much better at “hard” things• computers can be better at “easy” things

Human perception has its shortcomings…

Sinha and Poggio, Nature, 1996

Copyright A.Kitaoka 2003

Current state of the art

The next slides show some examples of what current vision systems can do

Earth viewers (3D modeling)

Image from Microsoft’s Virtual Earth(see also: Google Earth)

Photo Tourism

http://phototour.cs.washington.edu/http://labs.live.com/photosynth/

Optical character recognition (OCR)

Digit recognition, AT&T labshttp://www.research.att.com/~yann/

Technology to convert scanned docs to text• If you have a scanner, it probably came with OCR software

License plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Face detection

Many new digital cameras now detect faces• Canon, Sony, Fuji, …

Smile detection?

Sony Cyber-shot® T70 Digital Still Camera

Object recognition (in supermarkets)

LaneHawk by EvolutionRobotics“A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “

Face recognition

Who is she?

Vision-based biometrics

“How the Afghan Girl was Identified by Her Iris Patterns” Read the story

Login without a password…

Fingerprint scanners on many new laptops,

other devices

Face recognition systems now beginning to appear more widely

http://www.sensiblevision.com/

Object recognition (in mobile phones)

This is becoming real:• Microsoft Research• Point & Find, Nokia

The Matrix movies, ESC Entertainment, XYZRGB, NRC

Special effects: shape capture

Pirates of the Carribean, Industrial Light and Magic

Special effects: motion capture

Special effects: image-based rendering

Sports

Sportvision first down lineNice explanation on www.howstuffworks.com

Smart cars

Mobileye• Vision systems currently in high-end BMW, GM, Volvo models • By 2010: 70% of car manufacturers.

Slide content courtesy of Amnon Shashua

Vision-based interaction (and games)

Nintendo Wii has camera-based IRtracking built in. See Lee’s work atCMU on clever tricks on using it tocreate a multi-touch display!

Digimask: put your face on a 3D avatar.

“Game turns moviegoers into Human Joysticks”, CNETCamera tracking a crowd, based on this work.

Robotics

http://www.robocup.org/NASA’s Mars Spirit Roverhttp://en.wikipedia.org/wiki/Spirit_rover

Medical imaging

Image guided surgeryGrimson et al., MIT

3D imagingMRI, CT

Current state of the artYou just saw examples of current systems.

• Many of these are less than 5 years old

This is a very active research area, and rapidly changing• Many new apps in the next 5 years

To learn more about vision applications and companies• David Lowe maintains an excellent overview of vision

companies– http://www.cs.ubc.ca/spider/lowe/vision.html

This coursehttp://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/

index.html

Prerequisites• Calculus, linear algebra + probability helpful• Interest in playing with images

Emphasis on programming projects!• Best way to learn is to build something from scratch

• MATLAB has a low learning curve

• 5 projects (15% of grad) + final exam (25%)

• Project due every 2 weeks

• For larger projects, “part 1” due first week

Project 1: Demosaicing

1) Get feet wet with MATLAB2) Turn raw output of digital camera into a color image

Project 2: hole-filling and blendingThe fun stuff!

Tools: bayesian modelling, differential equations

Project 3: Image re-targeting

Click on video

Tools: combinatorial optimization, dynamic programming

Project 4: Automatic mosaicing

Tools: linear algebra, signal processing

Project 5: Face detection & recognition

Tools: probabilistic modeling

CamerasDon’t need for class, but really cool

Digital SLRs are ideal

Point–and-shoots still nice and not too expensive (<$200)

e.g. Canon A550

ReferencesThere is no required text. Various course notes and papers will be made available.  We will often use an online draft of an upcoming book:

Richard Szeliski, Computer Vision: Algorithms and Applications

There is a number of other fine texts that you can use for general reference:

Computer Vision: The Modern Approach, Forsyth and PonceVision Science: Photons to Phenomenology, Stephen Palmer Multiple View Geometry in Computer Vision, Hartley & Zisserman The Computer Image, Watt and Policarpo Linear Algebra and its Applications, Gilbert Strang

A little bit of teaching philosophy…

1) Prefer discussion vs lectures – ask questions!

2) Readings before class help ‘set the stage’

3) We learn best by doing it ourselves – progamming projects important!

4) Don’t like powerpoint (makes students sleep)

-But visual aides are nice

-Slides will be available online *after* class

-I encourage you to take notes during class

Favor to ask…

Need to boost enrollment - spread the gospel about this cool class!