Lecture1: Introduconto“ComputerVision”vision.stanford.edu/teaching/.../lecture1...cs231a.pdf ·...

Lecture 1 - !!!

Fei-Fei Li!

Lecture 1: Introduc.on to “Computer Vision”

Professor Fei-‐Fei Li Stanford Vision Lab

24-‐Sep-‐12 1

Lecture 1 - !!!

Fei-Fei Li!

Welcome to CS231a: Computer Vision

Slid

e ad

apte

d fr

om S

vetl

ana

Laze

bnik

24-‐Sep-‐12 2

Lecture 1 - !!!

Fei-Fei Li!

Today’s agenda

•  Introduc.on to computer vision •  Course overview

24-‐Sep-‐12 3

Lecture 1 - !!!

Fei-Fei Li!

Quiz?

24-‐Sep-‐12 4

Lecture 1 - !!!

Fei-Fei Li!

What about this?

24-‐Sep-‐12 5

Lecture 1 - !!!

Fei-Fei Li!

Image (or video) Sensing device Interpreting device Interpretations

garden, spring, bridge, water, trees, flower, green, etc.

What is (computer) vision?

24-‐Sep-‐12 6

Lecture 1 - !!!

Fei-Fei Li!

What is it related to?

Computer Vision

Neuroscience

Machine learning

Speech

Informa.on retrieval

Maths

Computer Science

Biology

Engineering

Physics

Robo.cs Cogni.ve sciences

Psychology

graphics,algorithms, system,theory,…

Image processing

24-‐Sep-‐12 7

Lecture 1 - !!!

Fei-Fei Li!

The goal of computer vision •  To bridge the gap between pixels and “meaning”

What we see What a computer sees Sou

rce:

S. N

aras

imha

n

24-‐Sep-‐12 8

Lecture 1 - !!!

Fei-Fei Li!




24-‐Sep-‐12 9

Lecture 1 - !!!

Fei-Fei Li!

1981: Nobel Prize in medicine

Hubel & Wiesel

24-‐Sep-‐12 10

Lecture 1 - !!!

Fei-Fei Li!

Potter, Biederman, etc. 1970s

Human vision is superbly efficient

24-‐Sep-‐12 11

Lecture 1 - !!!

Fei-Fei Li!

Thorpe, et al. Nature, 1996

24-‐Sep-‐12 12

Lecture 1 - !!!

Fei-Fei Li!

Thorpe, et al. Nature, 1996

150 ms !!

24-‐Sep-‐12 13

Lecture 1 - !!!

Fei-Fei Li!

Change blindess

Rensink, O’regan, Simon, etc.

24-‐Sep-‐12 14

Lecture 1 - !!!

Fei-Fei Li!

Rensink, O’regan, Simon, etc.

Change blindess

24-‐Sep-‐12 15

Lecture 1 - !!!

Fei-Fei Li!

segmenta.on

24-‐Sep-‐12 16

Lecture 1 - !!!

Fei-Fei Li!

Percep.on

24-‐Sep-‐12 17

Lecture 1 - !!!

Fei-Fei Li! 24-‐Sep-‐12 18

Lecture 1 - !!!

Fei-Fei Li! 24-‐Sep-‐12 19

Lecture 1 - !!!

Fei-Fei Li!




24-‐Sep-‐12 20

Lecture 1 - !!!

Fei-Fei Li!

The goal of computer vision •  To bridge the gap between pixels and “meaning”

What we see What a computer sees Sou

rce:

S. N

aras

imha

n

24-‐Sep-‐12 21

Lecture 1 - !!!

Fei-Fei Li!

Origins of computer vision: an MIT undergraduate summer project

L. G. Roberts, Machine Percep,on of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.

24-‐Sep-‐12 22

Lecture 1 - !!!

Fei-Fei Li!

What kind of informa.on can we extract from an image?

•  Metric 3D informa.on •  Seman.c informa.on

24-‐Sep-‐12 23

Lecture 1 - !!!

Fei-Fei Li!

Vision as measurement device Real-time stereo Structure from motion

NASA Mars Rover

Pollefeys et al.

Reconstruction from Internet photo collections

Goesele et al.

24-‐Sep-‐12 24

Lecture 1 - !!!

Fei-Fei Li!

Vision as a source of semantic information sky

water

Ferris wheel

amusement park

Cedar Point

12 E

tree

tree

tree

carousel deck

people waiting in line

ride

ride ride

umbrellas

pedestrians

maxair

bench

tree

Lake Erie

people sitting on ride

Objects Activities Scenes Locations Text / writing Faces Gestures Motions Emotions…

The Wicked Twister

Slid

e cr

edit

: Kr

iste

n G

raum

an

24-‐Sep-‐12 25

Lecture 1 - !!!

Fei-Fei Li!

Why study computer vision?

Personal photo albums

Surveillance and security

Movies, news, sports

Medical and scientific images

•  Vision is useful: Images and video are everywhere!

24-‐Sep-‐12 26

Lecture 1 - !!!

Fei-Fei Li!

Why study computer vision? •  Vision is useful •  Vision is interes.ng •  Vision is difficult

– Half of primate cerebral cortex is devoted to visual processing

– Achieving human-‐level visual percep.on is probably “AI-‐complete”

24-‐Sep-‐12 27

Lecture 1 - !!!

Fei-Fei Li!

Why is computer vision difficult?

24-‐Sep-‐12 28

Lecture 1 - !!!

Fei-Fei Li!

Challenges: viewpoint variation

Michelangelo 1475-1564

slide credit: Fei-Fei, Fergus & Torralba

24-‐Sep-‐12 29

Lecture 1 - !!!

Fei-Fei Li!

Challenges: illumination

image credit: J. Koenderink

24-‐Sep-‐12 30

Lecture 1 - !!!

Fei-Fei Li!

Challenges: scale

slid

e cr

edit:

Fei

-Fei

, Fer

gus

& T

orra

lba

24-‐Sep-‐12 31

Lecture 1 - !!!

Fei-Fei Li!

Challenges: deformation

Xu, Beihong 1943 slide credit: Fei-Fei, Fergus & Torralba

24-‐Sep-‐12 32

Lecture 1 - !!!

Fei-Fei Li!

Challenges: occlusion

Magritte, 1957

slide credit: Fei-Fei, Fergus & Torralba 24-‐Sep-‐12 33

Lecture 1 - !!!

Fei-Fei Li!

Challenges: background clutter

slid

e cr

edit:

Sve

tlana

Laz

ebni

k

24-‐Sep-‐12 34

Lecture 1 - !!!

Fei-Fei Li!

Challenges: Motion

slid

e cr

edit:

Sve

tlana

Laz

ebni

k

24-‐Sep-‐12 35

Lecture 1 - !!!

Fei-Fei Li!

Challenges: object intra-‐class varia.on

slid

e cr

edit:

Fei

-Fei

, Fer

gus

& T

orra

lba

24-‐Sep-‐12 36

Lecture 1 - !!!

Fei-Fei Li!

Challenges: local ambiguity

slid

e cr

edit:

Fei

-Fei

, Fer

gus

& T

orra

lba

24-‐Sep-‐12 37

Lecture 1 - !!!

Fei-Fei Li!

Challenges or opportuni.es? •  Images are confusing, but they also reveal the structure of the world through numerous cues

•  Our job is to interpret the cues!

Imag

e so

urce

: J. K

oend

erin

k

24-‐Sep-‐12 38

Lecture 1 - !!!

Fei-Fei Li!

Depth cues: Linear perspec.ve

slid

e cr

edit:

Sve

tlana

Laz

ebni

k

24-‐Sep-‐12 39

Lecture 1 - !!!

Fei-Fei Li!

Depth cues: Aerial perspec.ve

slid

e cr

edit:

Sve

tlana

Laz

ebni

k

24-‐Sep-‐12 40

Lecture 1 - !!!

Fei-Fei Li!

Depth ordering cues: Occlusion

Sou

rce:

J. K

oend

erin

k

24-‐Sep-‐12 41

Lecture 1 - !!!

Fei-Fei Li!

Shape cues: Texture gradient

slid

e cr

edit:

Sve

tlana

Laz

ebni

k

24-‐Sep-‐12 42

Lecture 1 - !!!

Fei-Fei Li!

Shape and ligh.ng cues: Shading

Sou

rce:

J. K

oend

erin

k

24-‐Sep-‐12 43

Lecture 1 - !!!

Fei-Fei Li!

Posi.on and ligh.ng cues: Cast shadows

Sou

rce:

J. K

oend

erin

k

24-‐Sep-‐12 44

Lecture 1 - !!!

Fei-Fei Li!

Grouping cues: Similarity (color, texture, proximity)

slid

e cr

edit:

Sve

tlana

Laz

ebni

k

24-‐Sep-‐12 45

Lecture 1 - !!!

Fei-Fei Li!

Grouping cues: “Common fate”

Imag

e cr

edit:

Arth

us-B

ertra

nd (v

ia F

. Dur

and)

24-‐Sep-‐12 46

Lecture 1 - !!!

Fei-Fei Li!

Bogom line •  Percep.on is an inherently ambiguous problem

–  Many different 3D scenes could have given rise to a par.cular 2D picture

24-‐Sep-‐12 47

Lecture 1 - !!!

Fei-Fei Li!

Bogom line •  Percep.on is an inherently ambiguous problem

–  Many different 3D scenes could have given rise to a par.cular 2D picture

•  Possible solu.ons –  Bring in more constraints (more images) –  Use prior knowledge about the structure of the world

•  Need a combina.on of different methods 24-‐Sep-‐12 48

Lecture 1 - !!!

Fei-Fei Li!

Computer Vision in the Real World

24-‐Sep-‐12 49

Lecture 1 - !!!

Fei-Fei Li!

Special effects: shape and mo.on capture

Sour

ce: S

. Sei

tz

24-‐Sep-‐12 50

Lecture 1 - !!!

Fei-Fei Li!

3D urban modeling

Bing maps, Google Streetview Source: S. Seitz

24-‐Sep-‐12 51

Lecture 1 - !!!

Fei-Fei Li!

3D urban modeling: Microsoj Photosynth

hgp://labs.live.com/photosynth/ Source: S. Seitz

24-‐Sep-‐12 52

Lecture 1 - !!!

Fei-Fei Li!

Face detec.on

•  Many new digital cameras now detect faces – Canon, Sony, Fuji, …

Source: S. Seitz

24-‐Sep-‐12 53

Lecture 1 - !!!

Fei-Fei Li!

Smile detec.on

Sony Cyber-shot® T70 Digital Still Camera Source: S. Seitz

24-‐Sep-‐12 54

Lecture 1 - !!!

Fei-Fei Li!

Face recogni.on: Apple iPhoto sojware

hgp://www.apple.com/ilife/iphoto/

24-‐Sep-‐12 55

Lecture 1 - !!!

Fei-Fei Li!

Biometrics

How the Afghan Girl was Iden.fied by Her Iris Pagerns

Source: S. Seitz

24-‐Sep-‐12 56

Lecture 1 - !!!

Fei-Fei Li!

Biometrics

Fingerprint scanners on many new laptops, other devices

Face recogni.on systems now beginning to appear more widely hgp://www.sensiblevision.com/ Source: S. Seitz

24-‐Sep-‐12 57

Lecture 1 - !!!

Fei-Fei Li!

Op.cal character recogni.on (OCR)

Digit recognition, AT&T labs

Technology to convert scanned docs to text •  If you have a scanner, it probably came with OCR sojware

License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Source: S. Seitz

24-‐Sep-‐12 58

Lecture 1 - !!!

Fei-Fei Li!

Toys and Robots

Lecture 1 - !!!

Fei-Fei Li!

Mobile visual search: Google Goggles

24-‐Sep-‐12 60

Lecture 1 - !!!

Fei-Fei Li!

Mobile visual search: iPhone Apps

24-‐Sep-‐12 61

Lecture 1 - !!!

Fei-Fei Li!

Automo.ve safety

•  Mobileye: Vision systems in high-‐end BMW, GM, Volvo models –  “In mid 2010 Mobileye will launch a world's first applica.on of full emergency braking for collision mi.ga.on for pedestrians where vision is the key technology for detec.ng pedestrians.”

Source: A. Shashua, S. Seitz

24-‐Sep-‐12 62

Lecture 1 - !!!

Fei-Fei Li!

Vision in supermarkets

LaneHawk by EvolutionRobotics “A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk, you are assured to get paid for it… “ Source: S. Seitz

24-‐Sep-‐12 63

Lecture 1 - !!!

Fei-Fei Li!

Vision-‐based interac.on (and games)

Microsoft’s Kinect

Source: S. Seitz Assistive technologies

Sony EyeToy

24-‐Sep-‐12 64

Lecture 1 - !!!

Fei-Fei Li!

Vision for robo.cs, space explora.on

Vision systems (JPL) used for several tasks •  Panorama s.tching •  3D terrain modeling •  Obstacle detec.on, posi.on tracking •  For more, read “Computer Vision on Mars” by Maghies et al.

NASA'S Mars Explora.on Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of 2007.

Sour

ce: S

. Sei

tz

24-‐Sep-‐12 65

Lecture 1 - !!!

Fei-Fei Li!

The computer vision industry

•  A list of companies here: hgp://www.cs.ubc.ca/spider/lowe/vision.html

24-‐Sep-‐12 66

Lecture 1 - !!!

Fei-Fei Li!

Today’s agenda

•  Introduc.on to computer vision •  Course overview

24-‐Sep-‐12 67

Lecture 1 - !!!

Fei-Fei Li!

Overall philosophy

•  Breadth –  Computer vision is a huge field –  It can impact every aspect of life and society –  It will drive the next informa.on and AI revolu.on –  Pixels are everywhere in our lives and cyber space –  Lectures are high-‐level, meant to be informa.ve, and covers many topics –  Lots of links to references. Know where to look for references –  Speak our “language”

•  Depth –  Computer vision is a highly technical field, i.e. know your math! –  Homework meant to be challenging, both theore.cal ques.ons and

programming exercises –  Master bread-‐and-‐buger techniques: face recogni.on, corners, lines, features,

op.cal flows, clustering and segmenta.on, basic object recogni.on techniques

–  Course projects are your hands-‐on experience in computer vision systems and research

24-‐Sep-‐12 68

Lecture 1 - !!!

Fei-Fei Li!

Contac.ng instructor and TAs •  ALL EMAIL CORRESPONDENCES TO ANYONE OF US:

–  cs231a-‐aut1213-‐[email protected]

•  Professor: Fei-‐Fei Li –  Office hour: Tues 3:30-‐4:30pm

•  Jon Krause, Ph.D, CS –  Office hour: Mon 4:30-‐5:30pm

•  Vignesh Ramanathan, Ph.D, EE –  Office hour: Wed 3:00-‐4:00pm

•  Jinchao Ye, master, CS –  Office hour: TBD

•  Zixuan Wang, master, CS –  Office hour: Fri 3:00-‐4:00pm

24-‐Sep-‐12 69

Lecture 1 - !!!

Fei-Fei Li!

Syllabus

•  Go to website…

24-‐Sep-‐12 70

Lecture 1 - !!!

Fei-Fei Li!

Course Project: overview

•  40% of your grade •  Form your team:

–  either 2 people or 1 person –  but the quality is judged regardless of the number of people on the team

–  be nice to your partner: do you plan to drop the course?

•  No late days •  Mandatory agendance on Dec 6 for all non-‐SCPD students

24-‐Sep-‐12 71

Lecture 1 - !!!

Fei-Fei Li!

Course Project: overview (con.nued) •  Start immediately •  Some important dates:

– Oct 16 •  Finalize team •  Project proposal due for “open project” teams

– Nov 6 •  Milestone due (2-‐3 pages)

– Dec 3 •  Final codes due

– Dec 4 •  Final writeup due

– Dec 6 •  Presenta.on

24-‐Sep-‐12 72

Lecture 1 - !!!

Fei-Fei Li!

Course Project Op.on #1: the Finding Mii Challenge

24-‐Sep-‐12 73

Lecture 1 - !!!

Fei-Fei Li!

•  Original research ideas encouraged •  Useful datasets:

–  ImageNet (www.image-‐net.org) – PASCAL

•  Need Fei-‐Fei’s approval – Email is the best way – Do it BEFORE Oct 16 (proposal submission deadline)

24-‐Sep-‐12 74

Course Project Op.on #2: Open Project

Lecture 1 - !!!

Fei-Fei Li!

Grading policy

•  Problem Sets: 40% – We have 5 problem sets –  Homework 0: very important! (more details…) –  Late policy

•  5 free late days – use them in your ways •  Ajerwards, 25% off per day late •  Not accepted ajer 3 late days per PS

–  Collabora.on policy •  Read the student code book, understand what is ‘collabora.on’ and what is ‘academic infrac.on’

•  Midterm Exam: 20% –  In class: Tues, Oct 30

24-‐Sep-‐12 75

Lecture 1 - !!!

Fei-Fei Li!

Grading policy

•  Course project: 40% –  presenta.on: 5% –  write-‐up: 10%

• clarity, structure, language, references: 3% • background literature survey, good understanding of the problem: 3% • good insights and discussions of methodology, analysis, results, etc.: 4%

–  technical: 15% • correctness: 5% • depth: 5% • innova.on: 5%

–  evalua.on and results: 10% • sound evalua.on metric: 3% • thoroughness in analysis and experimenta.on: 3%

•  A word about ‘the curve’

24-‐Sep-‐12 76

Lecture1: Introduconto“ComputerVision”vision.stanford.edu/teaching/.../lecture1...cs231a.pdf ·...

Documents

Transcript of Lecture1: Introduconto“ComputerVision”vision.stanford.edu/teaching/.../lecture1...cs231a.pdf ·...