Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision...

55
Lecture 4- Fei-Fei Li Lecture 4: Pixels and Filters Professor FeiFei Li Stanford Vision Lab 25Sep13 1

Transcript of Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision...

Page 1: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Lecture  4:    Pixels  and  Filters  

Professor  Fei-­‐Fei  Li  Stanford  Vision  Lab  

25-­‐Sep-­‐13  1  

Page 2: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

What  we  will  learn  today?  

•  Images  as  funcFons  •  Linear  systems  (filters)  •  ConvoluFon  and  correlaFon  

25-­‐Sep-­‐13  2  

Some  background  reading:  Forsyth  and  Ponce,  Computer  Vision,  Chapter  7  

Page 3: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

•  An  image  contains  discrete  number  of  pixels  – A  simple  example  – Pixel  value:  

•  “grayscale”    (or  “intensity”):  [0,255]  

2010.12.18  3  

Images  as  func,ons  

231  

75  

148  

Page 4: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

•  An  image  contains  discrete  number  of  pixels  – A  simple  example  – Pixel  value:  

•  “grayscale”    (or  “intensity”):  [0,255]  •  “color”  

–  RGB:  [R,  G,  B]  –  Lab:  [L,  a,  b]  –  HSV:  [H,  S,  V]  

2010.12.18  4  

Images  as  func,ons  

[249,  215,  203]  

[90,  0,  53]  

[213,  60,  67]  

Page 5: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  5  

Images  as  func,ons  •  An  Image  as  a  funcFon  f  from  R2  to  RM:

•  f( x, y ) gives  the  intensity  at  posiFon  ( x, y ) •  Defined  over  a  rectangle,  with  a  finite  range:  

f: [a,b] x [c,d ] à [0,255] Domain  support  

range  

Page 6: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  6  

Images  as  func,ons  •  An  Image  as  a  funcFon  f  from  R2  to  RM:

•  f( x, y ) gives  the  intensity  at  posiFon  ( x, y ) •  Defined  over  a  rectangle,  with  a  finite  range:  

f: [a,b] x [c,d ] à [0,255]

( , )( , ) ( , )

( , )

r x yf x y g x y

b x y

⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥⎣ ⎦

•  A  color  image:    

Domain  support  

range  

Page 7: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  7  

•  Images  are  usually  digital  (discrete):  –  Sample  the  2D  space  on  a  regular  grid  

•  Represented  as  a  matrix  of  integer  values  pixel  

Images  as  discrete  func,ons  

Page 8: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  8  

Cartesian  coordinates  

Images  as  discrete  func,ons  

NotaFon  for    discrete    funcFons  

Page 9: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

What  we  will  learn  today?  

•  Images  as  funcFons  •  Linear  systems  (filters)  •  ConvoluFon  and  correlaFon  

25-­‐Sep-­‐13  9  

Some  background  reading:  Forsyth  and  Ponce,  Computer  Vision,  Chapter  7  

Page 10: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  10  

Systems  and  Filters  •  Filtering:  

– Form  a  new  image  whose  pixels  are  a  combinaFon  original  pixel  values  

Goals:    -­‐ Extract  useful  informaFon  from  the  images  

•   Features  (edges,  corners,  blobs…)  

-­‐  Modify  or  enhance  image  properFes:    •     super-­‐resoluFon;  in-­‐painFng;  de-­‐noising  

Page 11: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  11  

Super-­‐resoluFon  De-­‐noising  

In-­‐painFng  

Bertamio  et  al    

Page 12: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

2D  discrete-­‐space  systems  (filters)  

25-­‐Sep-­‐13  12  

Page 13: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Filter  example  #1:  Moving  Average  

•  2D  DS  moving  average  over  a  3  ×  3  window  of  neighborhood  

25-­‐Sep-­‐13  13  

1  1  1  

1  1  1  

1  1  1  

],[],[91],)[(

,

lnkmhlkfnmhflk

−−=∗ ∑

h  

Page 14: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  14  

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Courtesy  of  S.  Seitz  ]ln,km[g]l,k[f]n,m)[gf(l,k

−−=∗ ∑

Filter  example  #1:  Moving  Average  

Page 15: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  15  

15  

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

]ln,km[g]l,k[f]n,m)[gf(l,k

−−=∗ ∑

Filter  example  #1:  Moving  Average  

Page 16: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  16  

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

]ln,km[g]l,k[f]n,m)[gf(l,k

−−=∗ ∑

Filter  example  #1:  Moving  Average  

Page 17: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  17  

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

]ln,km[g]l,k[f]n,m)[gf(l,k

−−=∗ ∑

Filter  example  #1:  Moving  Average  

Page 18: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  18  

0 10 20 30 30

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

]ln,km[g]l,k[f]n,m)[gf(l,k

−−=∗ ∑

Filter  example  #1:  Moving  Average  

Page 19: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  19  

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30 30 30 20 10

0 20 40 60 60 60 40 20

0 30 60 90 90 90 60 30

0 30 50 80 80 90 60 30

0 30 50 80 80 90 60 30

0 20 30 50 50 60 40 20

10 20 30 30 30 30 20 10

10 10 10 0 0 0 0 0

]ln,km[g]l,k[f]n,m)[gf(l,k

−−=∗ ∑Source: S. Seitz

Filter  example  #1:  Moving  Average  

Page 20: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  20  

In  summary:  •  Replaces  each  pixel  with  an  average  of  its  neighborhood.  

•  Achieve  smoothing  effect  (remove  sharp  features)  

1  1  1  

1  1  1  

1  1  1  

],[g ⋅⋅

Filter  example  #1:  Moving  Average  

Page 21: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  21  

Filter  example  #1:  Moving  Average  

Page 22: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  22  

•  Image  segmentaFon  based  on  a  simple  threshold:  

Filter  example  #2:  Image  SegmentaFon  

Page 23: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

ClassificaFon  of  systems  

25-­‐Sep-­‐13  23  

•  Amplitude  properFes  •  Linearity  •  Stability  •  InverFbility  

 

•  SpaFal  properFes  •  Causality  •  Separability  •  Memory  •  Shih  invariance  •  RotaFon  invariance  

Page 24: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Shih-­‐invariance  

25-­‐Sep-­‐13  24  

If                                                                                          then  

for  every  input  image  f[n,m]  and  shihs  n0,m0      

Page 25: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Is  the  moving  average  system  is  shih  invariant?      

25-­‐Sep-­‐13  25  

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30 30 30 20 10

0 20 40 60 60 60 40 20

0 30 60 90 90 90 60 30

0 30 50 80 80 90 60 30

0 30 50 80 80 90 60 30

0 20 30 50 50 60 40 20

10 20 30 30 30 30 20 10

10 10 10 0 0 0 0 0

Page 26: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Is  the  moving  average  system  is  shih  invariant?    

25-­‐Sep-­‐13  26  

Yes!  

Page 27: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Linear  Systems  (filters)  

25-­‐Sep-­‐13  27  

•  Linear  filtering:  –  Form  a  new  image  whose  pixels  are  a  weighted  sum  of  original  pixel  values  

–   Use  the  same  set  of  weights  at  each  point    

•  S  is  a  linear  system  (funcFon)  iff  it  S  sa&sfies  

superposiFon  property  

Page 28: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

•  Is  the  moving  average  a  linear  system?  

•  Is  thresholding  a  linear  system?  –  f1[n,m]  +  f2[n,m]  >  T  –  f1[n,m]  <  T  –  f2[n,m]<T  

25-­‐Sep-­‐13  28  

Linear  Systems  (filters)  

No!  

Page 29: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

LSI  (linear  shi$  invariant)  systems  

25-­‐Sep-­‐13  29  

Impulse  response                      

Page 30: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

LSI  (linear  shi$  invariant)  systems  

25-­‐Sep-­‐13  30  

1  1  1  

1  1  1  

1  1  1  

h  

Example:  impulse  response  of  the  3  by  3  moving  average  filter:  

Page 31: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

LSI  (linear  shi$  invariant)  systems  

25-­‐Sep-­‐13  31  

An  LSI  system  is  completely  specified  by  its  impulse  response.    

   

superposiFon  

Discrete  convoluFon  

sihing  property  of  the  delta  funcFon  

Page 32: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

What  we  will  learn  today?  

•  Images  as  funcFons  •  Linear  systems  (filters)  •  ConvoluFon  and  correlaFon  

25-­‐Sep-­‐13  32  

Some  background  reading:  Forsyth  and  Ponce,  Computer  Vision,  Chapter  7  

Page 33: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Discrete  convolu,on  

25-­‐Sep-­‐13  33  

•  Fold  h[n,m]  about  origin  to  form  h[−k,−l]  •  Shih  the  folded  results  by  n,m  to  form  h[n  −  k,m  −  l]  •  MulFply  h[n  −  k,m  −  l]  by  f[k,  l]  •  Sum  over  all  k,l  •  Repeat  for  every  n,m    

h[k,l]  

h[-­‐k,-­‐l]   h[n-­‐k,m-­‐l]  

n  

Page 34: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Discrete  convolu,on  

25-­‐Sep-­‐13  34  

•  Fold  h[n,m]  about  origin  to  form  h[−k,−l]  •  Shih  the  folded  results  by  n,m  to  form  h[n  −  k,m  −  l]  •  MulFply  h[n  −  k,m  −  l]  by  f[k,  l]  •  Sum  over  all  k,l  •  Repeat  for  every  n,m    

h[k,l]  

h[-­‐k,-­‐l]   h[n-­‐k,m-­‐l]  

n  

Page 35: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Discrete  convolu,on  

25-­‐Sep-­‐13  35  

•  Fold  h[n,m]  about  origin  to  form  h[−k,−l]  •  Shih  the  folded  results  by  n,m  to  form  h[n  −  k,m  −  l]  •  MulFply  h[n  −  k,m  −  l]  by  f[k,  l]  •  Sum  over  all  k,l  •  Repeat  for  every  n,m    

h[n-­‐k,m-­‐l]  

f[k,l]  

f[k,l]  x  h[n-­‐k,m-­‐l]    

Sum  (f[k,l]  x  h[n-­‐k,m-­‐l])    

n  

Page 36: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  36  

• 0  • 0  • 0  • 0  • 1  • 0  • 0  • 0  • 0  

Original

? =

Courtesy  of  D

 Low

e  

Page 37: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  37  

• 0  • 0  • 0  • 0  • 1  • 0  • 0  • 0  • 0  

Original Filtered (no change)

=

Courtesy  of  D

 Low

e  

Page 38: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  38  

• 0  • 0  • 0  • 1  • 0  • 0  • 0  • 0  • 0  

Original

? =

Courtesy  of  D

 Low

e  

Page 39: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  39  

• 0  • 0  • 0  • 1  • 0  • 0  • 0  • 0  • 0  

Shifted right By 1 pixel

=

Courtesy  of  D

 Low

e  Original

Page 40: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  40  

? • 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  

=

Courtesy  of  D

 Low

e  Original

Page 41: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  41  

• 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  

Blur (with a box filter)

=

Courtesy  of  D

 Low

e  Original

Page 42: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  -­‐  examples  

25-­‐Sep-­‐13  42  

• 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  

• 0  • 0  • 0  • 0  • 2  • 0  • 0  • 0  • 0   - = ?

(Note that filter sums to 1)

• 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  

• 0  • 0  • 0  • 0  • 1  • 0  • 0  • 0  • 0   -

• 0  • 0  • 0  • 0  • 1  • 0  • 0  • 0  • 0  

+

“details of the image”

Courtesy  of  D

 Low

e  

Original

Page 43: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  43  

•  What  does  blurring  take  away?  

original smoothed (5x5)

detail

=

sharpened

=

•  Let’s  add  it  back:  

original detail

+ a

Page 44: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  in  2D  –    Sharpening filter

 

25-­‐Sep-­‐13  44  

• 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  • 1  

• 0  • 0  • 0  • 0  • 2  • 0  • 0  • 0  • 0   -

Sharpening  filter:  Accentuates  differences  with  local  average  

=

Courtesy  of  D

 Low

e  

Original

Page 45: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  proper,es  

25-­‐Sep-­‐13  45  

•  Commuta,ve  property:      •  Associa,ve  property:      •  Distribu,ve  property:      

The  order  doesn’t  marer!  

Page 46: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Convolu,on  proper,es  

25-­‐Sep-­‐13  46  

•  ShiI  property:      •  ShiI-­‐invariance:            

Page 47: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Image  support  and  edge  effect  

25-­‐Sep-­‐13  47  

• A  computer  will  only  convolve  finite  support  signals.    

•     That  is:  images  that  are  zero  for  n,m  outside  some  rectangular  region  

•   MATLAB’s  conv2  performs  2D  DS  convoluFon  of  finite-­‐support  signals.  

 

 N1  ×M1  

N2  ×M2    (N1  +  N2  −  1)  ×  (M1  +M2  −  1)  *   =  

Page 48: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Image  support  and  edge  effect  

25-­‐Sep-­‐13  48  

• A  computer  will  only  convolve  finite  support  signals.    •   What  happens  at  the  edge?    

 f  h  

•  zero  “padding”  •  edge  value  replicaFon  •  mirror  extension  •  more  (beyond  the  scope  of  this  class)    -­‐>  Matlab  conv2  uses  zero-­‐padding    

Page 49: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

What  we  will  learn  today?  

•  Images  as  funcFons  •  Linear  systems  (filters)  •  ConvoluFon  and  correlaFon  

25-­‐Sep-­‐13  49  

Some  background  reading:  Forsyth  and  Ponce,  Computer  Vision,  Chapter  7  

Page 50: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Cross  correla,on  

25-­‐Sep-­‐13  50  

Cross  correlaFon  of  two  2D  signals  f[n,m]  and  g[n,m]                  •   Equivalent  to  a  convoluFon  without  the  flip      

(k,  l)  is  called  the  lag  

Page 51: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Cross  correla,on  –  example  

25-­‐Sep-­‐13  51  

   

MATLAB’s  xcorr2  

Courtesy  of  J.  Fessler  

Page 52: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li! 25-­‐Sep-­‐13  52  

Cross  correla,on  –  example  Leh   Right  

scanline  

Norm.  corr  

Page 53: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

ConvoluFon  vs.  CorrelaFon  

•  A  convolu,on  is  an  integral  that  expresses  the  amount  of  overlap  of  one  funcFon  as  it  is  shihed  over  another  funcFon.    –  convoluFon  is  a  filtering  operaFon  

•  Correla,on  compares  the  similarity  of  two  sets  of  data.  CorrelaFon  computes  a  measure  of  similarity  of  two  input  signals  as  they  are  shihed  by  one  another.  The  correlaFon  result  reaches  a  maximum  at  the  Fme  when  the  two  signals  match  best  .  –  correlaFon  is  a  measure  of  relatedness  of  two  signals  

2010.12.18  53  

Page 54: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications, 1998 copyright 1998, IEEE

Application: Vision system for TV remote control

- uses template matching

25-­‐Sep-­‐13  54  

Page 55: Lecture’4:’’ Pixels’and’Filters’ - Stanford Computer Vision Labvision.stanford.edu/teaching/cs131_fall1314_nope/lectures... · 2013-09-27 · Lecture 4- !!! Fei-Fei Li!

Lecture 4- !!!

Fei-Fei Li!

What  we  have  learned  today?  

•  Images  as  funcFons  •  Linear  systems  (filters)  •  ConvoluFon  and  correlaFon  

25-­‐Sep-­‐13  55