Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

9
Lecture 7 Object Descriptors <23@A @C< Uppsala University 2 Chapter 11.1 – 11.4 in G-W Reading instructions 3 $C@ >@=5@3AA 7< B63 /</:GA7A 16/7< The Image Analysis Chain 4 So far Today and next lecture The Next Steps 5 (63 <3FB AB3> /4B3@ A35;3<B/B7=< 7A B= @3>@3A3<B =0831BA B= ;/93 7B >=AA70:3 B= 23A1@703 B63; FB3@</: 0=C<2/@G &3>@3A3<B/B7=< %=:G5=< =4 B63 0=C<2/@G 3A1@7>B7=< (63 17@1C;43@3<13 <B3@</: @357=</: &3>@3A3<B/B7=< %7F3:A 7<A723 B63 =0831B 3A1@7>B7=< (63 /D3@/53 1=:=@ &3>@3A3<B/B7=<A /<2 23A1@7>B=@A 6 (63 &3>@3A3<B/B7=< =4 B63 $0831B < 3<1=27<5 =4 B63 =0831B (@CB64C: 0CB >=AA70:G />>@=F7;/B3 3A1@7>B=@ =4 B63 $0831B $<:G /< /A>31B =4 B63 =0831B 'C7B/0:3 4=@ 1:/AA7471/B7=< <D/@7/<B B= 35 <=7A3 B@/<A:/B7=< N +3 1/< 1=;>CB3 23A1@7>B=@A 4@=; B63 @3>@3A3<B/B7=< 0CB <=B D713 D3@A/ &3>@3A3<B/B7=<A /<2 23A1@7>B=@A

Transcript of Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

Page 1: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

Lecture 7 Object Descriptors

�<23@A��@C<�Uppsala University

2

Chapter 11.1 – 11.4 in G-W

Reading instructions

3

•  $C@�>@=5@3AA�7<�B63�/</:GA7A�16/7<�

The Image Analysis Chain 4

So far

Today and next lecture

The Next Steps

5

(63�<3FB�AB3>�/4B3@�A35;3<B/B7=<�7A�B=�@3>@3A3<B�=0831BA�B=�;/93�7B��>=AA70:3�B=�23A1@703�B63;�

•  �FB3@</:��0=C<2/@G���•  &3>@3A3<B/B7=<��%=:G5=<�=4�B63�0=C<2/@G�•  �3A1@7>B7=<��(63�17@1C;43@3<13�

•  �<B3@</:��@357=</:��•  &3>@3A3<B/B7=<��%7F3:A�7<A723�B63�=0831B ��•  �3A1@7>B7=<��(63�/D3@/53�1=:=@�

&3>@3A3<B/B7=<A�/<2�23A1@7>B=@AO6

•  (63�&3>@3A3<B/B7=<�=4�B63�$0831B�•  �<�3<1=27<5�=4�B63�=0831B�•  (@CB64C:�0CB�>=AA70:G�/>>@=F7;/B3�

•  ���3A1@7>B=@�=4�B63�$0831B��•  $<:G�/<�/A>31B�=4�B63�=0831B�•  'C7B/0:3�4=@�1:/AA7471/B7=<�•  �<D/@7/<B�B=�3�5��<=7A3��B@/<A:/B7=<��N�

•  �+3�1/<�1=;>CB3�23A1@7>B=@A�4@=;�B63�@3>@3A3<B/B7=<��0CB�<=B�D713�D3@A/��

&3>@3A3<B/B7=<A�/<2�23A1@7>B=@AO

Page 2: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

7

•  $4B3<�E3�E/<B�23A1@7>B=@A�B6/B�/@3�7<D/@7/<B�=4�A1/:3��@=B/B7=<�/<2�B@/<A:/B7=<��

•  �=E3D3@��<=B�/:E/GA���<�$>B71/:��6/@/1B3@�&31=5<7B7=<��$�&��@=B/B7=<�/<2�A1/:3�7A�7;>=@B/<B��3�5��L%M�/<2�L2M��

'1/:3��@=B/B7=<�/<2�B@/<A:/B7=<O8

•  +/:9�/@=C<2�B63�=0831B�0=C<2/@G�/<2�23A1@703�27@31B7=</:�16/<53�7<�3/16�AB3>�0G�/�<C;03@�

�6/7<��=27<5

9

•  Code become very long and noise sensitive •  Use larger grid spacing, 0710 = 00

•  Scale dependent •  Choose appropriate grid spacing

•  Start point determines result •  Treat code as circular (minimum magnitude integer)

754310 075431

•  Depends on rotation •  Calculate difference code (counterclockwise)

075431 767767

�6/7<��=27<5 10

•  A digital boundary can be approximated (simplified)

•  For closed boundaries: •  Approximation becomes exact when no. of

segments of the polygons is equal to the no. of points in the boundary

•  Goal is to capture the essence of the object shape •  Approximation can become a time consuming

iterative process

%=:G5=</:��>>@=F7;/B7=<AO

11

•  "7<7;C;�%3@7;3B3@�%=:G5=<A��"%%A��•  �=D3@�B63�0=C<2/@G�E7B6�13::A�=4�/�16=A3<�A7H3�/<2�4=@13�/�@C003@�0/<2�:793�AB@C1BC@3�B=�47B�7<A723�B63�13::A�

%=:G5=</:��>>@=F7;/B7=<AO12

•  "3@57<5�B316<7?C3A���  +/:9�/@=C<2�B63�0=C<2/@G�/<2�47B�/�:3/AB�A?C/@3�3@@=@�:7<3�B=�

B63�>=7<BA�C<B7:�/<�3@@=@�B6@3A6=:2�7A�3F133232���  'B/@B�/�<3E�:7<3��5=�B=��� �  +63<�AB/@B�>=7<B�7A�@3/1632�B63�7<B3@A31B7=<A�=4�/28/13<B�:7<3A�

/@3�B63�D3@B713A�=4�B63�>=:G5=<�

%=:G5=</:��>>@=F7;/B7=<AO

Page 3: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

13

•  '>:7BB7<5�B316<7?C3A���  'B/@B�E7B6�/<�7<7B7/:�5C3AA��3�5���0/A32�=<�;/8=@7BG�/F3A���  �/:1C:/B3�B63�=@B6=5=</:�27AB/<13�4@=;�:7<3A�B=�/::�>=7<BA� �  �4�;/F7;C;�27AB/<13���B6@3A6=:2��1@3/B3�<3E�D3@B3F�B63@3���  &3>3/B�C<B7:�<=�>=7<BA�3F1332�1@7B3@7=<�

%=:G5=</:��>>@=F7;/B7=<AO14

Signatures

15

•  �����@3>@3A3<B/B7=<�=4�/�0=C<2/@G�•  �=C:2�03�7;>:3;3<B32�7<�27443@3<B�E/GA�

•  �7AB/<13�4@=;�13<B@3�>=7<B�B=�0=@23@�/A�/�4C<1B7=<�=4�/<5:3�•  �<5:3�03BE33<�B63�B/<53<B�7<�3/16�>=7<B�/<2�/�@343@3<13�:7<3��A:=>3�23<A7BG�4C<1B7=<��

•  �<23>3<23<B�=4�B@/<A:/B7=<��0CB�<=B�@=B/B7=<���A1/:7<5��%=AA70:3�A=:CB7=<A��•  '3:31B�C<7?C3�AB/@B7<5�>=7<B��3�5��0/A32�=<�;/8=@�/F7A��•  #=@;/:7H3�/;>:7BC23�=4�A75</BC@3��27D723�0G�D/@7/<13��

Signatures 16

•  +63<�/�0=C<2/@G�1=<B/7<A�;/8=@�1=<1/D7B73A�B6/B�1/@@G�A6/>3�7<4=@;/B7=<�7B�1/<�03�E=@B6E67:3�B=�231=;>=A3�7B�7<B=�A35;3<BA�

•  ��5==2�E/G�B=�/1673D3�B67A�7A�B=�1/:1C:/B3�B63�1=<D3F��C::��=4�B63�@357=<�3<1:=A32�0G�B63��0=C<2/@G�

•  �/<�03�/�07B�<=7A3�A3<A7B7D3�•  ';==B6�>@7=@�B=��=<D3F�6C::�1/:1C:/B7=<�•  �/:1C:/B3��=<D3F��C::�=<�>=:G5=<�/>>@=F7;/B7=<�

�=C<2/@G�A35;3<BAO

17

•  �/<�03�/�07B�<=7A3�A3<A7B7D3�•  ';==B6�>@7=@�B=��=<D3F��C::�1/:1C:/B7=<��•  �/:1C:/B3��=<D3F��C::�=<�>=:G5=</:�/>>@=F7;/B7=<

�=C<2/@G�A35;3<BAO18

•  '93:3B=<A�1=C:2�03�CA32�/A�1C@D3�@3>@3A3<B/B7=<A�=4�/<�=0831B�

•  '6=C:2�7<�53<3@/:�03�B67<��13<B3@32��B=>=:=571/::G�3?C7D/:3<B�B=�=@757</:�=0831B�/<2�@3D3@A70:3��

'93:3B=<A

Page 4: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

19

•  �F/;>:3��

'93:3B=<A 20

•  After representation, the next step is to describe our regions so that we later can classify them (next lecture)

•  A description is an aspect of the representation •  What descriptor is useful for classification of

•  adults / children •  pears / bananas / tomatoes

�3A1@7>B=@A

21

•  !3<5B6��>3@7;3B3@��•  �7/;3B3@����� �����������������������;/8=@�/F7A�•  "7<=@�/F7A��>3@>3<271C:/@�B=�;/8=@�/F7A��•  �/A71�@31B/<5:3���;/8=@�I�;7<=@�•  �113<B@717BG���;/8=@��;7<=@�•  �C@D/BC@3��@/B3�=4�16/<53�=4�A:=>3�

'7;>:3�0=C<2/@G�23A1@7>B=@A 22

•  &32347<3�B63�F����G�1==@27</B3A�=4�B63�0=C<2/@G�/A�B63�@3/:�/<2�7;/57</@G�>/@BA�@3A>31B7D3:G�=4�/�1=;>:3F�<C;03@�

•  4�9����F�9����7�G�9���9����N�� ����•  (63���(�53<3@/B3A�B63� ��=C@73@�23A1@7>B=@A�

•  �<D3@A3�B@/<A4=@;/B7=<�=4�B63� �23A1@7>B=@A�@353<3@/B3�B63�=@757</:�0=C<2/@G�

•  �<D3@A3�B@/<A4=@;/B7=<�=4�B63�%� �47@AB�23A1@7>B=@A�53<3@/B3A�/<�/>>@=F7;/B7=<�=4�B63�0=C<2/@G�

�=C@73@�23A1@7>B=@A

23

•  &3>@3A3<B�B63�0=C<2/@G�/A�/�A3?C3<13�=4�1==@27</B3A�

•  (@3/B�3/16�1==@27</B3�>/7@�/A�/�1=;>:3F�<C;03@��

�=C@73@�23A1@7>B=@A 24

•  �@=;�B63���(�=4�B63�1=;>:3F�<C;03@�E3�53B�B63��=C@73@�23A1@7>B=@A��B63�1=;>:3F�1=3447173<BA��/�C��O

•  (63����(�4@=;�B63A3�1=3447173<BA�@3AB=@3A�A�9�O

•  +3�1/<�1@3/B3�/<�/>>@=F7;/B3�@31=<AB@C1B7=<�=4�A�9��74�E3�CA3�=<:G�B63�47@AB�%��=C@73@�1=3447173<BA�

�=C@73@�23A1@7>B=@A

Page 5: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

25

•  �=C<2/@G�@31=<AB@C1B7=<�CA7<5���������������������/<2����=C@73@�23A1@7>B=@A�=CB�=4�/�>=AA70:3������

�=C@73@�23A1@7>B=@A 26

•  (67A�0=C<2/@G�1=<A7ABA�=4����>=7<B��%�7A�B63�<C;03@�=4�23A1@7>B=@A�CA32�7<�B63�@31=<AB@C1B7=<�

�=C@73@�23A1@7>B=@A

27

•  )A34C:�4=@�23A1@707<5�B63�A6/>3�=4�0=C<2/@G�A35;3<BA�•  'C7B/0:3�4=@�23A1@707<5�B63�A6/>3�=4�1=<D3F�2347173<173A�•  (63�67AB=5@/;�=4�B63�4C<1B7=<��A35;3<B�1C@D3��1/<�/:A=�03�CA32�4=@�1/:1C:/B7<5�;=;3<BA�•  �<2�;=;3<B�57D3A�A>@3/2�/@=C<2�;3/<��D/@7/<13��•  @2�;=;3<B�57D3A�AG;;3B@G�/@=C<2�;3/<��A93E<3AA��

'B/B7AB71/:�;=;3<BA 28

•  �4����7A�/�27A1@3B3�@/<2=;�D/@7/0:3�@3>@3A3<B7<5�27A1@3B3�/;>:7BC23�7<�B63�@/<53�,����-�B63<�B63�<B6�AB/B7AB71/:�;=;3<B�=4�D��/0=CB�7BA�;3/<��7A�1/:1C:/B32�/A�O

'B/B7AB71/:�;=;3<BA

29

•  �@3/���<C;03@�=4�>7F3:A�7<�/�@357=<�•  �=;>/1B<3AA��%������>3@7;3B3@.���/@3/�•  �7@1C:/@7BG�@/B7=����IKI/@3/��>3@7;3B3@.��•  �@/G:3D3:�;3/AC@3A�

•  "3/<�•  "327/<�•  "/F�•  �B1��

'7;>:3�&357=</:��3A1@7>B=@AO30

•  Topology = The study of the properties of a figure that are unaffected by any deformation

•  Topological descriptors •  Number of holes in a region, H •  Number of connected components, C •  Euler number, E = C – H

A B

(=>=:=571/:�23A1@7>B=@AO

Page 6: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

31

•  Textures can be very valuable when describing objects

•  Example below: Smooth, coarse and regular textures

Texture'32

•  Statistical texture descriptors: •  Histogram based •  Co-occurence based (Statstical moments, Uniformity, entropy,... )

•  Spectral texture descriptor •  Use fourier transform

Texture'

33

•  %@=>3@B73A�=4�B63�5@/G:3D3:�67AB=5@/;��=4�/<�7;/53�=@�@357=<��CA32�E63<�1/:1C:/B7<5�AB/B7AB71/:�;=;3<BA�•  H���27A1@3B3�@/<2=;�D/@7/0:3�@3>@3A3<B7<5�27A1@3B3�5@/G:3D3:A�7<�B63�@/<53�,���!��-�

•  %�H7����<=@;/:7H32�67AB=5@/;�1=;>=<3<B��7�3��B63�>@=0/07:7BG�=4�47<27<5�/�5@/G�D/:C3�1=@@3A>=<27<5�B=�B63�7�B6�5@/G�:3D3:�H7��

�<2�;=;3<B���*/@7/<13�=4�H� @2�;=;3<B���'93E<3AA��B6�;=;3<B���&3:/B7D3�4:/B<3AA�

Histogram based descriptors 34

)<74=@;7BG�/<2�/D3@/53�3<B@=>G�7A�1=;>CB32�A7;7:/@:G��

•  )<74=@;7BG��;/F7;C;�4=@�7;/53�E7B6�8CAB�=<3�5@/GD/:C3����

•  �D3@/53�3<B@=>G��;3/AC@3�=4�D/@7/07:7BG���347<32�/A��4=@�1=<AB/<B�7;/53A��

Histogram based descriptors

35

•  �=@�/<�7;/53�E7B6�#�5@/G:3D3:A��/<2�%��/�>=A7B7=</:�=>3@/B=@��53<3@/B3����/�#�I�#�;/B@7F��E63@3�/7�8�7A�B63�<C;03@�=4�B7;3A�/�>7F3:�E7B6�5@/G:3D3:�D/:C3�H7�7A�7<�@3:/B7D3�>=A7B7=<�%�B=�5@/G:3D3:�D/:C3�H8�

•  �7D723�/::�3:3;3<BA�7<���E7B6�B63�AC;�=4�/::�3:3;3<BA�7<����(67A�57D3A�/�<3E�;/B@7F���E63@3�17�8�7A�B63�>@=0/07:7BG�B6/B�/�>/7@�=4�>7F3:A�4C:47::7<5�%�6/A�5@/G:3D3:�D/:C3A�H7�/<2�H8�E6716�7A�1/::32�B63�1=�=11C@@3<13�;/B@7F�

Co-occurrence matrix 36

Co-occurrence matrix

Page 7: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

37

•  Maximum probability (strongest response to P)

•  Uniformity

•  Entropy (randomness)

Co-occurrence matrix Descriptors 38

•  �F/;>:3���•  "/F7;C;�>@=0/07:7BG���� �•  )<74=@;7BG�P������•  �<B@=>G�P�C<2347<32�

Co-occurrence matrix Descriptors

39

•  Example 2 •  Maximum probability = 1/3 •  Uniformity ≈ 0.167 •  Entropy ≈ 2.918

Co-occurrence matrix Descriptors 40

•  Match image with a co-occurrence matrix!

Co-occurrence matrix

41

•  Peaks in the Fourier spectrum give information about direction and spatial period patterns

•  The spectrum can be described using polar coordinates S(r,θ)

•  For each angle θ, S(r,θ) is a 1D function Sθ(r) •  Similarly, for each frequency r, Sr(θ) is a 1D

function •  A global description can be obtained by summing

Sθ(r) and Sr(θ)

Spectral Analysis 42

Spectral Analysis

Page 8: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

43

S(θ) S(r)

Spectral Analysis 44

•  �=@�/����1=<B7<C=CA�4C<1B7=<�4�F�G���B63�;=;3<B�=4�=@23@��>���?��7A�2347<32�/A�O

4=@�>��?��������N�

•  (63�13<B@/:�;=;3<BA�/@3�2347<32�/AO

E63@3��

�3<B@/:�"=;3<BA

45

•  �4�4�F�G��7A�/�2757B/:�7;/53��B63�13<B@/:�;=;3<BA�031=;3O

•  (63�<=@;/:7H32�13<B@/:�;=;3<BA��23<=B32�J>?��/@3�2347<32�/A�

E63@3����������������4=@�>�?����� �N��O

Central Moments 46

•  ��A3B�=4�A3D3<�7<D/@7/<B�;=;3<BA�1/<�03�23@7D32�4@=;�B63��<2�/<2� @2�;=;3<BA�

•  (63A3�;=;3<BA�/@3�7<D/@7/<B�B=�16/<53A�7<�B@/<A:/B7=<��@=B/B7=<�/<2�A1/:3�O

Moment Invariants

47 48

How to Design Descriptors?

•  Find descriptors that are invariant to things that are unimportant for your task: •  I.e. Noise, scale, blur, …

•  Find descriptors that are ekvivariant with the your task •  height, to classify adults / children •  color and shape to separate bananas, pears and

tomatoes

Page 9: Reading instructions€¦ · 7 • $4b3< e3 e/b=@a b6/b /@3 7

49

Conclusions

•  We went from object boundaries and image regions to a set of invariant numbers that we call descriptors (or features)

•  Next lecture we take a look at classifiers! •  Exercises 11.19 and 11.25 are

recommended