Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo...
Transcript of Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo...
![Page 1: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/1.jpg)
Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Hannover, 2010-02-23
Image Understanding withOrganic Computing
Rolf P. Wurtz
Ruhr-Universitat BochumInstitut fur Neuroinformatik
http://www.neuroinformatik.rub.derolf.wuertz@[email protected]
![Page 2: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/2.jpg)
OverviewNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
• Introduction
• Problem of image understanding
• Controlled generalization in face recognition
• General object recognition
• Learning of articulated models
• Where to go from here
![Page 3: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/3.jpg)
Complexity problemsNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
• Artificial systems are rapidly getting too complex to understand
• Desirable are complex systems with trivial interfaces
• This requires restrictions on possible behaviors
• This asks for self-organization
![Page 4: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/4.jpg)
Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
. . . means establishing a symbolic description.
![Page 5: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/5.jpg)
Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
. . . in the brain is done under additional assumptions.
![Page 6: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/6.jpg)
Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
. . . requires extensive world knowledge.
![Page 7: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/7.jpg)
Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
. . . has a local-global problem.
![Page 8: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/8.jpg)
Face recognitionNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
= 6=
Different situations yield very different images.
![Page 9: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/9.jpg)
Invariance problemNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
All the same?
![Page 10: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/10.jpg)
Invariance . . .Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
6 6= 6
. . . is task-dependent.
![Page 11: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/11.jpg)
List of tasksNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
• We do need a formal model of images, but we don’t have any
• We do not even have a formalization of the problem
• Required is an imitation of human capability
• Identify constraints of visual data autonomously
• Learn computer vision routines from examples
• Start a positive feedback loop of learning vision
• Control generalization
![Page 12: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/12.jpg)
Controlled generalizationNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
• Neural networks learn complicated functions from examples
• They can generalize, but not always in the desired way
• Visual invariances must be built in explicitly(Neocognitron, Convolutional NN, . . . )
• Exception: Slow feature analysis (Wiskott & Sejnowski, 2002)
• Goal: Learn generalization dimensions from examples!
![Page 13: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/13.jpg)
Bunch GraphsNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
image graph
general facebunch graph Gabor wavelet jet
bunch of jets
a: b: c: Wiskott et al., 1997
![Page 14: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/14.jpg)
Graph similarityNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
• Jet similarity function SJ
• Probe graph P with N nodes Pn
• Gallery graphs Gg with N nodes Gg,n each
grec = arg maxg
1
N
N∑n=1
SJ(Pn, Gg,n) .
![Page 15: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/15.jpg)
Face GraphsNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
![Page 16: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/16.jpg)
Pose VariationNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
PM+45 PM+00 PM−45
![Page 17: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/17.jpg)
Pose VariationNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
Probe Gallery
Similarity?
Gg P
......
![Page 18: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/18.jpg)
Rank CorrelationNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
ModelProbe Gallery
SP SG
π = [7, 3, 9, . . .] γ = [7, 9, 3, . . .]
γ = [2, 1, 35, . . .]
......
......
![Page 19: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/19.jpg)
Neural rank list similarityNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
aj wjE
I
E =
K∑j=1
exp
(−order(aj)
λ
)wj
wj =1
Kexp
(−order(bj)
λ
)
Thorpe et al., 2001
![Page 20: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/20.jpg)
Neural rank list similarityNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
π(m)
wm,g
Ag
wm,g =1
NMexp
(−γg(m)
λ
)Ag =
∑m
exp
(−π(m)
λ
)wm,g
=1
NM
∑m
exp
(−π(m) + γg(m)
λ
)= Sneural(γg, π)
grec = arg maxg
1
N
∑n
Sneural(γg,n, πn)
Muller and Wurtz, ICANN 2009
![Page 21: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/21.jpg)
Face recognition Mainz Hbf.Neuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
![Page 22: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/22.jpg)
CAS-PEAL DatabaseNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
PM+45 FM+00 FM−45 FM−90
PM+00 FD+00 FD−45 FD−90
PM−45 FU+00 FU−45 FU−90
![Page 23: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/23.jpg)
Rank Correlation: ResultsNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
Pose Illumination
Recognition percentage with given situation 99.02 89.01
Percentage of correct situation estimation 99.89 ± 0.09 91.96 ± 0.89
Recognition percentage with automaticallydetermined situation
97.75 ± 0.50 89.97 ± 1.36
Best recognition percentage reported indatabase description
71 51
![Page 24: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/24.jpg)
Rank Correlation: Early stoppingNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100
Rec
ogni
tion
rate
Spike number
poseillumination
![Page 25: Image Understanding with Organic Computingprojects.aifb.kit.edu/effalg/oc/inhalte/Files...Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention. Created](https://reader036.fdocuments.net/reader036/viewer/2022062507/5fc9ddd16d42c24e6c757503/html5/thumbnails/25.jpg)
Thanks toNeuroinformatik / Ruhr-Universitat Bochum
Rolf P. Wurtz
Image Understanding with Organic Computing Hannover, 2010-02-23
Marco Muller Rank correlation memoryGunter Westphal Object recognitionThomas Walther Body tracking
Manuel Gunter Statistical face recognitionMarkus Lessmann Scene analysisOliver Lomp Neuronal dynamicsMathis Richter Clustering of image patchesGuillermo Donatti Object memory, Neural Map
DFG, EU, NRW Funding
All of you Attention