GRAPH_MATCHING_FOR_EFFICIENT_CLASSIFIERS_ADAPTATION.pdf
-
Upload
grssieee -
Category
Technology
-
view
214 -
download
0
Transcript of GRAPH_MATCHING_FOR_EFFICIENT_CLASSIFIERS_ADAPTATION.pdf
The problem of adaptationVQ adaptation
Summary
Graph Matching for Classifier Adaptation
Devis Tuia, Jordi Munoz-Marı and Jesus Malo
Image and Signal Processing GroupUniversity of Valencia, Spain
International Geoscience and Remote Sensing Symposium
Vancouver, 28nd of July 2011
1/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Acquisition conditions count
During the day, illumination conditions change >> changes in thespectral response of surfaces.
16h 17h
18h 19h
2/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Acquisition conditions count
These changes can be seen clearly in the RGB space
Sky
100 120 140140
160
180
200
R
G
100 120 140140
160
180
200
R
G
100 120 140140
160
180
200
R
G
100 120 140140
160
180
200
R
G
Veg
etat
ion
0 100 2000
100
200
R
G
0 100 2000
100
200
R
G
0 100 2000
100
200
R
G
0 100 2000
100
200
R
G
Bu
ildin
gs
0 100 2000
100
200
R
G
0 100 2000
100
200
R
G
0 100 2000
100
200
R
G
0 100 2000
100
200
R
G
16h 17h 18h 19h
3/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Lessons learned
I Shadowing corresponds to a reduction of the intensity of thespectrum, along with a subtle change of chroma.
400 500 600 700 800 9000
200
400
600
800
1000
1200
Wavelength (nm)
DN
Water
Vegetation
Buildings
Roads
Water with shadow
Vegetation with shadow
Buildings with shadow
Roads with shadow
Original image GT Observed spectra
I Illumination produces rotations of the spectral space.It can be catastrophic for a classifier.
I Other effects? ex: vegetation cycles, haze, ...
4/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
But oftenImage t1 Image t2
I We do not have labeled information on the scene consideredI We have information on scenes at other time instants or taken
by other sensorsI Direct classification can be catastrophic
Source Target OA Kappaimage image µ σ µ σt1 t1 93.46 0.22 0.901 0.003t1 t2 80.67 0.96 0.730 0.013
5/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
How to solve it?
By adaptation(or domain adaptation or transfer learning or color constancy , ....)
Deform the spectrum to match the distributions[useful in change detection or multitemporal]
Deform the classifier to the new data distribution[Domain adaptation, Bruzzone and Marconcini 2009, Gomez-Chova et al 2009]
Look for new training samples, to discover the differentconfigurations
We consider local deformation of the data structure through graphmatching
6/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
How to solve it?
By adaptation(or domain adaptation or transfer learning or color constancy , ....)
I Deform the spectrum to match the distributions[useful in change detection or multitemporal]
I Deform the classifier to the new data distribution[Domain adaptation, Bruzzone and Marconcini 2009, Gomez-Chova et al 2009]
I Look for new training samples, to discover the differentconfigurations
We consider local deformation of the data structure through graphmatching
7/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
How to solve it?
By adaptation(or domain adaptation or transfer learning or color constancy , ....)
I Deform the spectrum to match the distributions[useful in change detection or multitemporal] >> NOW!
I Deform the classifier to the new data distribution[Domain adaptation, Bruzzone and Marconcini 2009, Gomez-Chova et al 2009]
I Look for new training samples, to discover the differentconfigurations >> Next two talks!
We consider local deformation of the data structure through graphmatching
8/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
A multitemporal example
Image t1
G - R - IR space t1
G
R
NIR
01500
1400
2000
Image t2
G - R - IR space t2
G
R
NIR
01500
1400
2000
9/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Create image representations
Quantization using k-means
0 1500 0
20000
1400
RG
NIR
0 1500 0
20000
1400
RG
NIR
t1 t2
I Slight shifts of distribution
I Rotations and translation
I Local distorsions
10/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
An intuition about good and bad matching
t1 t2 Superposition
I Matching the distributions seems to be the way
11/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
An intuition about good and bad matching
Superposition
(centroids)
Euclidean match
(bad)
Graph match
(good)
I A straight euclidean match can be catastrophic
I Need a more sophisticated matching >> graph matching
12/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
A graph can be described by a series ofI nodes c, the vertices of the graph
c1
c3
c5
c4
c2
I edges (or weights) w , that can be defined by the nearestneighbors rule
c1
c2
c3
c4
c5
W0
1
1
1
1 1
1
1
1
13/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
To match the graphs t1/t2, we displace the nodes at t1 towardsthe nodes at t2. The modified nodes c∗ are defined as
c∗︸︷︷︸New nodes
= minc∗
{ ∑c∈ct1‖ct1 − c∗‖2︸ ︷︷ ︸
(1)
+ ‖W t1 −W ∗‖1︸ ︷︷ ︸(2)
}
(1) first term minimize nodes displacement>> stay close to the original nodes
(2) second term avoids graph structure changes>> keep the structure among nodes
14/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Avoid structure changes
1
2
3
4
5
1
3
5
4
2
3
55
1
3
5
4
2
3
1
3
5
4
2
Original t1 Modification 1 Modification 2
W0 W1 W2
1
1
1
1 1
1
1
1
1 1
1 1
1 1
11
11
1
1 1
1
1
1
|| W0 - W1 ||1 = 0 || W0 - W2 ||1 = 4
15/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
I Iterative procedure (node by node in t1)
I Considers a node and its nearest neighbors in t1
I Each node can move towards its nearest neighbors on the t2 graph (blue)
I When stable, move the t1 training points cloud wrt c∗
cc11
c3
c55
cc4
cc22
16/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
I Iterative procedure (node by node in t1)
I Considers a node and its nearest neighbors in t1
I Each node can move towards its nearest neighbors on the t2 graph (blue)
I When stable, move the t1 training points cloud wrt c∗
c1
c3
c5
c4
cc22
17/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
I Iterative procedure (node by node in t1)
I Considers a node and its nearest neighbors in t1
I Each node can move towards its nearest neighbors on the t2 graph (blue)
I When stable, move the t1 training points cloud wrt c∗
c1
c3
c5
c4
c2
c
c
c
c
c
18/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
I Iterative procedure (node by node in t1)
I Considers a node and its nearest neighbors in t1
I Each node can move towards its nearest neighbors on the t2 graph (blue)
I When stable, move the t1 training points cloud wrt c∗
c1
c3
c5 c4
c2
11c
3333c
5c 4
c
c
19/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
I Iterative procedure (node by node in t1)
I Considers a node and its nearest neighbors in t1
I Each node can move towards its nearest neighbors on the t2 graph (blue)
I When stable, move the t1 training points cloud wrt c∗
cc11
c33
c55 c4
cc22
20/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching
I Iterative procedure (node by node in t1)
I Considers a node and its nearest neighbors in t1
I Each node can move towards its nearest neighbors on the t2 graph (blue)
I When stable, move the t1 training points cloud wrt c∗
Original After matching
21/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Graph matching result
0 0.1 0.2 0.3 0.4 0.5 0.6 0
0.5
0
0.1
0.2
0.3
0.4
0.5
RG
IR
0 1500 0
20000
1400
RG
NIR
Movements allowed Final local transform
22/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Observing the manifoldGraph matchingExperiments
Results
Model = SVMκ
# training from t1 143 286 1430 2860
Classification of t1 t1 → t1 0.857 0.878 0.901 0.909Classification of t2 t2 → t2 0.794 0.826 0.854 0.860Transfer, no modeladaptation
t1 → t2 0.730 0.747 0.730 0.746
Transfer, withmodel adaptation(k = 50)
t1 → t∗1 → t2 0.761 0.780 0.783 0.788
Transfer, withmodel adaptation(k = 100)
t1 → t∗1 → t2 0.753 0.772 0.792 0.808
I Improvement in all experiments (+0.03 to +0.06 in κ)I Training samples are just translated, they should be rotated as
well (ongoing)I Does not reach performance of t2 since no labeled information
in t2 is added
23/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Summary
The model proposed
I Enhances adaptation without additional information
I Is based on simple vector quantization based on clustering
I Proved to perform local adaptation maintaining graphstructures
I Next stepsI More elegant cost functionsI Allow smoother movements of the graphI Rotate training pixels with t2 distributionI Going semi-supervised
24/24 Devis Tuia
The problem of adaptationVQ adaptation
Summary
Thank you!
http://isp.uv.es/
http://devis.tuia.googlepages.com
25/24 Devis Tuia