Online Robust Dictionary Learning
description
Transcript of Online Robust Dictionary Learning
![Page 1: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/1.jpg)
Online Robust Dictionary Learning
Cewu Lu, Jianping Shi, and Jiaya JiaThe Chinese University of Hong Kong
![Page 2: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/2.jpg)
Dictionary Learning
Denoise [Mairal et al. 2008] Upsampling[Couzinie-Devy 2010]
Image Classification [Wang et al. 2010]
Background Subtraction [Cong et al. 2010]
![Page 3: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/3.jpg)
Dictionary Learning
Let be a set of “basis vectors”.1{ ,...., }qD d d
Let be a set of signal.1{ ,...., }nX x x
1{ ,...., }nX x x1{ ,...., }qD d d
![Page 4: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/4.jpg)
Dictionary Learning
is “adapted” to if it can represent with a few basis vector.D X
1 1x D
2 2x D
n nx D
......3 3x D
Spare
X
![Page 5: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/5.jpg)
Dictionary Learning
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
![Page 6: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/6.jpg)
Dictionary Learning
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
![Page 7: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/7.jpg)
Robust Dictionary Learning
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
2min ii
x
X={2,5,6,9,10,12,14,15,18}
A toy example:
110.1
9 iix
L2 norm data fitting is not a robust measure.
![Page 8: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/8.jpg)
Robust Dictionary Learning
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
2min ii
x
X={2,5,6,9,10,12,14,15,80000}
A toy example:
18897
9 iix
Outliers
L2 norm data fitting is not a robust measure.
Inliers
![Page 9: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/9.jpg)
Robust Dictionary Learning
1
1
1 1,..., ,1
1min
n
n
i i iD
i
x Dn
L1 norm is a robust measure.
[ Wagner et al 2009],[ Wang et al 2012],[ Zhao et al 2011 ]
min ii
x
A toy example:
21min i
i i
xx
X={2,5,6,9,10,12,14,15,80000}
![Page 10: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/10.jpg)
Robust Dictionary Learning
1
1
1 1,..., ,1
1min
n
n
i i iD
i
x Dn
min ii
x
A toy example: 2min i i
i
x
1
iix
X={2,5,6,9,10,12,14,15,80000}
10i iix
199 1.26 10
[ Wagner et al 2009],[ Wang et al 2012],[ Zhao et al 2011 ]
L1 norm is a robust measure.
![Page 11: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/11.jpg)
Robust Dictionary Learning
1
1
1 1,..., ,1
1min
n
n
i i iD
i
x Dn
Inliers
Outliers
Incorrect Dictionary
Non-Robust Dictionary Learning
![Page 12: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/12.jpg)
Robust Dictionary Learning
1
1
1 1,..., ,1
1min
n
n
i i iD
i
x Dn
Inliers
Outliers
Correct Dictionary
Robust Dictionary Learning
![Page 13: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/13.jpg)
Robust Dictionary Learning
1
1
1 1,..., ,1
1min
n
n
i i iD
i
x Dn
[ Wagner et al 2009],[ Wang et al 2012],[ Zhao et al 2011 ]
Inliers
Outliers
Dictionary
Non-Robust Dictionary Learning
But, it is not widely used….
Why?
![Page 14: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/14.jpg)
Online Dictionary Learning
Large-scale data Dynamic data
Because…. L1 norm data-fitting hasn’t closed-form
heavy computation
We need Online.
![Page 15: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/15.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
Current
Online Dictionary Learning
![Page 16: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/16.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 17: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/17.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 18: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/18.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 19: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/19.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 20: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/20.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 21: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/21.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 22: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/22.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
History Current
Online Dictionary Learning
![Page 23: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/23.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Data
Dictionary Update
CurrentHistory
Dictionary
Online Dictionary Learning
![Page 24: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/24.jpg)
1
2
2 1,..., ,1
1min
n
n
i i iD
i
x Dn
Online Solver [Mairal et al 2010]:
Batch Method Online Method
Time Complexity
polynomial O(n)
Memory Complexity
O(n) O(1)
Online Dictionary Learning
![Page 25: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/25.jpg)
Online Robust Dictionary Learning
Make robust dictionary learning online.
Our goal:
Less ComputationLess Memory
![Page 26: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/26.jpg)
Online Robust Dictionary Learning
Inliers
Outliers
Dictionary
Dictionary Update
History Current
online
Robust
Forget history Data
Require Whole Data
![Page 27: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/27.jpg)
Current
Online Robust Dictionary Learning
Inliers
Outliers
Dictionary
Dictionary Update
History
online
Robust
Forget history Data
Require Whole Data
Challenging
![Page 28: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/28.jpg)
Online Robust Dictionary Learning
• Our Online Approach (Online)• Robustness Analysis (Robust)• Discussion
![Page 29: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/29.jpg)
Online Robust Dictionary Learning
• Our Online Approach• Robustness Analysis• Discussion
![Page 30: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/30.jpg)
Settings:
We have two parameter matrixes and .
Online Dictionary Learning
jtM
jtC
…
Each min-batch data contains h data point.
Data (Min-batch)
…
![Page 31: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/31.jpg)
Initialization: and are zero matrixes.
Data
Update
Current
Online Dictionary Learning
0jM0
jC
D
Dictionary D is a random matrix.
0jC 0
jM
… …
1jC 1
jM
![Page 32: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/32.jpg)
Data
Update
History Current
Online Dictionary Learning
…D
…
1jC 1
jM
General Framework
2jC 2
jM
![Page 33: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/33.jpg)
Data
Update
History Current
Online Dictionary Learning
D
……
2jC 2
jM
General Framework
3jC 3
jM
![Page 34: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/34.jpg)
Data
Update
History Current
Online Dictionary Learning
…D
…
3jC 3
jM
General Framework
![Page 35: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/35.jpg)
…Data
Update
History Current
Online Dictionary Learning
D
…
jtC
jtM
General Framework
1jtC 1
jtM
![Page 36: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/36.jpg)
…Data
Update
History Current
Online Dictionary Learning
D
…
1jtC 1
jtM
General Framework
![Page 37: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/37.jpg)
……Data
Update
CurrentHistory
Dictionary
Online Dictionary Learning
D
jnC
jnM
General Framework
![Page 38: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/38.jpg)
Our Online Approach
Time Complexity O(n/h) = O(n)
Memory Complexity O(1)
![Page 39: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/39.jpg)
…
Our Online Approach
Datat
History Current
Min-batch
1,...,t h tx x
1,...,t h t Sparse code:
Data:
1jtM
1jtC
DPrevious Dictionary
…
In step:tht
![Page 40: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/40.jpg)
Our Online Approach
Datat
History Current
Min-batch
1,...,t h tx x
1,...,t h t Sparse code:
Data:
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1 1
tj j j Tt t i i ii t h
M M
1 1
tj j j Tt t i ij ii t hC C x
Solve
1:j qfor
1jtM
1jtC
jtMjtC
1,:
2,:
...
,:
D
DD
D q
DCurrentDictionary
![Page 41: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/41.jpg)
Our Online Approach
Datat
History Current
Min-batch
1,...,t h tx x
1,...,t h t Sparse code:
Data:
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1 1
tj j j Tt t i i ii t h
M M
1 1
tj j j Tt t i ij ii t hC C x
Solve
1:j qfor
jtMjtC
DCurrentDictionary
History Information
New Data Information
1jtM
1jtC
![Page 42: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/42.jpg)
Our Online Approach
Time Complexity O(n/h) = O(n)
Memory Complexity O(1)
jtM
jtCRecord and only.
![Page 43: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/43.jpg)
Online Robust Dictionary Learning
• Our Online Approach• Robustness Analysis• Discussion
![Page 44: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/44.jpg)
Our Online Approach
Data
History Current
Min-batch
1,...,t h tx x
1,...,t h t Sparse code:
Data:
jt hM
jt hC
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1
tj j j Tt t h i i ii t h
M M
1
tj j j Tt t h i ij ii t hC C x
Solve
1:j qfor
jtMjtC
DCurrentDictionary
![Page 45: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/45.jpg)
Robustness Analysis
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1
tj j j Tt t h i i ii t h
M M
1
tj j j Tt t h i ij ii t hC C x
Solve
2
,:1 1
min ,: ,:t h t
ji ij i ij i
D ji i t h
x D j x D j
![Page 46: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/46.jpg)
Robustness Analysis (Proof)
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1
tj j j Tt t h i i ii t h
M M
1
tj j j Tt t h i ij ii t hC C x
Solve
2
,:1 1
min ,: ,:t h t
ji ij i ij i
D ji i t h
x D j x D j
![Page 47: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/47.jpg)
Robustness Analysis (Proof)
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1
tj j j Tt t h i i ii t h
M M
1
tj j j Tt t h i ij ii t hC C x
Solve
2 2
,:1 1
min ,: ,:t h t
j ji ij i i ij i
D ji i t h
x D j x D j
21
,:
ji
ijx D j
, 1,...,i t h t
Iterative Reweighted Least Squares
![Page 48: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/48.jpg)
Robustness Analysis (Proof)
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1
tj j j Tt t h i i ii t h
M M
1
tj j j Tt t h i ij ii t hC C x
Solve
21
,:
ji
ijx D j
, 1,...,i t h t
,:j jt tC D j M
1 1
t h tj j T j Tt i i i i i ii i t h
M
1 1
t h tj j T j Tt i ij i i ij ii i t hC x x
Solve
1
kj j Tk i ij iiC x
1
kj j Tk i i ii
M
![Page 49: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/49.jpg)
Robustness Analysis (Proof)
,:j jt tC D j M
, 1,...,i t h t 21
,:
ji
ijx D j
1
tj j j Tt t h i i ii t h
M M
1
tj j j Tt t h i ij ii t hC C x
Solve
21
,:
ji
ijx D j
, 1,...,i t h t
,:j jt tC D j MSolve
1 1
tj j j Tt t i i ii t h
M M
1 1
tj j j Tt t i ij ii t hC C x
![Page 50: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/50.jpg)
Robustness Analysis
,:minD j
History data New data
1
,:t
ij ii t h
x D j
21
,:t h
ji ij i
i
x D j
![Page 51: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/51.jpg)
Robustness Analysis
,:minD j
History data New data
1
,:t
ij ii t h
x D j
![Page 52: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/52.jpg)
Robustness Analysis
,:minD j
History data New data
1
,:ji
ij ix D j
21
,:t
ji ij i
i t h
x D j
Outliers have small weights
![Page 53: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/53.jpg)
Robustness Analysis
,:minD j
History data New data
21
,:t h
ji ij i
i
x D j
Help New Data Term
![Page 54: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/54.jpg)
Robustness Analysis
Robustness Boosting
,:minD j
History data New data
1
,:t
ij ii t h
x D j
21
,:t h
ji ij i
i
x D j
ji , 1,...,i t h t
Weights of new data (robust information)
Become history weights
![Page 55: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/55.jpg)
Robustness Analysis
work together
,:minD j
History data New data
1
,:t
ij ii t h
x D j
21
,:t h
ji ij i
i
x D j
![Page 56: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/56.jpg)
Online Robust Dictionary Learning
• Our Online Approach• Robustness Analysis• Discussion
![Page 57: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/57.jpg)
Current
Discussion
Inliers
Outliers
Dictionary
Dictionary Update
History
online
Robust
Forget history Data
Require Whole Data
Challenging
![Page 58: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/58.jpg)
Discussion
online
Robust
Forget history Data
Require Whole Data
1 1
tj j j Tt t i i ii t h
M M
1 1
tj j j Tt t i ij ii t hC C x
Robust information update:
Robustness: encode robust information.
ji
2
,:1 1
min ,: ,:t h t
ji ij i ij i
D ji i t h
x D j x D j
by
,:j jt tC D j MStatistic parameter:
![Page 59: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/59.jpg)
Discussion (Limitation)
,:minD j
1
,:t
ij ii t h
x D j
21
,:t h
ji ij i
i
x D j
Initial bias: it cannot handle the extreme case where the initial data are primarily outliers.
History data New data
Online:
Batch: ,:minD j
1
,:t
ij ii t h
x D j
1
,:t h
ij ii
x D j
![Page 60: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/60.jpg)
Experiments
![Page 61: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/61.jpg)
Synthetic Data
Synthetic Data:
Generate
Sparse coefficient: Dictionary: 1,..., n D
clear data: , 1,...,i ix D i n
Training data: 0 , 1,...,i i ix x i n
Outlier (Laplace noise)
![Page 62: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/62.jpg)
Synthetic Data
Synthetic Data:
BRDL ORDL KSVD ODL15dB 0.051 0.067 0.412 0.50520dB 0.034 0.042 0.224 0.28225dB 0.021 0.030 0.121 0.159
KSVD [Aharon et al 2006]ODL [Mairal et al 2010]
20
21
1n
i L ii
x Dn
Learnt Dictionary
L2 norm data fitting
![Page 63: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/63.jpg)
Synthetic Data
Synthetic Data:
![Page 64: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/64.jpg)
Digit Recognition
Digit Recognition:
Digital with Outliers
Data set: MNIST and USPS
![Page 65: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/65.jpg)
Digit Recognition
Digit Recognition:
Our Learnt Dictionary
Learnt Dictionary of L2 norm data-fitting
![Page 66: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/66.jpg)
Online Robust Dictionary Learning
Digit Recognition:
BRDL ORDL KSVD ODLMNIST 18.1 22.7 39.3 34.3USPS 27.3 29.4 45.3 42.5
KSVD [Aharon et al 2006]ODL [Mairal et al 2010]
L2 norm data fitting
![Page 67: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/67.jpg)
Digit Recognition
Digit Recognition:
Time Comparison (in second) in digit recognition in MNIST dataset (with Outlier)
for each digit
Digit 0 1 2 3 4 5 6 7 8 9
Size 5323 6742 5958 6131 5842 5421 5918 6265 5851 5949
ORDL 187 214 188 194 185 171 187 198 185 188
BRDL 2424 3466 2440 2587 2392 1979 2359 4115 2344 2402
![Page 68: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/68.jpg)
Background Subtraction
Video frames(Training Data)
Background Outlier
![Page 69: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/69.jpg)
Background Subtraction
Dataset [Zhao 2011]
KSVD [Aharon et al 2006]ORDL: Online Robust Dictionary LearningBRDL: Batch Robust Dictionary Learning [Zhao 2011]
KSVD ORDL BRDL
![Page 70: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/70.jpg)
Background Subtraction
Traffic light status 1
ORDL BRDL
![Page 71: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/71.jpg)
Background Subtraction
Traffic light status 2
ORDL BRDL
![Page 72: Online Robust Dictionary Learning](https://reader036.fdocuments.net/reader036/viewer/2022062309/5681521f550346895dc06442/html5/thumbnails/72.jpg)
Background Subtraction
Comparison of time and memory