ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines
-
Upload
shinichi-tamura -
Category
Data & Analytics
-
view
56 -
download
3
Transcript of ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines
![Page 1: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/1.jpg)
ESL 17.3.2-17.4
Graphical Lasso& Boltzmann Machines
June 8, 2015Talk by Shinichi TAMURA
Mathematical Informatics Lab @ NAIST
![Page 2: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/2.jpg)
Today's topics
![Page 3: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/3.jpg)
Today's topics
![Page 4: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/4.jpg)
[Review] Properties of graphical models
What is the Graphical Model
![Page 5: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/5.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
1
2
34
5
・ ・ 0 0 ・
・ ・ ・ 0 0
0 ・ ・ ・ 0
0 0 ・ ・ ・
・ 0 0 ・ ・
![Page 6: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/6.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
![Page 7: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/7.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
![Page 8: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/8.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
![Page 9: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/9.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
θjk=0
![Page 10: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/10.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
![Page 11: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/11.jpg)
[Review] Properties of graphical models
Fitting Gaussian Graphical Models
![Page 12: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/12.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
![Page 13: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/13.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
WΘ Γ
? ? ?
1
2
34
5
![Page 14: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/14.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0
WΘ Γ0 0
0 0
0 0
0 0
0 0
?
1
2
34
5
?
![Page 15: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/15.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0
WΘ Γ0 0
0 0
0 0
0 0
0 0
?
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
1
2
34
5
![Page 16: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/16.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0wjk=sjk
WΘ Γ0 0
0 0
0 0
0 0
0 0
s11 s12 s15
s21 s22 s23
s32 s33 s34
s43 s44 s45
s51 s54 s55
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
1
2
34
5
![Page 17: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/17.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0wjk=sjk
WΘ Γ? ? 0 0 ?
? ? ? 0 0
0 ? ? ? 0
0 0 ? ? ?
? 0 0 ? ?
s11 s12 ? ? s15
s21 s22 s23 ? ?
? s32 s33 s34 ?
? ? s43 s44 s45
s51 ? ? s54 s55
0 0 ? ? 0
0 0 0 ? ?
? 0 0 0 ?
? ? 0 0 0
0 ? ? 0 0
1
2
34
5
![Page 18: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/18.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0wjk=sjk
θjk=0 wjk=0
WΘ Γ? ? 0 0 ?
? ? ? 0 0
0 ? ? ? 0
0 0 ? ? ?
? 0 0 ? ?
s11 s12 ? ? s15
s21 s22 s23 ? ?
? s32 s33 s34 ?
? ? s43 s44 s45
s51 ? ? s54 s55
0 0 ? ? 0
0 0 0 ? ?
? 0 0 0 ?
? ? 0 0 0
0 ? ? 0 0
1
2
34
5
![Page 19: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/19.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0wjk=sjk
θjk=0 wjk=0
WΘ Γ? ? 0 0 ?
? ? ? 0 0
0 ? ? ? 0
0 0 ? ? ?
? 0 0 ? ?
s11 s12 ? ? s15
s21 s22 s23 ? ?
? s32 s33 s34 ?
? ? s43 s44 s45
s51 ? ? s54 s55
0 0 ? ? 0
0 0 0 ? ?
? 0 0 0 ?
? ? 0 0 0
0 ? ? 0 0
1
2
34
5
![Page 20: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/20.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0wjk=sjk
θjk=0 wjk=0
WΘ Γ? ? 0 0 ?
? ? ? 0 0
0 ? ? ? 0
0 0 ? ? ?
? 0 0 ? ?
s11 s12 ? ? s15
s21 s22 s23 ? ?
? s32 s33 s34 ?
? ? s43 s44 s45
s51 ? ? s54 s55
0 0 ? ? 0
0 0 0 ? ?
? 0 0 0 ?
? ? 0 0 0
0 ? ? 0 0
1
2
34
5
![Page 21: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/21.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
θjk=0 γjk=0wjk=sjk
θjk=0 wjk=0
WΘ Γ? ? 0 0 ?
? ? ? 0 0
0 ? ? ? 0
0 0 ? ? ?
? 0 0 ? ?
s11 s12 ? ? s15
s21 s22 s23 ? ?
? s32 s33 s34 ?
? ? s43 s44 s45
s51 ? ? s54 s55
0 0 ? ? 0
0 0 0 ? ?
? 0 0 0 ?
? ? 0 0 0
0 ? ? 0 0
1
2
34
5
![Page 22: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/22.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W−S−Γ=0.
WΘ Γ
Θ W Γ
1
2
34
5
![Page 23: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/23.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
w12−s12−γ12=0
WΘ Γ
Θ11 θ12
θT12 θ22
W11 w12
wT12 w22
Γ11 γ12
γT12 γ22
1
2
34
5
![Page 24: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/24.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
w12−s12−γ12=0
w12−s12−γ12=0
W11β−s12−γ12=0 (β=θ12/θ22)
W*11β*−s*12=0
WΘ Γ
Θ11 θ12
θT12 θ22
W11 w12
wT12 w22
Γ11 γ12
γT12 γ22
1
2
34
5
![Page 25: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/25.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
WΘ Γ1
2
34
5
![Page 26: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/26.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
WΘ Γ1
2
34
5
![Page 27: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/27.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
WΘ Γ1
2
34
5
![Page 28: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/28.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
WΘ Γ1
2
34
5
![Page 29: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/29.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
WΘ Γ1
2
34
5
![Page 30: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/30.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
WΘ Γ1
2
34
5
![Page 31: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/31.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W=S+λI
W*11β*−s*12=0
θ12=−βθ22
![Page 32: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/32.jpg)
[Review] Properties of graphical models
Block-wise Algorithm for GGM
W=S+λI
W*11β*−s*12=0
θ12=−βθ22
Θ=W-1
![Page 33: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/33.jpg)
Today's topics
![Page 34: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/34.jpg)
Today's topics
![Page 35: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/35.jpg)
Graphical Lasso
How to Estimate Graph Structure
θjk
![Page 36: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/36.jpg)
Graphical Lasso
How to Estimate Graph Structure
θjk
![Page 37: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/37.jpg)
Graphical Lasso
How to Estimate Graph Structure
θjk
![Page 38: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/38.jpg)
Graphical Lasso
How to Estimate Graph Structure
θjk
![Page 39: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/39.jpg)
Graphical Lasso
How to Estimate Graph Structure
θjk
![Page 40: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/40.jpg)
Graphical Lasso
How to Estimate Graph Structure
θjk
![Page 41: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/41.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 42: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/42.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 43: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/43.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 44: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/44.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 45: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/45.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 46: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/46.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 47: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/47.jpg)
Graphical Lasso
Problem of lasso regularization
![Page 48: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/48.jpg)
Graphical Lasso
Problem of lasso regularization
![Page 49: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/49.jpg)
Graphical Lasso
Sub-derivative
![Page 50: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/50.jpg)
Graphical Lasso
Sub-derivative
x0
yƒ( )=| |
![Page 51: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/51.jpg)
Graphical Lasso
Sub-derivative
f(x)=|x|{-1} x<0[-1,1] x=0{1} x>0
![Page 52: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/52.jpg)
Graphical Lasso
Sub-derivative
f(x)=|x|{-1} x<0[-1,1] x=0{1} x>0
“Sign” “sign”= -1 (θ<0)
Sign(θ) ∈ [-1,1] (θ=0)= 1 (θ>0){
![Page 53: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/53.jpg)
Graphical Lasso
Sub-derivative
f(x)=|x|{-1} x<0[-1,1] x=0{1} x>0
“Sign” “sign”= -1 (θ<0)
Sign(θ) ∈ [-1,1] (θ=0)= 1 (θ>0)
Sign(0) sign(0)=0
{
![Page 54: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/54.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 55: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/55.jpg)
Graphical Lasso
Gradient Equation
![Page 56: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/56.jpg)
Graphical Lasso
Gradient Equation
![Page 57: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/57.jpg)
Graphical Lasso
Gradient Equation
wii=sii+λ Θ
![Page 58: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/58.jpg)
Graphical Lasso
Gradient Equation
wii=sii+λ Θ
wii=sii
![Page 59: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/59.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 60: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/60.jpg)
Graphical Lasso
Block-wise algorithm
w12–s12–λ∙Sign(θ12)=0
WΘ
Θ11 θ12
θT12 θ22
W11 w12
wT12 w22
![Page 61: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/61.jpg)
Graphical Lasso
Block-wise algorithm
w12–s12–λ∙Sign(θ12)=0
θ12=–θ22W-111w12 and θ22>0
w12–s12+λ∙Sign(W-111w12)=0
![Page 62: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/62.jpg)
Graphical Lasso
Block-wise algorithm
w12–s12–λ∙Sign(θ12)=0
θ12=–θ22W-111w12 and θ22>0
w12–s12+λ∙Sign(W-111w12)=0
W11 w12
wT12 w22
![Page 63: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/63.jpg)
Graphical Lasso
Block-wise algorithm
w12–s12–λ∙Sign(θ12)=0
θ12=–θ22W-111w12 and θ22>0
w12–s12+λ∙Sign(W-111w12)=0
β=W-111w12
W11β–s12+λ∙Sign(β)=0
![Page 64: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/64.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 65: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/65.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 66: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/66.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
W11β–s12+λ∙Sign(β)=0
![Page 67: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/67.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
W11β–s12+λ∙Sign(β)=0
![Page 68: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/68.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
W11β–s12+λ∙Sign(β)=0
![Page 69: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/69.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
W11β–s12+λ∙Sign(β)=0
![Page 70: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/70.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
![Page 71: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/71.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
y
![Page 72: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/72.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
y
![Page 73: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/73.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
y
![Page 74: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/74.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
y
![Page 75: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/75.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
y y
0
y
![Page 76: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/76.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
f(βj)
y y
0
y
![Page 77: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/77.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
![Page 78: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/78.jpg)
Graphical Lasso
Coordinate-wise Decent Algorithm
j=1,2,…,p-1
![Page 79: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/79.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 80: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/80.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 81: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/81.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 82: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/82.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 83: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/83.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 84: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/84.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 85: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/85.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 86: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/86.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 87: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/87.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 88: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/88.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 89: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/89.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 90: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/90.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 91: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/91.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 92: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/92.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 93: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/93.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 94: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/94.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 95: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/95.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 96: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/96.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 97: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/97.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 98: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/98.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 99: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/99.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 100: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/100.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 101: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/101.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 102: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/102.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 103: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/103.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 104: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/104.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 105: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/105.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 106: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/106.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 107: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/107.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 108: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/108.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 109: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/109.jpg)
Graphical Lasso
Overview of the Algorithm
W=S+λI
f(βj)=0
θ12=−βθ22
![Page 110: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/110.jpg)
Graphical Lasso
Key Points of the Algorithm
![Page 111: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/111.jpg)
Graphical Lasso
Setting of λ
λ
y
0
![Page 112: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/112.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λ=0
![Page 113: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/113.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λ=7
![Page 114: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/114.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λ=14
![Page 115: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/115.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λ=27
![Page 116: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/116.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λ=36
![Page 117: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/117.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λ=0 λ=7 λ=14 λ=27 λ=36
![Page 118: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/118.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λjkθjk
![Page 119: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/119.jpg)
Graphical Lasso
Setting of λ
λ
y
0
λjkθjk
j k λjk=∞
![Page 120: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/120.jpg)
Graphical Lasso
Treating Unobserved Nodes
![Page 121: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/121.jpg)
Graphical Lasso
Treating Unobserved Nodes
![Page 122: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/122.jpg)
Today's topics
![Page 123: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/123.jpg)
Today's topics
![Page 124: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/124.jpg)
Boltzmann Machines
What is the Boltzmann Machine
![Page 125: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/125.jpg)
Boltzmann Machines
What is the Boltzmann Machine
![Page 126: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/126.jpg)
Boltzmann Machines
What is the Boltzmann Machine
![Page 127: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/127.jpg)
Boltzmann Machines
What is the Boltzmann Machine
![Page 128: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/128.jpg)
Boltzmann Machines
What is the Boltzmann Machine
![Page 129: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/129.jpg)
Boltzmann Machines
The Joint Distribution
Z
![Page 130: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/130.jpg)
Boltzmann Machines
The Joint Distribution
Z
Only pairwiserelation is modeled.
![Page 131: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/131.jpg)
Boltzmann Machines
The Joint Distribution
Z
X0≡1
![Page 132: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/132.jpg)
Boltzmann Machines
The Conditional Distribution
Xj X-j
![Page 133: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/133.jpg)
Boltzmann Machines
The Conditional Distribution
Xj X-j
![Page 134: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/134.jpg)
Boltzmann Machines
The Conditional Distribution
![Page 135: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/135.jpg)
Boltzmann Machines
Estimation for Known Graph
![Page 136: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/136.jpg)
Boltzmann Machines
Estimation for Known Graph
![Page 137: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/137.jpg)
Boltzmann Machines
Estimation for Known Graph
![Page 138: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/138.jpg)
Boltzmann Machines
Estimation for Known Graph
xi=(xi1,…,xip) i=1,…,N
![Page 139: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/139.jpg)
Boltzmann Machines
Estimation for Known Graph
xi=(xi1,…,xip) i=1,…,N
![Page 140: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/140.jpg)
Boltzmann Machines
Estimation for Known Graph
xi=(xi1,…,xip) i=1,…,N
![Page 141: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/141.jpg)
Boltzmann Machines
Estimation for Known Graph
![Page 142: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/142.jpg)
Boltzmann Machines
Estimation for Known Graph
2p
![Page 143: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/143.jpg)
Boltzmann Machines
Estimation for Known Graph
p (<30)
![Page 144: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/144.jpg)
Boltzmann Machines
Estimation for Known Graph
p (<30)
•
![Page 145: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/145.jpg)
Boltzmann Machines
Estimation for Known Graph
p (<30)
•
•
![Page 146: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/146.jpg)
Boltzmann Machines
Estimation for Known Graph
p (<30)
•
•
•
![Page 147: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/147.jpg)
Boltzmann Machines
Estimation for Known Graph
p (≥30)
![Page 148: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/148.jpg)
Boltzmann Machines
Estimation for Known Graph
p (≥30)
•
![Page 149: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/149.jpg)
Boltzmann Machines
Estimation for Known Graph
p (≥30)
•
•
![Page 150: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/150.jpg)
Boltzmann Machines
Estimation for Known Graph
![Page 151: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/151.jpg)
Boltzmann Machines
Hidden Nodes
1 2
34
![Page 152: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/152.jpg)
Boltzmann Machines
Hidden Nodes
1 2
34
5
![Page 153: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/153.jpg)
Boltzmann Machines
Hidden Nodes
1 2
34
5
![Page 154: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/154.jpg)
Boltzmann Machines
Hidden Nodes
![Page 155: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/155.jpg)
Boltzmann Machines
Hidden Nodes
![Page 156: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/156.jpg)
Boltzmann Machines
Hidden Nodes
![Page 157: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/157.jpg)
Boltzmann Machines
Estimating Graph Structure
•
•
•
![Page 158: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/158.jpg)
Boltzmann Machines
Difference from Graphical Lasso
![Page 159: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/159.jpg)
Boltzmann Machines
Difference from Graphical Lasso
![Page 160: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/160.jpg)
Boltzmann Machines
Difference from Graphical Lasso
![Page 161: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/161.jpg)
Boltzmann Machines
Restricted Boltzmann Machine
![Page 162: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/162.jpg)
Boltzmann Machines
Restricted Boltzmann Machines
![Page 163: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/163.jpg)
Boltzmann Machines
Restricted Boltzmann Machines
![Page 164: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/164.jpg)
Boltzmann Machines
Restricted Boltzmann Machines
![Page 165: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/165.jpg)
Boltzmann Machines
Restricted Boltzmann Machines
![Page 166: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/166.jpg)
Boltzmann Machines
Restricted Boltzmann Machines
![Page 167: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/167.jpg)
Boltzmann Machines
Restricted Boltzmann Machines
![Page 168: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/168.jpg)
Boltzmann Machines
Learning RBM
![Page 169: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/169.jpg)
Boltzmann Machines
Learning RBM
![Page 170: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/170.jpg)
Boltzmann Machines
Learning RBM
•
![Page 171: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/171.jpg)
Boltzmann Machines
Learning RBM
•
•
![Page 172: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/172.jpg)
Boltzmann Machines
Learning RBM
•
•
![Page 173: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/173.jpg)
Boltzmann Machines
Learning RBM
•
•
![Page 174: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/174.jpg)
Boltzmann Machines
Learning RBM
•
•
![Page 175: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/175.jpg)
Boltzmann Machines
Learning RBM
•
•
![Page 176: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/176.jpg)
Today's topics
![Page 177: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/177.jpg)
Today's topics
![Page 178: ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines](https://reader030.fdocuments.net/reader030/viewer/2022032623/55d03107bb61eb0b0e8b466b/html5/thumbnails/178.jpg)
Today's topics