LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B:...
Transcript of LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B:...
![Page 1: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/1.jpg)
Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.
Course EE210B Spring Quarter 2015
1
Professor Ali H. Sayed UCLA Electrical Engineering
INFERENCE OVER NETWORKS
LECTURE #2: Complex Gradients Vectors
![Page 2: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/2.jpg)
Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.
Course EE210B Spring Quarter 2015
Part I: Background Material
![Page 3: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/3.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 3
Reference
Appendix A (Complex Gradient Vectors, pp. 712-719):
A. H. Sayed, ``Adaptation, learning, and optimization over networks,'' Foundations and Trends in Machine Learning, vol. 7, issue 4-5, pp. 311-801, NOW Publishers, 2014.
![Page 4: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/4.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 4
Setting
.
![Page 5: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/5.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 5
Cauchy-Riemann Conditions
![Page 6: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/6.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 6
Cauchy-Riemann Conditions
![Page 7: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/7.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 7
Cauchy-Riemann Conditions
![Page 8: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/8.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 8
Cauchy-Riemann Conditions
![Page 9: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/9.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 9
Cauchy-Riemann Conditions
![Page 10: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/10.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 10
Example #A.1 10 10
![Page 11: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/11.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 11
Example #A.1 11 11
![Page 12: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/12.jpg)
Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.
Course EE210B Spring Quarter 2015
Scalar Arguments
![Page 13: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/13.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 13
Scalar Arguments
![Page 14: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/14.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 14
Scalar Arguments
![Page 15: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/15.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 15
Scalar Arguments
![Page 16: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/16.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 16
Scalar Arguments
![Page 17: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/17.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 17
Scalar Arguments
![Page 18: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/18.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 18
Example #A.2 18 18
![Page 19: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/19.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 19
Example #A.2 19 19
![Page 20: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/20.jpg)
Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.
Course EE210B Spring Quarter 2015
Vector Arguments
![Page 21: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/21.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 21
Vector Arguments
![Page 22: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/22.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 22
Vector Arguments
![Page 23: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/23.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 23
Vector Arguments
![Page 24: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/24.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 24
Vector Arguments
![Page 25: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/25.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 25
Vector Arguments
![Page 26: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/26.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 26
Example #A.3 26 26
![Page 27: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/27.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 27
Example #A.3 27 27
![Page 28: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/28.jpg)
Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.
Course EE210B Spring Quarter 2015
Real Arguments
![Page 29: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/29.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 29
Real Arguments
![Page 30: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/30.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 30
Real Arguments
![Page 31: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/31.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 31
Example #A.4 31 31
![Page 32: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/32.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 32
Example #A.4 32 32
![Page 33: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/33.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 33
Example #A.4 33 33
![Page 34: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/34.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 34
Example #A.4 34 34
![Page 35: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/35.jpg)
Lecture #2: Complex Gradient Vectors EE210B: Inference over Networks (A. H. Sayed) 35
Example #A.5 35 35
![Page 36: LECTURE #2 C: ompel x Gradients Vectors · 3 . Lecture #2: Complex Gradient Vectors. EE210B: Inference over Networks (A. H. Sayed) Reference . Appendix A (Complex Gradient Vectors,](https://reader036.fdocuments.net/reader036/viewer/2022081607/5ed902be6714ca7f4768fad1/html5/thumbnails/36.jpg)
Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.
Course EE210B Spring Quarter 2015
End of Lecture