Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING...

38
ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) EE210A: Adaptation and Learning Professor Ali H. Sayed

Transcript of Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING...

Page 1: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED)

EE210A: Adaptation and Learning Professor Ali H. Sayed

Page 2: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2

MAIN RESULT FROM LAST LECTURE

Page 3: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 3

MAIN RESULT FROM LAST LECTURE

Page 4: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 4

LECTURE #02

VECTOR AND COMPLEX-VALUED MEAN-SQUARE ERROR ESTIMATION Sections in order: B.1, A.3, A.4, 2.1, 2.2, 2.3

Page 5: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 5

HERMITIAN MATRICES

Page 6: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 6

SPECTRAL DECOMPOSITION

Page 7: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 7

POSITIVE-DEFINITE MATRICES

Page 8: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 8

POSITIVE-DEFINITE MATRICES

Page 9: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 9

COMPLEX VARIABLES

Page 10: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 10

ORTHOGONAL RANDOM VARIABLES

Page 11: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 11

EXAMPLE

Page 12: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 12

VECTOR-VALUED RANDOM VARIABLES

Page 13: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 13

EXAMPLE

Page 14: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 14

PROPERTIES OF COVARIANCE MATRICES

Page 15: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 15

ESTIMATING A SCALAR FROM A VECTOR

Page 16: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 16

EXAMPLE

Page 17: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 17

EXAMPLE

Page 18: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 18

ESTIMATING A VECTOR FROM A VECTOR

Page 19: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 19

EQUIVALENT CRITERION

Page 20: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 20

EQUIVALENT CRITERION

Page 21: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 21

OPTIMAL ESTIMATOR

Page 22: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 22

GAUSSIAN VARIABLES

Page 23: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 23

VECTOR GAUSSIAN VARIABLES

Page 24: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 24

CIRCULARITY ASSUMPTIONS

Page 25: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 25

CIRCULAR GAUSSIAN VARIABLES

Page 26: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 26

ESTIMATION WITH GAUSSIAN DATA

Page 27: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 27

TRIANGULAR FACTORIZATION

Page 28: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 28

AFFINE RELATION

Page 29: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 29

ZERO MEAN CASE

Page 30: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 30

EQUIVALENT CRITERION

Page 31: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 31

EQUIVALENT CRITERION

Page 32: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 32

EQUIVALENT CRITERION

Page 33: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 33

EQUIVALENT CRITERION

Page 34: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 34

EQUIVALENT CRITERION

Page 35: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 35

COMPUTER PROJECT

Page 36: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 36

COMPUTER PROJECT

Page 37: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 37

COMPUTER PROJECT SOLUTION

Page 38: Professor Ali H. Sayed - ASL · 2019-01-13 · Professor Ali H. Sayed . ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2 ... LECTURE #02 . VECTOR

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 38

COMPUTER PROJECT SOLUTION