At A ATA'T T ATA'T ATA ATA symmetric nxn matrix AtA symmetric ATA orthogonally diagonalizable Let B

download At A ATA'T T ATA'T ATA ATA symmetric nxn matrix AtA symmetric ATA orthogonally diagonalizable Let B

of 7

  • date post

    13-Jul-2020
  • Category

    Documents

  • view

    13
  • download

    0

Embed Size (px)

Transcript of At A ATA'T T ATA'T ATA ATA symmetric nxn matrix AtA symmetric ATA orthogonally diagonalizable Let B

  • Singular Value Decompositions

    A men matrix

    Aim Find orthonormal bases Bc R and Cc IR such that

    Apic is as simple as possible A

    Matrix of Ta with respect to B and C

    Important Observation AtA T ATA'T ATA ATA symmetric

    nxn matrix

    AtA symmetric ATA orthogonally diagonalizable Let B Ii In C 112 an orthogonal basis at eigenvectors A ATA

    eigenvalue ATA y Tiki 7W all i

    Lit ATA z Litt y recall Hei11 1 Kitui l A I care Xi HA If X T 0 For all i

    Perhaps after reordering assume 7 3727 7 Tu 0

    Definition The singular values of A are the numbers 5 7 027 70 70 where T TX

    key Fact c HA till

    Theneur Assume 5 Tr F O and Gre Ju o Then Rank A r and Are A 3 is an orthogonal basis For Col A

  • Range Ta Proof he 4 C 112 a basis Ay Ayn spans Cska

    HA 11 ti Aei e Ei o

    A Yi A Ir spans Col A and all elements are non zero Let i j A I A I Aei TAej E'T ATA Ij VIT Ij

    Xj Iitej Xj Ei Yj 0 Ei Ij orthogonal it i j

    A Yi Ayer is orthogonal set with non zero vectors L I

    AI AI is an orthogonal basis For ICA

    Rank A r

    Example

    a IF E a IF E

    p.io a a

    Choose Yi Ei Iz 13 13 T T TAI 4 y AA Er Iz A AI E 4 I 0 are eigenvalues

    2 I 0 are singular values

    Ay AE Free l i basis For GILA

    obvious in this case as I Ei Iz Ez

  • t HEY is y y.AE FAve

    2

    I I T T maps the

    hollow sphere to this

    solid ellipse restricted to

    sphere A radius

    1

    Important General Fact center e in IR

    Max HAIK where HE11 1 Max OT Tn 5

    Max 11Hell where 11111 d do

    I restricted to sphere A radius d center e in R

    M in HAIK where HE11 1 Mino on on

    Min 11Hell where 11111 d don

    Back to general case

    let Ii Aei Ati Fm sie

    Key Facts

    y I Iir is an orthonormal basis Fu Colla

    Y Aki Ti Ii Tor sie r Observation we can extend I Iir is an orthonormal basis

    for all 112 as follows

    a Find basis for NullAT Colla t

  • b Apply Gram Schmidt and normalize to get orthonormal basis Ertl In must have size m as dim call A I mdim Colla f

    c Take union to give ki Er art Em an orthonormal

    basis for IR

    Singular Value Decompositions

    Let It be an mxn matrix together with

    413 y In C1R orthonormalbasis at eigenvectors et ATA with singular values zero

    01 3 02 3 Jr a S u CATA Ii Tiki z C I Em C Rm orthonormal basis with AI o y For all I E n Ap c AI c CAIL Aem AI cA

    Matrix associatedto

    y withrespect to

    o le Lorenc E e c Band C

    µ 0 JT often denoted ZO 0 Important Consequence If U y Lm ki En

    T T u i'Av no o TITEL F v

    O O

    A

    g v u or o

    O O

    7 Called a singular value decomposition

    of A

  • Remarks

    n TA m R s R I I AI non zero

    e OT di r

    ti o Imma

    I Ir Yrt In ki Er _ur i Im

    1CAT Nuh A Colla NU AT

    Overview at Finding SVD i

    1 Orthogonally diagonalize ATA giving orthonormalbasis I In

    z Reorder so that eigenvalues are decreasing 7 77 7 In o non zero zero

    Take square roots to give T za 70 707 7 En i

    Define Ii Aki Fm I sie r

    Extend y yr to orthonormal basis ki Ir 4mi i Em

    6 U Lee Eun V y Lin A U I v't E no o

    s.v.is O 0

    Terminology ki Eun left singular vectors at A

    er yrs right singular vectors at A

    SVD have many important applications

  • Example Image Processing

    Imagine we have a greyscale image such as the following

    512 512

    resolution

    We can encode this data with a 512 512 matrix A

    where each entry represents the brightness at the corresponding

    pixel

    SVD A U I v't

    For any lek E 512 let II

    And Ar U I zvT

    It K E Ronk A Rank Ate k

    Important Fact

    A is a best possible rank k approximation of A

    P entries are

    as

    close as possible

  • Ak E E VI x 0 212 t t 5k IF T

    Need 12 2k 512 numbers todetermine

    Each Ak corresponds to a compressed version of original

    image A

    This has all sorts of applications in A.I cognative science

    and machine learning not to mention Letting you stare more

    pictures on your phone