Approximation Techniques for Special Eigenvalue Problems ... fileApproximation Techniques for...

Technische Universitat Munchen

Approximation Techniques for SpecialEigenvalue Problems

Konrad Waldherr

June 2011

Joint work with Thomas Huckle

K. Waldherr: Approximation Techniques for Special Eigenvalue Problems

HDA 2011, Bonn, June 2011 1

Outline1 Problem Setting: Computation of Ground States

Physical ModelMathematical Model

2 Data-Sparse Representation Formats

3 Matrix Product StatesFormalismComputations with MPSUniqueness of MPSMinimization in terms of MPS

4 Subspace Methods in Terms of MPS

5 Numerical results

6 Conclusions

Problem: Computation of Ground StatesGiven:

• Physical system: 1D spin chain with N particles

1 2 3 4 5

• Interaction within the system (e.g. nearest-neighbor interaction)

1 2 3 4 5

• External interaction (e.g. exterior magnetic field)

1 2 3 4 5

Goal:• Find ground state, i.e. the state related to the smallest energy of

the system.

1 2 3 4 5

the system.

1 2 3 4 5

the system.

1 2 3 4 5

the system.

Mathematical Model• Any state of the system is represented by a vector x ∈ C2N

.• The physical system can be described by the HamiltonianH ∈ C2N×2N

• The Hamiltonian may be formulated as a weighted sum ofKronecker products of Pauli and identity matrices.

• Pauli matrices:

(0 11 0

), σy =

(0 −ii 0

), σz =

(1 00 −1

• Kronecker product:

A⊗B :=

a11 . . . a1n

.... . .

...am1 . . . amn

⊗B =

a11B . . . a1nB...

. . ....

am1B . . . amnB

• Then, the ground state corresponds to the eigenvector related

to the smallest eigenvalue.

.• The Hamiltonian may be formulated as a weighted sum of

Kronecker products of Pauli and identity matrices.• Pauli matrices:

(0 11 0

), σy =

(0 −ii 0

), σz =

(1 00 −1

A⊗B :=

a11 . . . a1n

.... . .

...am1 . . . amn

⊗B =

a11B . . . a1nB...

. . ....

am1B . . . amnB

• Then, the ground state corresponds to the eigenvector relatedto the smallest eigenvalue.

.• The Hamiltonian may be formulated as a weighted sum of

Kronecker products of Pauli and identity matrices.• Pauli matrices:

(0 11 0

), σy =

(0 −ii 0

), σz =

(1 00 −1

A⊗B :=

a11 . . . a1n

.... . .

...am1 . . . amn

⊗B =

a11B . . . a1nB...

. . ....

am1B . . . amnB

• Then, the ground state corresponds to the eigenvector related

to the smallest eigenvalue.

Example: Ising-type Hamiltonian

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I

+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I

+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I

+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz

+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I

+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I

+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I

+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I

+ I ⊗ I ⊗ I ⊗ I ⊗ σx

1 2 3 4 5

H = σz ⊗ σz ⊗ I ⊗ I ⊗ I+ I ⊗ σz ⊗ σz ⊗ I ⊗ I+ I ⊗ I ⊗ σz ⊗ σz ⊗ I+ I ⊗ I ⊗ I ⊗ σz ⊗ σz+ σx ⊗ I ⊗ I ⊗ I ⊗ I+ I ⊗ σx ⊗ I ⊗ I ⊗ I+ I ⊗ I ⊗ σx ⊗ I ⊗ I+ I ⊗ I ⊗ I ⊗ σx ⊗ I+ I ⊗ I ⊗ I ⊗ I ⊗ σx

Problem Setting• General formulation of the Hamiltonian H:

M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

N , Q(k)i ∈ {id, σx, σy, σz} ⊂ C2×2

• Variational ansatz: Minimization of the Rayleigh quotient:

minx 6=0

• Problems:

• the vector space V = C2N

grows exponentially in N ,• the problem is not generic.

• Idea: Find data-sparse representation formats that allow toovercome the curse of dimensionality.

M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

N , Q(k)i ∈ {id, σx, σy, σz} ⊂ C2×2

minx 6=0

• Problems:

• the vector space V = C2N

M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

N , Q(k)i ∈ {id, σx, σy, σz} ⊂ C2×2

minx 6=0

• Problems:• the vector space V = C2N

M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

N , Q(k)i ∈ {id, σx, σy, σz} ⊂ C2×2

minx 6=0

• Problems:• the vector space V = C2N

Data-sparse representation formats• The data-sparse format has to allow for

1. proper reproduction of the physical setting (e.g.nearest-neighbor interaction),

2. representations that are only polynomial in N ,3. good approximation properties,4. efficient evaluation of the Rayleigh quotient

minx∈U

xHx(1)

=⇒ efficient computation of both Hx and yHx.

• Problem setting:• Not: Find a data-sparse format of a given data structure• But: Use the data-sparse format as an ansatz for the

minimization of the Rayleigh quotient (1).

Data-sparse representation formats• The data-sparse format has to allow for

1. proper reproduction of the physical setting (e.g.nearest-neighbor interaction),

2. representations that are only polynomial in N ,3. good approximation properties,4. efficient evaluation of the Rayleigh quotient

minx∈U

xHx(1)

=⇒ efficient computation of both Hx and yHx.• Problem setting:

• Not: Find a data-sparse format of a given data structure• But: Use the data-sparse format as an ansatz for the

minimization of the Rayleigh quotient (1).

Tensor Decompositions• Any state x ∈ C2N

may be considered as a N th order tensor:

x = (xi1i2...iN )ij=0,1

i1 i2 iN

• Goal: Find convenient tensor decomposition as ansatz for (1)• Classical decomposition formats:

• Tucker decomposition,• Canonical decomposition.

• Physically motivated formats:• Matrix Product States (MPS) for 1D problems,• Projected Entangled Pair States (PEPS) for 2D problems.

x = (xi1i2...iN )ij=0,1

i1 i2 iN

• Goal: Find convenient tensor decomposition as ansatz for (1)

• Classical decomposition formats:• Tucker decomposition,• Canonical decomposition.

x = (xi1i2...iN )ij=0,1

i1 i2 iN

x = (xi1i2...iN )ij=0,1

i1 i2 iN

Decompositions of a tensor• Tucker decomposition:

Not advantageous in our case of a binary tensor.

• Canonical decomposition (CP):

Allows generalizations in several ways: (see [Huckle 2011])• blockings and mixed blockings,• permutations.

Decompositions of a tensor• Tucker decomposition:

Not advantageous in our case of a binary tensor.• Canonical decomposition (CP):

Allows generalizations in several ways: (see [Huckle 2011])• blockings and mixed blockings,• permutations.

Matrix Product States (MPS)• Consider again the linear 1D spin chain.

b b b b b

site j

• Each site j is related to a matrix pair A(0)j , A

(1)j ∈ CDj×Dj+1

b b b b b

A(ij)j

• For i = (i1, i2, . . . , iN )2, the vector component xi is given by

xi = xi1i2...iN = tr(A

(i1)1 ·A(i2)

2 · · ·A(iN )N

b b b b b

ij−1 ij ij+1 ij+2 ij+3

mj−1 mj mj+1 mj+2

HDA 2011, Bonn, June 2011 10

b b b b b

site j

(1)j ∈ CDj×Dj+1

b b b b b

A(ij)j

(i1)1 ·A(i2)

2 · · ·A(iN )N

b b b b b

mj−1 mj mj+1 mj+2

HDA 2011, Bonn, June 2011 10

b b b b b

site j

(1)j ∈ CDj×Dj+1

b b b b b

A(ij)j

(i1)1 ·A(i2)

2 · · ·A(iN )N

b b b b b

mj−1 mj mj+1 mj+2

HDA 2011, Bonn, June 2011 10

Matrix Product States• Thus, the vector x may be written as

2N∑i=1

xiei =∑

i1,...,ip

(i1)1 ·A(i2)

2 · · ·A(iN )N

)︸︷︷︸

=xi=xi1,...,iN

ei1i2...iN︸︷︷︸=ei

• Binary version of the Tensor Train concept. (see [Oseledets andTyrtyshnikov, 2009])

• MPS representation requires 2ND2 instead of 2N entries. (2.X)• Any state can be represented exactly by an MPS (but at the cost

of exponentially growing matrix dimension D),• For approximation problems, D has to scale only polynomially.

HDA 2011, Bonn, June 2011 11

2N∑i=1

xiei =∑

i1,...,ip

(i1)1 ·A(i2)

2 · · ·A(iN )N

)︸︷︷︸

=xi=xi1,...,iN

• MPS representation requires 2ND2 instead of 2N entries. (2.X)

• Any state can be represented exactly by an MPS (but at the costof exponentially growing matrix dimension D),

• For approximation problems, D has to scale only polynomially.(3.X)

HDA 2011, Bonn, June 2011 11

2N∑i=1

xiei =∑

i1,...,ip

(i1)1 ·A(i2)

2 · · ·A(iN )N

)︸︷︷︸

=xi=xi1,...,iN

• MPS representation requires 2ND2 instead of 2N entries. (2.X)• Any state can be represented exactly by an MPS (but at the cost

of exponentially growing matrix dimension D),• For approximation problems, D has to scale only polynomially.

HDA 2011, Bonn, June 2011 11

Matrix Product StatesExample

• Consider the case N = 2 and D = 1:

1∑i1,i2=0

(i1)1 a

)︸︷︷︸

=xi=xi1i2

(ei1 ⊗ ei2)︸︷︷︸=ei=ei1i2

(0)1 a

a(0)1 a

a(1)1 a

• For exact representation:

x1x4 = x2x3

• e1, e2, e3, e4 can be represented,• However, e1 + e4 = (1, 0, 0, 1)T cannot be represented.

• MPS are not closed under addition!

HDA 2011, Bonn, June 2011 12

1∑i1,i2=0

(i1)1 a

)︸︷︷︸

=xi=xi1i2

(ei1 ⊗ ei2)︸︷︷︸=ei=ei1i2

(0)1 a

a(0)1 a

a(1)1 a

x1x4 = x2x3

HDA 2011, Bonn, June 2011 12

1∑i1,i2=0

(i1)1 a

)︸︷︷︸

=xi=xi1i2

(ei1 ⊗ ei2)︸︷︷︸=ei=ei1i2

(0)1 a

a(0)1 a

a(1)1 a

x1x4 = x2x3

HDA 2011, Bonn, June 2011 12

1∑i1,i2=0

(i1)1 a

)︸︷︷︸

=xi=xi1i2

(ei1 ⊗ ei2)︸︷︷︸=ei=ei1i2

(0)1 a

a(0)1 a

a(1)1 a

x1x4 = x2x3

HDA 2011, Bonn, June 2011 12

Computations with MPS• Sum of two MPS vectors

x =∑

i1,...,iN

tr(A(i1)1 A

(i2)2 . . . A

(iN )N )ei1...iN

y =∑

i1,...,iN

tr(B(i1)1 B

(i2)2 . . . B

(iN )N )ei1...iN

x+ y =∑i1,...,iN

(i1)1 0

0 B(i1)1

(i2)2 0

0 B(i2)2

)︸︷︷︸

D′×D′

(iN )N 0

0 B(iN )N

)]ei1...iN

• Summing MPS vectors leads to an augmentation of the matrixsizes D → D′.

• Use compression techniques to keep the size of the MPSmatrices constant.

HDA 2011, Bonn, June 2011 13

x =∑

i1,...,iN

tr(A(i1)1 A

(i2)2 . . . A

(iN )N )ei1...iN

y =∑

i1,...,iN

tr(B(i1)1 B

(i2)2 . . . B

(iN )N )ei1...iN

x+ y =∑i1,...,iN

(i1)1 0

0 B(i1)1

(i2)2 0

0 B(i2)2

)︸︷︷︸

D′×D′

(iN )N 0

0 B(iN )N

)]ei1...iN

HDA 2011, Bonn, June 2011 13

x =∑

i1,...,iN

tr(A(i1)1 A

(i2)2 . . . A

(iN )N )ei1...iN

y =∑

i1,...,iN

tr(B(i1)1 B

(i2)2 . . . B

(iN )N )ei1...iN

x+ y =∑i1,...,iN

(i1)1 0

0 B(i1)1

(i2)2 0

0 B(i2)2

)︸︷︷︸

D′×D′

(iN )N 0

0 B(iN )N

)]ei1...iN

HDA 2011, Bonn, June 2011 13

Computations with MPS• Inner products

xHy =∑

i1,...,iN

xi1,...,iN︷︸︸︷(tr(A

(i1)1 · · · A(iN )

yi1,...,iN︷︸︸︷(tr(B

(i1)1 · · ·B(iN )

))=∑ij

a(i1)1;k1,k2

· · · a(iN )N ;kN ,k1

b(i1)1;m1,m2

· · · b(iN )N ;mN ,m1

Find an efficient ordering of the summations!• Matrix × vector

(M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

) ∑i1,...,iN

tr(A(i1)1 . . . A

(iN )N )ei1...iN

M∑k=1

y(k)MPS

=⇒ 4. XK. Waldherr: Approximation Techniques for Special Eigenvalue Problems

HDA 2011, Bonn, June 2011 14

Uniqueness of MPS• The MPS formalism is not unique:∑

i1,...,iN

(i1)1 ·A(i2)

2 · · ·A(iN )N

)ei1,...,iN

i1,...,iN

(A(i1)1 M−1

2 )(M2A(i2)2 M−1

3 ) · · · (MNA(iN )N )

)ei1,...,iN

• Replace the matrices A(0)j , A

(1)j by parts of unitary matrices using

the SVD: (A

)ΣjVj .

• Multiply the ΣjVj parts on the right neighbour.• Replace all A matrices by U matrices, up to one.• This remaining factor is considered to be optimized.• Leads to better convergence properties and numerical stability.

HDA 2011, Bonn, June 2011 15

i1,...,iN

(i1)1 ·A(i2)

2 · · ·A(iN )N

)ei1,...,iN

i1,...,iN

(A(i1)1 M−1

2 )(M2A(i2)2 M−1

3 ) · · · (MNA(iN )N )

)ei1,...,iN

the SVD: (A

)ΣjVj .

HDA 2011, Bonn, June 2011 15

i1,...,iN

(i1)1 ·A(i2)

2 · · ·A(iN )N

)ei1,...,iN

i1,...,iN

(A(i1)1 M−1

2 )(M2A(i2)2 M−1

3 ) · · · (MNA(iN )N )

)ei1,...,iN

the SVD: (A

)ΣjVj .

• Multiply the ΣjVj parts on the right neighbour.

• Replace all A matrices by U matrices, up to one.• This remaining factor is considered to be optimized.• Leads to better convergence properties and numerical stability.

HDA 2011, Bonn, June 2011 15

i1,...,iN

(i1)1 ·A(i2)

2 · · ·A(iN )N

)ei1,...,iN

i1,...,iN

(A(i1)1 M−1

2 )(M2A(i2)2 M−1

3 ) · · · (MNA(iN )N )

)ei1,...,iN

the SVD: (A

)ΣjVj .

• Multiply the ΣjVj parts on the right neighbour.• Replace all A matrices by U matrices, up to one.• This remaining factor is considered to be optimized.

• Leads to better convergence properties and numerical stability.

HDA 2011, Bonn, June 2011 15

i1,...,iN

(i1)1 ·A(i2)

2 · · ·A(iN )N

)ei1,...,iN

i1,...,iN

(A(i1)1 M−1

2 )(M2A(i2)2 M−1

3 ) · · · (MNA(iN )N )

)ei1,...,iN

the SVD: (A

)ΣjVj .

HDA 2011, Bonn, June 2011 15

Computation of the Ground StateHow to calculate the smallest eigenvalue of the Hamiltonian H?

1. Variational ansatz:

minxMPS

xHMPSHxMPS

xHMPSxMPS

• Find optimal matrix pairs A(ij)j for all sites j

• Use Alternating Least Squares: Keep all matrix pairs up toone fixed and optimize the remaining one

2. New idea: Use subspace iteration

minx∈Um

• Find optimal solution in the subspace Um

• Um is defined by MPS vectors

HDA 2011, Bonn, June 2011 16

Computation of the Ground StateHow to calculate the smallest eigenvalue of the Hamiltonian H?

1. Variational ansatz:

minxMPS

xHMPSHxMPS

xHMPSxMPS

• Find optimal matrix pairs A(ij)j for all sites j

• Use Alternating Least Squares: Keep all matrix pairs up toone fixed and optimize the remaining one

2. New idea: Use subspace iteration

minx∈Um

• Find optimal solution in the subspace Um

• Um is defined by MPS vectors

HDA 2011, Bonn, June 2011 16

Minimization Algorithm in terms of MPS• Start with initial guesses for the A(ij)

j matrices.

• Replace A(ij)j by U (ij)

j up to A(i1)1 .

• Consider all U matrices as fixed and optimize X1 = A1:

(∑ij

(i1)1 U

(i2)2 · · ·U(iN )

(∑ij

(i1)1 U

(i2)2 · · ·U(iN )

(i1)1 U

(i2)2 · · ·U(iN )

)H(∑ij

(i1)1 U

(i2)2 · · ·U(iN )

= minX1

X1HR1X1

with effective Hamiltonian R1

• =⇒ dense eigenvalue problem for the (2D2 × 2D2) matrix R1.• Orthogonalize the solution and continue with the right neighbor.

Modifications of this algorithm lead to the DMRG method.

HDA 2011, Bonn, June 2011 17

Subspace iteration for the smallest eigenvalue• Idea: Minimize the Rayleigh quotient xHHx

xHx over a subspace (!)Um := span{q1, . . . , qm}.

• Let Vm = (q1, . . . , qm) ∈ C2N×m

• The minimization over Um yields

minx∈Um\{0}

x=Vmy= min

y∈Cm\{0}

(Vmy)HH(Vmy)

(Vmy)HVmy

= miny∈Cm\{0}

yH(V HmHVm

yH (V HmVm) y

= miny∈Cm\{0}

a generalized eigenvalue problem Ay = λminBy of size m×m.• x = Vmy or restart with q1 = Vmy

• How to choose an appropriate subspace Um?

HDA 2011, Bonn, June 2011 18

• Let Vm = (q1, . . . , qm) ∈ C2N×m

minx∈Um\{0}

x=Vmy= min

y∈Cm\{0}

(Vmy)HH(Vmy)

(Vmy)HVmy

= miny∈Cm\{0}

yH(V HmHVm

yH (V HmVm) y

= miny∈Cm\{0}

a generalized eigenvalue problem Ay = λminBy of size m×m.

• x = Vmy or restart with q1 = Vmy

HDA 2011, Bonn, June 2011 18

• Let Vm = (q1, . . . , qm) ∈ C2N×m

minx∈Um\{0}

x=Vmy= min

y∈Cm\{0}

(Vmy)HH(Vmy)

(Vmy)HVmy

= miny∈Cm\{0}

yH(V HmHVm

yH (V HmVm) y

= miny∈Cm\{0}

HDA 2011, Bonn, June 2011 18

• Let Vm = (q1, . . . , qm) ∈ C2N×m

minx∈Um\{0}

x=Vmy= min

y∈Cm\{0}

(Vmy)HH(Vmy)

(Vmy)HVmy

= miny∈Cm\{0}

yH(V HmHVm

yH (V HmVm) y

= miny∈Cm\{0}

HDA 2011, Bonn, June 2011 18

Subspace iteration for the smallest eigenvalue• Choose the Krylov space Km(H, b) as subspace:

Um = Km(H, b) := span{b,Hb, . . . ,Hm−1b}

• This ansatz yields

A = V HmHVm =

bHHb bHH2b . . . bHHmbbHH2b bHH3b . . . bHHm+1b

......

. . ....

bHHmb bHHm+1b . . . bHH2m−1b

B = V HmVm =

bHb bHHb . . . bHHm−1bbHHb bHH3b . . . bHHmb

......

. . ....

bHHm−1b bHHmb . . . bHH2m−2b

HDA 2011, Bonn, June 2011 19

Subspace iteration for the smallest eigenvalue• Choose the Krylov space Km(H, b) as subspace:

Um = Km(H, b) := span{b,Hb, . . . ,Hm−1b}

• This ansatz yields

A = V HmHVm =

bHHb bHH2b . . . bHHmbbHH2b bHH3b . . . bHHm+1b

......

. . ....

bHHmb bHHm+1b . . . bHH2m−1b

B = V HmVm =

bHb bHHb . . . bHHm−1bbHHb bHH3b . . . bHHmb

......

. . ....

bHHm−1b bHHmb . . . bHH2m−2b

HDA 2011, Bonn, June 2011 19

Subspace iteration in terms of MPS• b = bMPS andKm(H, bMPS) = span{bMPS , HbMPS , . . . ,H

m−1bMPS}:

• In principle the same proceeding, BUT

• Computation of the matrix vector products:

HjbMPS =

(M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

M∑k1=1

· · ·M∑

b(k1,...,kj)MPS

Number of summands grows dramatically!• Application of the back-transformation:

Vmymin =m∑j=1

ymin,jHj−1bMPS

does not lie in the MPS space.

HDA 2011, Bonn, June 2011 20

m−1bMPS}:• In principle the same proceeding, BUT

HjbMPS =

(M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

· · ·M∑

b(k1,...,kj)MPS

Vmymin =m∑j=1

ymin,jHj−1bMPS

HDA 2011, Bonn, June 2011 20

HjbMPS =

(M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

M∑k1=1

· · ·M∑

b(k1,...,kj)MPS

Number of summands grows dramatically!

• Application of the back-transformation:

Vmymin =m∑j=1

ymin,jHj−1bMPS

HDA 2011, Bonn, June 2011 20

HjbMPS =

(M∑k=1

αkQ(k)1 ⊗ · · · ⊗Q(k)

M∑k1=1

· · ·M∑

b(k1,...,kj)MPS

Vmymin =

m∑j=1

ymin,jHj−1bMPS

does not lie in the MPS space.K. Waldherr: Approximation Techniques for Special Eigenvalue Problems

HDA 2011, Bonn, June 2011 20

Subspace iteration in terms of MPS• Solution: Application of a projection P onto the MPS space:

M∑k=1

y(k)MPS 7→ argminxMPS

∥∥∥∥∥M∑k=1

y(k)MPS − xMPS

∥∥∥∥∥

• Solve the optimization problem

∥∥∥∥∥∥∑

i1,...,iN

(M∑k=1

(B(i1;k)1 · · ·B(iN ;k)

N )−A(i1)1 · · ·A(iN )

)ei1...iN

∥∥∥∥∥∥• Two possible projection methods:

• SVD-based truncation• ALS-based optimization

• Use ALS to find optimal matrix pair A(0)r , A

• Forcing the derivative to be zero leads to a linear system.

HDA 2011, Bonn, June 2011 21

M∑k=1

∥∥∥∥∥M∑k=1

y(k)MPS − xMPS

∥∥∥∥∥• Solve the optimization problem

∥∥∥∥∥∥∑

i1,...,iN

(M∑k=1

(B(i1;k)1 · · ·B(iN ;k)

N )−A(i1)1 · · ·A(iN )

)ei1...iN

∥∥∥∥∥∥

• Two possible projection methods:• SVD-based truncation• ALS-based optimization

HDA 2011, Bonn, June 2011 21

M∑k=1

∥∥∥∥∥M∑k=1

y(k)MPS − xMPS

∥∥∥∥∥∥∑

i1,...,iN

(M∑k=1

(B(i1;k)1 · · ·B(iN ;k)

N )−A(i1)1 · · ·A(iN )

)ei1...iN

HDA 2011, Bonn, June 2011 21

M∑k=1

∥∥∥∥∥M∑k=1

y(k)MPS − xMPS

∥∥∥∥∥∥∑

i1,...,iN

(M∑k=1

(B(i1;k)1 · · ·B(iN ;k)

N )−A(i1)1 · · ·A(iN )

)ei1...iN

HDA 2011, Bonn, June 2011 21

Numerical resultsIsing-type Hamiltonian, N = 10, dim = 2

1 5 10 15 20 25 30 35 40 45 5010

10−4

10−2

Number of iterations

Full vectorMPS, D=1MPS, D=2MPS, D=3MPS, D=4

HDA 2011, Bonn, June 2011 22

1 5 10 15 20 25 30 35 40 45 5010

10−4

10−2

HDA 2011, Bonn, June 2011 23

5 10 15 20 25 30 35 40 45 50110

10−4

10−2

HDA 2011, Bonn, June 2011 24

1 5 10 1510

10−3

10−2

10−1

MPS, D=2MPS, D=3MPS, D=4MPS, D=5

HDA 2011, Bonn, June 2011 25

Conclusions• Matrix Product States

• data-sparse representation format• represent ground states faithfully• DMRG in terms of MPS

• Subspace methods in terms of MPS• more flexibility (choice of subspace dimension)• approximation not only of the lowest eigenvalue• projection required to keep the number of summands limited• projection can even improve the approximation• convergence to DMRG solution• convergence properties of subspace iteration and

approximation properties of MPS

Thank you for your attention.

HDA 2011, Bonn, June 2011 26

Thank you for your attention.K. Waldherr: Approximation Techniques for Special Eigenvalue Problems

HDA 2011, Bonn, June 2011 26

Approximation Techniques for Special Eigenvalue Problems ... fileApproximation Techniques for...

Documents

Transcript of Approximation Techniques for Special Eigenvalue Problems ... fileApproximation Techniques for...

Analysis of Generalized Eigenvalue Problems - IWR: · PDF fileAnalysis of Generalized Eigenvalue Problems Rike Betten and Eberhard Michel (1/65) Topics A standard eigenvalue problem

Quadratic Eigenvalue Problems - Sandia National …infoserve.sandia.gov/sand_doc/2007/072072.pdf · Quadratic Eigenvalue Problems David M. Day, ... scribe their vibration behaviour.

Classroom Tips and Techniques: Eigenvalue Problems for ...

› woudschoten-conferences › 2017... · Krylov methods for eigenvalue problems: applications2 Krylov methods for eigenvalue problems: applications I Nonlinear eigenvalue problems

Lecture 4. Sturm-Liouville eigenvalue problems

Optimization Approaches Inverse Quadratic Eigenvalue Problems · Inverse Quadratic Eigenvalue Problems Zheng-Jian Bai Department of Information and Computational Mathematics Xiamen

nguyen.hong.hai.free.frnguyen.hong.hai.free.fr/EBOOKS/SCIENCE AND... · Chapter 10 Eigenvalue Problems and Applications 10.1 Introduction Eigenvalue problems occur often in mechanics,

NUMERICAL METHODS FOR LARGE EIGENVALUE PROBLEMS ...

Contour Integral Methods for Nonlinear Eigenvalue Problems ...intranet.math.vt.edu/people/embree/enla_talk.pdfNonlinear Eigenvalue Problems: Stability of Delay Systems Consider the

Modern Numerical Methods for Large Scale Eigenvalue Problems · Introduction Algorithms for Linear Problems Methods for Nonlinear Eigenvalue Problems Overview 1 Introduction 2 Algorithms

Eigenvalue Problems - University of Utah

Structured inverse eigenvalue problems - WWW4 Server - North

Transcendental eigenvalue problems associated with ...

Reduced order modeling of parameter dependent nonlinear eigenvalue bifurcation problems · Reduced order modeling of parameter dependent nonlinear eigenvalue bifurcation problems

Nonlinear eigenvalue-eigenvector problems for STP …pinkus/papers/ev-STP.pdf · ProceedingsoftheRoyalSocietyofEdinburgh,132A, 1307{1331,2002 Nonlinear eigenvalue{eigenvector problems

Nonlinear Eigenvalue Problems: A Challenge for Modern ... · gamm header will be provided by the publisher Nonlinear Eigenvalue Problems: A Challenge for Modern Eigenvalue Methods

Chapter 4 Matrices, Eigenvalue problems

Numerical Solution of Large Nonsymmetric Eigenvalue Problems · · 2013-08-30Numerical Solution of Large Nonsymmetric Eigenvalue Problems Youcef Saad November, ... Numerical Solution

LINEAR PDES AND EIGENVALUE PROBLEMS CORRESPONDING …

Some Global Results for Nonlinear Eigenvalue Problemsjxshix.people.wm.edu/2013-taiwan/rabinowitz-1971-JFA.pdf · Some Global Results for Nonlinear Eigenvalue Problems ... problems