Fast Matrix Multiplication Algorithms
Transcript of Fast Matrix Multiplication Algorithms
![Page 1: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/1.jpg)
Fast Matrix Multiplication
Algorithms
![Page 2: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/2.jpg)
Why should we care?
Complexity of matrix multiplication = Complexity of โalmost allโ matrix problems
Solving linear systems
Evaluating determinants
LU factorization
Many more
P. Bรผrgisser, M. Clausen, M. A.
Shokrollahi
Algebraic complexity theory.
![Page 3: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/3.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
![Page 4: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/4.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
1969: Strassenโs algorithm, ๐(๐2,808) (V. Strassen. Gaussian elimination is not
optimal)
![Page 5: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/5.jpg)
Strassenโs algorithm
๐ = ๐๐ + ๐โ=๐1+๐2
๐ก = ๐๐ + ๐๐=๐3+๐4
๐ = ๐๐ + ๐๐=๐5+๐4 โ๐2 +๐6
๐ข = ๐๐ + ๐โ=๐5+๐1 โ ๐3+๐7
![Page 6: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/6.jpg)
Strassenโs algorithm
1. ๐1 = ๐ด1๐ต1 = ๐(๐ โ โ) = ๐๐ โ ๐โ
2. ๐2 = ๐ด2๐ต2 = (๐ + ๐)โ = ๐โ + ๐โ
3. ๐3 = ๐ด3๐ต3 = (๐ + ๐)๐ = ๐๐ + ๐๐
4. ๐4 = ๐ด4๐ต4 = ๐(๐ โ ๐) = ๐๐ โ ๐๐
5. ๐5 = ๐ด5๐ต5 = ๐ + ๐ ๐ + โ = ๐๐ + ๐โ + ๐๐ + ๐โ
6. ๐6 = ๐ด6๐ต6 = (๐ โ ๐)(๐ + โ) = ๐๐ + ๐โ โ ๐๐ โ ๐โ
7. ๐7 = ๐ด7๐ต7 = (๐ โ ๐)(๐ + ๐) = ๐๐ + ๐๐ โ ๐๐ โ ๐๐
๐ = ๐๐ + ๐โ=๐1+๐2
๐ก = ๐๐ + ๐๐=๐3+๐4
๐ = ๐๐ + ๐๐=๐5+๐4 โ๐2 +๐6
๐ข = ๐๐ + ๐โ=๐5+๐1 โ ๐3+๐7
![Page 7: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/7.jpg)
Strassenโs algorithm
1. ๐1 = ๐ด1๐ต1 = ๐(๐ โ โ) = ๐๐ โ ๐โ
2. ๐2 = ๐ด2๐ต2 = (๐ + ๐)โ = ๐โ + ๐โ
3. ๐3 = ๐ด3๐ต3 = (๐ + ๐)๐ = ๐๐ + ๐๐
4. ๐4 = ๐ด4๐ต4 = ๐(๐ โ ๐) = ๐๐ โ ๐๐
5. ๐5 = ๐ด5๐ต5 = ๐ + ๐ ๐ + โ = ๐๐ + ๐โ + ๐๐ + ๐โ
6. ๐6 = ๐ด6๐ต6 = (๐ โ ๐)(๐ + โ) = ๐๐ + ๐โ โ ๐๐ โ ๐โ
7. ๐7 = ๐ด7๐ต7 = (๐ โ ๐)(๐ + ๐) = ๐๐ + ๐๐ โ ๐๐ โ ๐๐
๐ = ๐๐ + ๐โ=๐1+๐2
๐ก = ๐๐ + ๐๐=๐3+๐4
๐ = ๐๐ + ๐๐=๐5+๐4 โ๐2 +๐6
๐ข = ๐๐ + ๐โ=๐5+๐1 โ ๐3+๐7
T n
= 7Tn
2+ ฮ(n2)
Time
complexity:
๐(๐2.808)
![Page 8: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/8.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
1969: Strassenโs algorithm, ๐(๐2,808) (V. Strassen. Gaussian elimination is not
optimal)
1978: Pan, ๐ < 2.796 (V. Y. Pan. Strassenโs algorithm is not optimal)
![Page 9: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/9.jpg)
Bilinear Algorithms
Given two matrices ๐ด, ๐ต๐๐ =
๐,๐ ๐ข๐๐๐๐ด ๐, ๐ ๐,๐ ๐ฃ๐๐๐๐ต ๐, ๐ , ๐linear combinations.
๐ด๐ต ๐, ๐ =
๐
๐ค๐๐๐๐๐
![Page 10: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/10.jpg)
The minimum number of products r that a bilinear algorithm can use to
compute the product of two ๐ ร ๐ matrices is called the rank of ๐ ร ๐ matrix
multiplication ๐ ( ๐, ๐, ๐ )
The product of two k๐ ร ๐๐ matrices can be viewed as the product of two ๐ รk matrices the entries of which are ๐ ร ๐ matrices
We can create a recursive algorithm ALG for multiplication of ๐ ร k.
View the ๐๐ ร ๐๐ as ๐ ร k matrices with entries ๐๐โ1 ร ๐๐โ1 matrices
The recursive approach using an upper bound of ๐ on ๐ ( ๐, ๐, ๐ ) gives a
bound ๐ < log๐ ๐, (the number of additions that one has to do in each step is
no more than 3๐๐2
As long as ๐ < ๐3 we get a non trivial bound for ๐
Strassen: ๐ = 2, r = 7
Pan: ๐ = 70, ๐ = 143640
![Page 11: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/11.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
1969: Strassenโs algorithm, ๐(๐2,808) (V. Strassen. Gaussian elimination is not
optimal)
1978: Pan, ๐ < 2.796 (V. Y. Pan. Strassenโs algorithm is not optimal)
1979: Bini (border rank), ๐ < 2.78 (D. Bini, M. Capovani, F. Romani, and G.
Lotti. ๐(๐2.7799) complexity for n n approximate matrix multiplication)
![Page 12: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/12.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
1969: Strassenโs algorithm, ๐(๐2,808) (V. Strassen. Gaussian elimination is not
optimal)
1978: Pan, ๐ < 2.796 (V. Y. Pan. Strassenโs algorithm is not optimal)
1979: Bini (border rank), ๐ < 2.78 (D. Bini, M. Capovani, F. Romani, and G.
Lotti. ๐(๐2.7799) complexity for n n approximate matrix multiplication)
1981: Shonhage ฯ-theorem(asymptotic sum inequality), ๐ < 2.548
in the same paper, ๐ < 2.522 (A. Schonhage. Partial and total matrix
multiplication)
![Page 13: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/13.jpg)
Approximate Bilinear Algorithms (ABA)
In bilinear algorithms the coefficients ๐ข๐๐๐, ๐ฃ๐๐๐, ๐ค๐๐๐ were constants.
In ABA these coefficients are linear combinations of the integer powers of a
indeterminate ๐.
The entries of ๐ด๐ต are then only approximately computed: ๐ด๐ต ๐, ๐ = ๐๐ค๐๐๐๐๐ + ๐(๐)
๐(๐): linear combination of positive powers of ๐.
When ๐ โ 0, then the product is almost exactly.
The minimum number of products ๐ for an ABA to compute the product of two ๐ ร ๐ matrices, is called border rank of a matrix multiplication ๐ ( ๐, ๐, ๐ )
![Page 14: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/14.jpg)
Bini showed that when dealing with the asymptotic complexity of matrix
multiplication, approximate algorithms suffice obtaining bounds for ๐
If ๐ ( ๐, ๐, ๐ ) โค ๐, then ๐ โค log๐ ๐
Bini used 10 entry products to multiply a 2 ร 3matrix with a 3 ร 3matrix ๐ = 12,๐ = 1000
Shonhage ฯ-theorem: Suppose we have an upper bound of ๐ on the border
rank of computing ๐ independent instances of matrix multiplication with dimensions ๐๐ ร๐๐ by ๐๐ ร๐๐ for ๐ = 1, โฆ , ๐. Then ๐ < 3๐, where ๐(๐๐๐๐๐๐)
๐= ๐
In particular he showed that one can approximately compute the product of a
3 ร 1 by 1 ร 3 vector and the product of a 1 ร 4 by 4 ร 1 vector together using
only 10 products, whereas any exact bilinear algorithm needs at least 13
products.
![Page 15: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/15.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
1969: Strassenโs algorithm, ๐(๐2,808) (V. Strassen. Gaussian elimination is not
optimal)
1978: Pan, ๐ < 2.796 (V. Y. Pan. Strassenโs algorithm is not optimal)
1979: Bini (border rank), ๐ < 2.78 (D. Bini, M. Capovani, F. Romani, and G.
Lotti. ๐(๐2.7799) complexity for n n approximate matrix multiplication)
1981: Shonhage ฯ-theorem(asymptotic sum inequality), ๐ < 2.548
in the same paper, ๐ < 2.522 (A. Schยจonhage. Partial and total matrix
multiplication)
1982: Romani, ๐ < 2.517 (F. Romani. Some properties of disjoint sums of
tensors related to matrix multiplication.)
![Page 16: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/16.jpg)
A brief historyโฆ
Until the late 1960โs: naรฏve algorithm, ๐(๐3)
1969: Strassenโs algorithm, ๐(๐2,808) (V. Strassen. Gaussian elimination is not
optimal)
1978: Pan, ๐ < 2.796 (V. Y. Pan. Strassenโs algorithm is not optimal)
1979: Bini (border rank), ๐ < 2.78 (D. Bini, M. Capovani, F. Romani, and G.
Lotti. ๐(๐2.7799) complexity for n n approximate matrix multiplication)
1981: Shonhage ฯ-theorem(asymptotic sum inequality), ๐ < 2.548
in the same paper, ๐ < 2.522 (A. Schยจonhage. Partial and total matrix
multiplication)
1982: Romani, ๐ < 2.517 (F. Romani. Some properties of disjoint sums of
tensors related to matrix multiplication.)
1982: Coppersmith & Winograd, ๐ < 2.496 (D. Coppersmith and S.
Winograd. On the asymptotic complexity of matrix multiplication)
![Page 17: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/17.jpg)
Continueโฆ
1986: Strassen, laser method, ๐ < 2.479 (entirely new approach on the
matrix multiplication problem) (V. Strassen. The asymptotic spectrum of
tensors and the exponent of matrix multiplication)
![Page 18: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/18.jpg)
Continueโฆ
1986: Strassen, laser method, ๐ < 2.479 (entirely new approach on the
matrix multiplication problem) (V. Strassen. The asymptotic spectrum of
tensors and the exponent of matrix multiplication)
1989: Coppersmith & Winograd, combine Strassenโs laser method with a
novel from analysis based on large sets avoiding arithmetic progression,
๐ < 2.376 (D. Coppersmith and S. Winograd. Matrix multiplication via
arithmetic progressions.)
![Page 19: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/19.jpg)
Continueโฆ
1986: Strassen, laser method, ๐ < 2.479 (entirely new approach on the
matrix multiplication problem) (V. Strassen. The asymptotic spectrum of
tensors and the exponent of matrix multiplication)
1989: Coppersmith & Winograd, combine Strassenโs laser method with a
novel from analysis based on large sets avoiding arithmetic progression,
๐ < 2.376 (D. Coppersmith and S. Winograd. Matrix multiplication via
arithmetic progressions.)
2003: Cohn & Umans: group theoretic framework for designing and
analyzing matrix multiplication algorithms
![Page 20: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/20.jpg)
Continueโฆ
1986: Strassen, laser method, ๐ < 2.479 (entirely new approach on the
matrix multiplication problem) (V. Strassen. The asymptotic spectrum of
tensors and the exponent of matrix multiplication)
1989: Coppersmith & Winograd, combine Strassenโs laser method with a
novel from analysis based on large sets avoiding arithmetic progression,
๐ < 2.376 (D. Coppersmith and S. Winograd. Matrix multiplication via
arithmetic progressions.)
2003: Cohn & Umans: group theoretic framework for designing and
analyzing matrix multiplication algorithms
2005: Cohn, Umans, Kleinberg, Szegedy, ๐(๐2.41) (H. Cohn, R. Kleinberg, B.
Szegedy, and C. Umans. Group-theoretic algorithms for matrix
multiplication)
Conjectures that can lead to: ๐ = 2.
![Page 21: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/21.jpg)
Continueโฆ
1986: Strassen, laser method, ๐ < 2.479 (entirely new approach on the
matrix multiplication problem) (V. Strassen. The asymptotic spectrum of
tensors and the exponent of matrix multiplication)
1989: Coppersmith & Winograd, combine Strassenโs laser method with a
novel from analysis based on large sets avoiding arithmetic progression,
๐ < 2.376 (D. Coppersmith and S. Winograd. Matrix multiplication via
arithmetic progressions.)
2003: Cohn & Umans: group theoretic framework for designing and
analyzing matrix multiplication algorithms
2005: Cohn, Umans, Kleinberg, Szegedy, ๐(๐2.41) (H. Cohn, R. Kleinberg, B.
Szegedy, and C. Umans. Group-theoretic algorithms for matrix
multiplication)
Conjectures that can lead to: ๐ = 2.
2014 Williams: ๐ < 2.373 (Multiplying matrices in ๐ ๐2.373 time)
![Page 22: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/22.jpg)
Continueโฆ
For some ๐ they provide a way to
multiply ๐ ร ๐ matrices using ๐ โช ๐3
multiplications and apply the technique
recursively to show that ๐ < log๐๐
![Page 23: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/23.jpg)
Basic group theory definitions
Group
A group is a non empty set ๐บ with a binary operation โ defined on ๐บ such that the
following conditions hold:
1. For all ๐, ๐, ๐ โ ๐บ, we have ๐ โ ๐ โ ๐ = (๐ โ ๐) โ ๐
2. There exists an element 1โ ๐บ such that 1 โ ๐ = ๐ and ๐ โ 1 = ๐ for all ๐ โ ๐บ
3. For all ๐ โ ๐บ there exists an element ๐โ1 โ ๐บ such that ๐ โ ๐โ1 = 1 and๐โ1 โ ๐ = 1
Order of a group
The order ๐บ of a group is its cardinality, i.e. the number of elements in its set.
![Page 24: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/24.jpg)
Cyclic Group
A group is said to be cyclic if it is generated by a single element.
(We say that ๐ generates ๐บ if ๐บ = ๐ if every element of ๐บ can be written as a finite
product of elements from ๐ and their inverses. Note that the order of an element ๐of a group is the order of the subgroup ๐ it generates)
Abelian Group
The group ๐บ is said to be abelian if ๐ โ ๐ = ๐ โ ๐ far all ๐, ๐ โ ๐บ.
![Page 25: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/25.jpg)
Group Algebra
The group algebra ๐น ๐บ of ๐บ is defined to be the ๐น-vector space with basis the
elements of ๐บ endowed with the multiplication extending that on ๐บ. Thus:
1. An element of ๐น ๐บ is a sum ๐โ๐บ ๐๐๐, ๐๐โ๐น
2. Two elements ๐โ๐บ ๐๐๐, ๐โ๐บ ๐๐โฒ๐ of ๐น ๐บ are equal if and only if ๐๐ = ๐๐
โฒ for all ๐
3. ๐โ๐บ ๐๐๐ ๐โ๐บ ๐๐โฒ๐ = ๐โ๐บ ๐๐
โฒโฒ๐ , ๐๐โฒโฒ = ๐1๐2 = ๐๐๐๐๐
โฒ
Homomorphism
A homomorphism for a group ๐บ to a group ๐บโฒ is a map ๐ถ: ๐บ โ ๐บโฒ such that ๐ถ ๐๐ =๐ถ(๐)๐ถ(๐) for all ๐, ๐ โ ๐บ. An isomorphism is a bijective homomorphism.
![Page 26: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/26.jpg)
Multiplying polynomials via FFT
Standard method requires time complexity of ๐(๐2)
We think of the coefficient vectors of the polynomials as elements of the
group algebra โ ๐บ of a finite group ๐บ
If the group is large (order at least 2๐), convolution of two vectors in the
group algebra corresponds to the polynomial product.
![Page 27: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/27.jpg)
Multiplying polynomials via FFT
Standard method requires time complexity of ๐(๐2)
We think of the coefficient vectors of the polynomials as elements of the
group algebra โ ๐บ of a finite group ๐บ
If the group is large (order at least 2๐), convolution of two vectors in the
group algebra corresponds to the polynomial product.
Discrete convolution
Suppose we have two complex vectors in ๐ธ๐:๐ = ๐ง0 ๐ง1 โฏ ๐ง๐โ1
๐ ๐ = ๐ฆ0 ๐ฆ โฏ ๐ฆ๐โ1๐
The discrete convolution of these two vectors is another vector, which we
denote ๐ โ ๐, defined componentwise by (๐ โ ๐)๐= ๐=0๐โ1 ๐ง๐โ๐๐ฆ๐ , ๐ = 0,1,2, โฆ
![Page 28: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/28.jpg)
Multiplying polynomials via FFT
Standard method requires time complexity of ๐(๐2)
We think of the coefficient vectors of the polynomials as elements of the
group algebra โ ๐บ of a finite group ๐บ
If the group is large (order at least 2๐), convolution of two vectors in the
group algebra corresponds to the polynomial product.
Convolution in the group algebra can be computed quickly using the FFT.
Time complexity of FFT and inverse FFT: ๐(๐๐๐๐๐)
![Page 29: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/29.jpg)
Discrete Fourier Transform for
polynomials
Discrete Fourier Transform
Embed polynomials as elements of the group algebra โ[๐บ]:
Let ๐บ = ๐ง be a cyclic group of order ๐ โฅ 2๐. Define
๐ด = ๐=0๐โ1๐๐๐ง
๐ ๐ต = ๐=0๐โ1๐๐๐ง
๐
Discrete Fourier Transform is an invertible linear transformation
๐ท: โ[๐บ] โ โ ๐บ , such that
๐ท ๐ด = ( ๐=0๐โ1๐๐๐ฅ0
๐ , ๐=0๐โ1๐๐๐ฅ1
๐ , โฆ , ๐=0๐โ1๐๐๐ฅ๐โ1
๐ , ), ๐ฅ๐ = ๐2๐๐
๐๐
Then ๐ด ๐ต = ๐ทโ1(๐ท( ๐ด)๐ท( ๐ต))
![Page 30: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/30.jpg)
Embedding matrices A, B into elements ๐ด, ๐ตof the group algebra โ[๐บ]
Cohn & Umans
Matrix multiplication can be embedded into the group algebra of a finite
group ๐บ (๐บ must be non-abelian)
Let ๐น be a field and ๐, ๐ and ๐ be subsets of ๐บ.
๐ด = (๐๐ ,๐ก)๐ โ๐,๐กโ๐ and B = (๐๐ก,๐ข)๐กโ๐,๐ขโ๐
are ๐ ร ๐ and ๐ ร ๐ , indexed by elements of ๐, ๐ and ๐, ๐ respectively.
Then embed ๐ด, ๐ต as elements ๐ด, ๐ต โ ๐น ๐บ :
๐ด = ๐ โ๐,๐กโ๐ ๐๐ ,๐ก๐ โ1๐ก and ๐ต = ๐กโ๐,๐ขโ๐ ๐๐ก,๐ข๐ก
โ1๐ข
![Page 31: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/31.jpg)
Using the FFT
As with the polynomial method, the Fourier transform provides an efficient way
to compute the convolution product.
For a non-abelian group a fundamental theorem of Weddeburn says that the
group algebra is isomorphic, via a Fourier transform, to an algebra of block
diagonal matrices having block dimensions ๐1โฆ๐๐, with ๐๐2 = ๐บ .
Convolution in โ ๐บ is thus transformed into block diagonal matrix
multiplication.
![Page 32: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/32.jpg)
Embed matrices A,
B into elements ๐ด, ๐ต of the group
algebra โ[๐บ]
Multiplication of ๐ดand ๐ต in the
group algebra is carried out in the Fourier domain
after performing the Discrete
Fourier Transform (DFT) of ๐ด and ๐ต
FFT
The product ๐ด ๐ต is found
by performing the inverse
DFT
Inverse FFT
Entries of the matrix
AB can be read off from
the group algebra product ๐ด ๐ต
![Page 33: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/33.jpg)
Triple Product Property
The approach works only if the group ๐บ admits an embedding of matrix
multiplication into its group algebra.
The coefficients of the convolution product correspond to the entries of the
product matrix.
Such an embedding is possible whenever the group ๐บ has three subgroups,
๐ป1, ๐ป2, ๐ป3 with the property that whenever โ1๐ ๐ป1, โ2๐ ๐ป2 and โ3๐ ๐ป3 with
โ1โ2โ3 = 1, then โ1 = โ2 = โ3 = 1
(The condition can be generalized to subsets of ๐บ rather than subgroups)
![Page 34: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/34.jpg)
Beating the sum of cubes
In order for ๐ to be less than 3, the group must satisfy more conditions.
In particular, it must be the case that:
๐ป1 ๐ป2 ๐ป3 > ๐๐3,
๐๐: the block dimensions of the block matrices
![Page 35: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/35.jpg)
Group Theory Definitions
Permutation Groups
Let ๐ be a set and let ๐๐ฆ๐ ๐ be the set of bijections ๐: ๐ โ ๐
๐๐ฆ๐ ๐ is a group, called the group of symmetries of ๐. For example, the permutation group on n letters ๐๐ is defined to be the group of symmetries of the set 1,โฆ , ๐ โ it has order ๐!.
Groups Acting on Sets
Let ๐ be a set and let ๐บ be a group. A left action of ๐บ on ๐ is a mapping ๐, ๐ฅ โผ๐๐ฅ: ๐บ ร ๐ โ ๐ such that
a) 1๐ฅ = ๐ฅ, for all ๐ฅ โ ๐
b) (๐1๐2)๐ฅ = ๐1(๐2๐ฅ), all ๐1, ๐2 โ ๐บ, ๐ฅ โ ๐
A set together with a (left) action of ๐บ is called a (left) ๐บ-set. An action is trivial if ๐๐ฅ =๐ฅ for all ๐ โ ๐บ
![Page 36: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/36.jpg)
Direct product
When ๐บ and ๐ป are groups, we can construct a new group ๐บ ร ๐ป, called the
(direct) product of ๐บ and ๐ป. As a set, it is the Cartesian product of ๐บ and ๐ป, and
multiplication is defined by: ๐, โ ๐โฒ, โโฒ = (๐๐โฒ, โโโฒ)
Normal subgroups
A subgroup ๐ of ๐บ is normal, denoted ๐ โฒ ๐บ, if ๐๐๐โ1 = ๐ for all ๐ โ ๐บ
Semidirect product
A group ๐บ ๐s a semidirect product of its subgroups ๐ and ๐ if ๐ is normal and the
homomorphism ๐บ โ ๐บ ๐ induces an isomorphism ๐ โ ๐บ ๐.
We write ๐บ = ๐ โ ๐.
![Page 37: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/37.jpg)
Wreath Product
The wreath product of two groups ๐ด and ๐ต is constructed in the following way.
Let ๐ด๐ต be the set of all functions defined on ๐ต with values in ๐ด.
With respect to the componentwise multiplication, this set is a group which is the
complete direct product of ๐ต copies of ๐ด.
The semidirect product ๐ of ๐ต and ๐ด๐ต is called the Cartesian wreath product of
๐ด and ๐ต, and is denoted by ๐ด๐๐ ๐ต.
If instead of ๐ด๐ต one takes the smaller group ๐ด(๐ต) consisting of all functions with
finite support, that is, functions taking only non-identity values on a finite set of
points, then one obtains a subgroup of๐ called the wreath product of ๐ด and ๐ตand is denoted by ๐ด ๐ค๐ ๐ต.
![Page 38: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/38.jpg)
Beating the sum of cubes, finallyโฆ
The elusive group ๐บ that managed to โbeat the sum of cubesโ turned out to
be a wreath product of:
Abelian group of order 173
Symmetric group of order 2
![Page 39: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/39.jpg)
Symmetric group of order 2
Wreath Product
Abelian group of order 173
๐ ๐2.91
![Page 40: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/40.jpg)
Beating the sum of cubes, finallyโฆ
The elusive group ๐บ that managed to โbeat
the sum of cubesโ turned out to be a wreath
product of:
Abelian group of order 173
Symmetric group of order 2
Elementary fact of group representation theory
The index of the largest abelian subgroup of a
group is an upper bound on the size of the
maximum block of the block diagonal matrix
representation given by Wedderburnโs
theorem.
For a non-abelian group a
fundamental theorem of
Weddeburn says that the
group algebra is isomorphic,
via a Fourier transform, to an
algebra of block diagonal
matrices having block
dimensions ๐1โฆ๐๐, with ๐๐2 = ๐บ .
![Page 41: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/41.jpg)
Improving the bounds for ๐
Szegedy realized that some of the combinatory structures of the 1987
Coppersmith - Winograd paper could be used to select the three subsets in
the wreath product groups in amore sophisticated way.
The researchers managed to achieve exponential bound: ๐ < 2.48
The researchers distilled their insights into two conjectures, one that has an
algebraic flavor and one that has a combinatorial.
![Page 42: Fast Matrix Multiplication Algorithms](https://reader031.fdocuments.net/reader031/viewer/2022012104/616a18d111a7b741a34ec3e3/html5/thumbnails/42.jpg)
A 6ร6 strong USP, along with 2 of its 18
pieces