Fanny Dufoss e & Bora U˘carperso.ens-lyon.fr/bora.ucar/CR12/lecture-sparseMatrices.pdf · Sparse...
Transcript of Fanny Dufoss e & Bora U˘carperso.ens-lyon.fr/bora.ucar/CR12/lecture-sparseMatrices.pdf · Sparse...
Sparse matrices and graphs (Basics)
Fanny Dufosse & Bora Ucar
Inria, Grenoble Rhone-Alpes & CNRS and ENS Lyon, France
CR12: Combinatorial scientific computing, September 2018http://perso.ens-lyon.fr/bora.ucar/CR12/
1/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Outline
1 Course presentation
2 Sparse matrices
3 Graphs: Definitions and some problems
2/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
CR12: What is and what is not?
CR12 is about sparse matrices andcomputations with them.
Sparse matrix: It is a matrix withmany zeros which are not stored.
Sparse matrix computations: Mostlyoperate on the nonzero elements.
Sparse matrices are everywhere:Google’s PageRank, network analysis(social, biological), simulation science(circuits, numerical,...).
CR08: What is expected?
We will follow: Lecture notes, articles, books. Solve problems.Presentations (survey/research).
A =
26666664
4 �1 0 0 0 �1�1 3 �1 �1 0 0
0 �1 3 0 �1 00 �1 0 2 0 00 0 �1 0 2 �1�1 0 0 0 �1 2
37777775
4 CR08
2
1
6
5
4
3
3/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
CR12: What is and what is not?
CR12 is about graphs.
Nonzeros of a matrix can been seen asthe edges of a bipartite graph, adirected graph, an undirected graph, ora hypergraph.
Computations on sparse matrices canbe modeled using graphs.
CR12 is about this correspondence in abroader sense.
2
1
6
5
4
3
1
26
4
5 3
4/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
CR12: What is and what is not?
...not a scientific computing/linear algebra course.
...not a full-fledged combinatorial optimization course.
...not a parallel computing course.
We will see some exciting problems having to do withtopics from these three domains.
5/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
CR12: A sample problem
Let A be an n × n symmetric positive definite matrix
The LLT factorization can be described by the equation:
A = H0 =
(d1 vT
1
v1 H1
)
=
( √d1 0
v1√d1
In−1
)(1 00 H1
)( √d1
vT1√d1
0 In−1
)
= L1A1LT1 , where
H1 = H1 −v1vT
1
d1
The basic step is applied on H1,H2 . . . to obtain :
A = (L1L2 · · ·Ln−1) In(LTn−1 · · ·LT
2 LT1
)= LLT
6/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
The basic step: H1 = H1 − v1vT1
d1
What is v1vT1 in terms of structure?
v1
v1
T
v1v1
T
v1vT1 : a dense sub-matrix in H1.
v1 is a column of A, hence the neigh-bors of the corresponding vertex.
The neighbors of the correspondingvertex form a clique.
Basic step modeled using a graph: Pick the vertex, form a clique from itsneighbors, and remove the vertex.
7/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
CR12: A sample problem
4
1
6
5
4
3
2
1
26
4
35
26
4
35 6
5
4
3
2
6
4
356
5
4
3
6
4
5
G0
G1
G2
G3
H0
H1
H2
H3
6
5
2
1
26
4
35
1
6
5
4
3
2
1
26
4
35
1
6
5
4
3
G (L + LT ) L + LT
What can we say about thefinal matrix by just lookingat the original graph?
8/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
CR12: Planning
Instructor: Lecture notes (references to articles, books); problems.
Students: Participation to solving problems;
Exam or presentations (can be a survey, of typeexpository, or about a research problem). We will decidethe form according to the number of participants.
Dates: Sep: 10, 17, 24; Oct: 1, 8, 15, 22; Nov: 5, 19; Dec: 3, 10, 17.
Office: M7.213 (LIP)–BoraContact: [email protected]
Course web: http://perso.ens-lyon.fr/bora.ucar/CR12/
Lecture notes will be posted.
9/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Outline
1 Course presentation
2 Sparse matrices
3 Graphs: Definitions and some problems
10/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices
Harry Markowitz: “I got aNobel Prize...”“I coined the term “Sparsematrix”...”Nobel in 1990 (work ’52).
When we started putting these models togetherthey became very large as compared to linearprogramming capability at that time... It struckme that our matrices were mostly full of zeros,and if you have a set of simultaneous equationsthat are mostly zeros, if you pick your pivotsright, you could just solve it by hand. Then Ithought, well, maybe we could get thecomputer to do the same thing. This led to“Sparse Matrices.”As far as I know, I coinedthe word “Sparse Matrix.”
I published [in this area, refers to a 1957 paper
on selecting pivots for Gaussian elimination]
and then forgot about it...[In a] decade, it
started to catch on and expanded very fast. I
was completely oblivious to what was going on
and to what extent.
11/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Coordinate format
There are many ways to store a sparse matrix.
We will look at three standard representations which store only the nonzeroentries. We use τ to denote the number of nnzs.
1.1 0.0 0.0 0.0 0.00.0 2.2 0.0 2.4 0.03.1 0.0 3.3 0.0 0.00.0 0.0 0.0 4.4 0.00.0 5.2 0.0 5.4 5.5
Coordinate (COO) formatMatrixMarket format
Two integer arrays (irn, jcn) and a doublearray A, each of size τ :
irn = [ 1 2 2 3 3 4 5 5 5 ]
jcn = [ 1 2 4 1 3 4 2 4 5 ]
A = [1.1 2.2 2.4 3.1 3.3 4.4 5.2 5.4 5.5 ]
The kth entry aij is stored as irn[k] = i , jcn[k] = j , a[k] = aij .
The storage is 2τ integers and τ doubles (or single or complex). In general,τ = O(m + n).
Remark: Nonzeros can be stored in any order.
12/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Coordinate format
Question (Class): Let A be an m × n matrix. R and C be two diagonalmatrices of size m ×m and n × n. A can be scaled as A = RAC.
Write a function/routine that does this operation for matrices stored inthe coordinate format. R and C are stored as arrays r [1, . . . ,m] andc[1, . . . , n]. Create new arrays for A.
13/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Compressed storage by rows
There are many ways to store a sparse matrix.
We will look at three standard representations which store only thenonzero entries.
1.1 0.0 0.0 0.0 0.00.0 2.2 0.0 2.4 0.03.1 0.0 3.3 0.0 0.00.0 0.0 0.0 4.4 0.00.0 5.2 0.0 5.4 5.5
Compressed storage by rows (CSR)
Two integer arrays (ia, jcn) and adouble array A:
ia = [ 1 2 4 6 7 10 ]
jcn = [ 1 2 4 1 3 4 2 4 5 ]
A = [1.1 2.2 2.4 3.1 3.3 4.4 5.2 5.4 5.5 ]
The nonzeros of the ith row are stored at theia[i]· · · ia[i+1]-1 positions of jcn and A .
For example the 3rd row: starts at ia[3] = 4 and finishes at
ia[3+1]-1 = 5 . The column indices are therefore jcn[4,5] = 1 3
and values are A[4,5] = 3.1 3.3 .
14/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Compressed storage by rows
There are many ways to store a sparse matrix.
We will look at three standard representations which store only the nonzeroentries.
1.1 0.0 0.0 0.0 0.00.0 2.2 0.0 2.4 0.03.1 0.0 3.3 0.0 0.00.0 0.0 0.0 4.4 0.00.0 5.2 0.0 5.4 5.5
Compressed storage by rows
Two integer arrays (ia, jcn) and a doublearray A:
ia = [ 1 2 4 6 7 10 ]
jcn = [ 1 2 4 1 3 4 2 4 5 ]
A = [1.1 2.2 2.4 3.1 3.3 4.4 5.2 5.4 5.5 ]
The nonzeros of the ith row are stored at the ia[i]· · · ia[i+1]-1 positions
of jcn and A .
Let matrix be of size m× n, and τ be the number of nonzeros, then the storageis m + 1 + τ integer and τ double (or single or complex).
Remark: Nonzeros in a row can be stored in any order.
15/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Compressed storage by rows
Question: Let A be an m × n matrix stored in the compressed storage byrows (CSR) format. What is the number of nonzeros?
Question: Let A be an m × n matrix stored in the CSR format. Write afunction/routine that displays A.
Let A be an m × n matrix. R and C be two diagonal matrices of sizem ×m and n × n. A can be scaled as A = RAC.
Question: Write a function/routine that does this operation for matricesstored in the CSR format.
16/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Compressed storage by columnsThere are many ways to store a sparse matrix.
We will look at three standard representations which store only thenonzero entries.
1.1 0.0 0.0 0.0 0.00.0 2.2 0.0 2.4 0.03.1 0.0 3.3 0.0 0.00.0 0.0 0.0 4.4 0.00.0 5.2 0.0 5.4 5.5
Compressed storage by columns (CSC)
Two integer arrays (irn, ja) and adouble array A:
ja = [ 1 3 5 6 9 10 ]
irn = [ 1 3 2 5 3 2 4 5 5 ]
A = [1.1 3.1 2.2 5.2 3.3 2.4 4.4 5.4 5.5 ]
The nonzeros of the jth column are stored at the ja[j]· · · ja[j+1]-1positions of irn and A .
For example the 2nd col: starts at ja[2] = 3 and finishes at
ja[2+1]-1 = 4 . The row indices are therefore irn[3,4] = 2 5 and
values are A[3,4] = 2.2 5.2 .
17/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Compressed storage by columns
There are many ways to store a sparse matrix.
We will look at three standard representations which store only the nonzeroentries.
1.1 0.0 0.0 0.0 0.00.0 2.2 0.0 2.4 0.03.1 0.0 3.3 0.0 0.00.0 0.0 0.0 4.4 0.00.0 5.2 0.0 5.4 5.5
Compressed storage by columns
Two integer arrays (irn, ja) and a doublearray A:
ja = [ 1 3 5 6 9 10 ]
irn = [ 1 3 2 5 3 2 4 5 5 ]
A = [1.1 3.1 2.2 5.2 3.3 2.4 4.4 5.4 5.5 ]
The nonzeros of the jth column are stored at theja[j]· · · ja[j+1]-1 positions of irn and A .
Let matrix be of size m× n, and τ be the number of nonzeros, then the storageis n + 1 + τ integer and τ double (or single or complex).
Remark: Nonzeros in a column can be stored in any order.
18/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Compressed storage by columns
Question (Class): Let A be an m × n matrix stored in the compressedstorage by columns format. What is the number of nonzeros?
Question: Let A be an m × n matrix stored in the coordinate format.Write a function/routine that creates A in csc format.
19/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Reminder: Dense matrix vector multiplication
Need to compute y ← Ax for an m × n dense/ matrix A and suitabledense vectors y and x .
Row-major order
for i = 1 to m doy [i ]← 0.0for j = 1 to n do
y [i ]← y [i ] + A[i , j ]× x [j ]
Column-major order
for i = 1 to m doy [i ]← 0.0
for j = 1 to n dofor i = 1 to m do
y [i ]← y [i ] + A[i , j ]× x [j ]
20/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Need to compute y ← Ax for an m × n sparse matrix A and suitabledense vectors y and x .
Algorithm when A is stored in the coordinate format.
21/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Need to compute y ← Ax for an m × n sparse matrix A and suitabledense vectors y and x .
Algorithm when A is stored in the coordinate format.
Coordinate format with τ nonzeros (irn, jcn, A)
for i = 1 to m doy [i ]← 0.0
for k = 1 to τ doy [irn[k]]← y [irn[k]] + A[k]× x [jcn[k]]
22/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Need to compute y ← Ax for an m × n sparse matrix A and suitabledense vectors y and x .
Algorithm when A is stored in the CSR format.
23/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Need to compute y ← Ax for an m × n sparse matrix A and suitabledense vectors y and x .
Algorithm when A is stored in the CSR format.
Compressed storage by rows(ia, jcn, A)
for i = 1 to m doval ← 0.0for k = ia[i ] to ia[i + 1]− 1 do
val ← val + A[k]× x [jcn[k]]y [i ]← val
24/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Need to compute y ← Ax for an m × n sparse matrix A and suitabledense vectors y and x .
Algorithm when A is stored in the CSC format.
25/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Need to compute y ← Ax for an m × n sparse matrix A and suitabledense vectors y and x .
Algorithm when A is stored in the CSC format.
Compressed storage by columns(ja, irn, A)
for i = 1 to m doy [i ]← 0.0
for j = 1 to n doxval ← x [j ]for k = ja[j ] to ja[j + 1]− 1 do
y [irn[k]]← y [irn[k]] + A[k]× xval
26/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
Characterizes a wide range of applications with irregularcomputational dependency.
reduction operation from inputs (here entries of x) to outputs (hereentries of y), e.g., get the max in a set, for many different sets. How?
A fine grain computation: each nnz is read/operated on once.Guaranteeing efficiency will guarantee efficiency in applications witha coarser grain computation.
27/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Sparse matrices: Sparse matrix vector multiplies
SpMxV’s of the form y ← Ax are the computational kernel of manyscientific computations
Solvers for linear systems, linear programs, eigensystems, leastsquares problems,
Repeated SpMxV with the same large sparse matrix A,
The matrix A can be symmetric, unsymmetric, rectangular,
Sometimes multiplies are of the form y ← ADAT z with a diagonalD (in interior point methods for linear programs).
computation proceeds (why?) as w ← AT z , then x ← Dw , theny ← Ax
Sometimes we have multiplies with A and AT independent; y ← Axand w ← AT z (QMR, CGNE, and CGNR methods with squareunsymmetric A; rectangular A in Lanczos method).
28/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Further questions
Given a real matrix A of size m × n, with τ nonzeros.
Question: Write a function to create its transpose (three times: stored inthe COO, CSR, and CSC formats).
Question (Class): Write a function to determine if A is symmetric.Assume A is stored in the CSC format.
29/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Outline
1 Course presentation
2 Sparse matrices
3 Graphs: Definitions and some problems
30/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Graph notations and definitions
A graph G = (V ,E ) consists of a finite set V , called the vertex set and afinite, binary relation E on V , called the edge set.
Three standard graph models
Undirected graph: The edges are unordered pair of vertices, i.e.,{u, v} ∈ E for some u, v ∈ V .
Directed graph: The edges are ordered pair of vertices, that is, (u, v) and(v , u) are two different edges.
Bipartite graph: G = (U ∪ V ,E ) consists of two disjoint vertex sets Uand V such that for each edge (u, v) ∈ E , u ∈ U and v ∈ V .
An ordering or labelling of G = (V ,E ) having n vertices, i.e., |V | = n, isa mapping of V onto 1, 2, . . . , n.
31/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Matrices and graphs: Rectangular matrices
The rows/columns and nonzeros of a given sparse matrix correspond (withnatural labelling) to the vertices and edges, respectively, of a graph.
Rectangular matrices
A =
1 2 3 4
1 × ×2 × ×3 × ×
Bipartite graph
1
r2
r3
1c
2c
3c
4c
r
The set of rows corresponds to R, the set of columns corresponds to C such
that for each aij 6= 0, (ri , cj) is an edge.
32/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Matrices and graphs: Square unsymmetric pattern
The rows/columns and nonzeros of a given sparse matrix correspond (withnatural labelling) to the vertices and edges, respectively, of a graph.
Square unsymmetricpattern matrices
A =
1 2 3
1 × ×2 × ×3 ×
Graph models
Bipartite graph as before.
Directed graph
v2 3v
1v
The set of rows/cols corresponds the vertex set V such that for each aij 6= 0,
(vi , vj) is an edge. Transposed view possible too, i.e., the edge (vi , vj) directed
from column i to row j . Usually self-loops are omitted.
33/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Matrices and graphs: Square unsymmetric pattern
A special subclass
Directed acyclic graphs (DAG):A directed graphs with no loops(maybe except for self-loops).
DAGs
We can sort the vertices such thatif (u, v) is an edge, then u appearsbefore v in the ordering.
Question (Class): What kind of square matrices have a DAG?
34/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Matrices and graphs: Symmetric pattern
The rows/columns and nonzeros of a given sparse matrix correspond (withnatural labelling) to the vertices and edges, respectively, of a graph.
Square symmetricpattern matrices
A =
1 2 3
1 ×2 × × ×3 × ×
Graph models
Bipartite and directed graphs as before.
Undirected graph
v2 3v
1v
The set of rows/cols corresponds the vertex set V such that for each
aij , aji 6= 0, {vi , vj} is an edge. No self-loops; usually the main diagonal is
assumed to be zero-free.
35/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Edges, degrees, and paths
Many definitions for directed and undirected graphs are the same. Wewill use (u, v) to refer to an edge of an undirected or directed graph toavoid repeated definitions.
An edge (u, v) is said to incident on the vertices u and v .
For any vertex u, the set of vertices in adj(u) = {v : (u, v) ∈ E} arecalled the neighbors of u. The vertices in adj(u) are said to beadjacent to u.
The degree of a vertex is the number of edges incident on it.
A path p of length k is a sequence of vertices 〈v0, v1, . . . , vk〉 where(vi−1, vi ) ∈ E for i = 1, . . . , k . The two end points v0 and vk aresaid to be connected by the path p, and the vertex vk is said to bereachable from v0.
Simple path: has no vertex repetitions.
36/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Components
An undirected graph is said to be connected if every pair of verticesis connected by a path.
The connected components of an undirected graph are theequivalence classes of vertices under the “is reachable” fromrelation.
A directed graph is said to be strongly connected if every pair ofvertices are reachable from each other.
The strongly connected components of a directed graph are theequivalence classes of vertices under the “are mutually reachable”relation.
37/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Trees and spanning trees
A tree is a connected, acyclic, undirected graph. If an undirected graph isacyclic but disconnected, then it is a forest.
Properties of trees
Any two vertices are connected by a unique path.
|E | = |V | − 1
A rooted tree is a tree with a distinguished vertex r , called the root.
There is a unique path from the root r to every other vertex v . Anyvertex y in that path is called an ancestor of v . If y is an ancestor of v ,then v is a descendant of y .
The subtree rooted at v is the tree induced by the descendants of v ,rooted at v .
A spanning tree of a connected graph G = (V ,E ) is a tree T = (V ,F ),such that F ⊆ E .
38/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Ordering of the vertices of a rooted tree
A topological ordering of a rooted tree is an ordering that numberschildren vertices before their parent.
A postorder is a topological ordering which numbers the vertices inany subtree consecutively.
u w
x
y
z
v
with topological ordering
1 3
2
4
5
6
Rooted spanning tree
w
yz
xu
v
Connected graph G
u w
x
y
z
v
1
6
54
3
2
Rooted spanning treewith postordering
39/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Postordering the vertices of a rooted tree
[porder ]=PostOrder(T , r)
porder ← [·]seen(v)← False for all v ∈ Tseen(r)← TruePush(S , r)while NotEmpty(S) do
v ←Top(S)if ∃ a child c of v with seen(c) = False thenseen(c)← TruePush(S , c)
elsePop(S)porder ← [porder , v ]
Again, have to run for each root, if T is a forest.
The algorithm runs in O(n) time for a tree with n nodes.
40/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Permutation matrices
A permutation matrix is a square (0, 1)-matrix where each row andcolumn has a single 1.
If P is a permutation matrix, PPT = I, i.e., it is an orthogonal matrix.Let,
A =
1 2 3
1 × ×2 × ×3 × ×
and suppose we want to permute columns as [2, 3, 1]. Define p2,1 = 1,p3,2 = 1, p1,3 = 1, and B = AP (if column j to be at position i , setpji = 1)
B =
2 3 1
1 × ×2 × ×3 × ×
=
1 2 3
1 × ×2 × ×3 × ×
1 2 3
1 12 13 1
41/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Permutation matrices
Question: Given a matrix A, we want to permute the rows according to agiven permutation.
Let π(i) denote the new position of row i . Write a simple for-loop tocreate the permutation matrix P so that the matrix PA is permutedaccording to π.
Verify your solution on the matrix in the previous slide.
42/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Permutation matrices
Question (Class): For P and Q permutation matrices, what can be saidabout their multiplication PQ?
Question: Let A be a square matrix and P be a permutation matrix.What is PAPT ?
Question: Let A be a square matrix and P be a permutation matrix.What can you say about the diagonal elements of PAPT ?
Question (Class): Let A be a square matrix and P be a permutationmatrix. What can you say about the graphs of A and PAPT ?
43/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Matching in bipartite graphs and permutations
A matching in a graph is a set of edges no two of which share a commonvertex. We will be mostly dealing with matchings in bipartite graphs.
In matrix terms, a matching in the bipartite graph of a matrixcorresponds to a set of nonzero entries no two of which are in the samerow or column.
A vertex is said to be matched if there is an edge in the matchingincident on the vertex, and to be unmatched otherwise. In a perfectmatching, all vertices are matched.
The cardinality of a matching is the number of edges in it. A maximumcardinality matching or a maximum matching is a matching of maximumcardinality. Solvable in polynomial time.
44/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Matching in bipartite graphs and permutations
Given a square matrix whose bipartite graph has a perfect matching, sucha matching can be used to permute the matrix to have the matchingentries are along the main diagonal.
1
r2
r3 3
c
2c
1cr
1 2 3
1 × •2 × •3 • ×
2 3 1
1 • ×2 • ×3 × •
=
1 2 3
1 × •2 × •3 • ×
1 2 3
1 12 13 1
45/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Perfect matchings and zero-free diagonals
Question: Let A be an n × n sparse matrix having a perfect matching inits bipartite graph. Let M be such a matching, where colmatch(j)=i
show that the column j is matched to row i in M.
Write a pseudocode to create a permutation matrix Q such that AQ hasthe matching entries along its diagonal.
Write a pseudocode to create a permutation matrix P such that PA hasthe matching entries along its diagonal.
46/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Reducibility
Reducible matrix: An n× n square matrix A is reducible if there exists ann × n permutation matrix P such that
PAPT =
(A11 A12
O A22
),
where A11 is an r × r submatrix, A22 is an (n − r)× (n − r) submatrix,where 1 ≤ r < n.
Irreducible matrix: There is no such a permutation matrix.
Theorem: An n × n square matrix is irreducible iff its directed graph isstrongly connected.
Proof: Follows by definition.
47/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Why care irreducibility
Matrix IterativeAnalysis, by Richard S.Varga, Second (revisedand expanded editionof Prentice-Hall,Englewood Cliffs, NewJersey, 1962), Springer,2000.
(page 18 contains
historical references to
irreducibility)
48/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
More on reducibility
The matrix A =
(0 11 0
)is irreducible (we just do not have many
alternatives to permute it symmetrically and obtain the required form).
Yet, clearly it is not hard to solve Ax = b; the solution is x2 = b1 andx1 = b2.
Here comes something more general than irreducibility.
49/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Fully indecomposability
Fully indecomposable matrix: There is no permutation matrices P and Qsuch that
PAQ =
(A11 A12
O A22
),
with the same condition on the blocks and their sizes as above.
Theorem: An n × n square matrix A is fully indecomposable iff for somepermutation matrix Q, the matrix AQ is irreducible and has a zero-freemain diagonal.
Proof: Skipped the proof in the class but looked at the implication of thetheorem. What the algorithm is to determine if a given matrix is fullyindecomposable: find perfect matching, permute the matrix so that thediagonal has the determined matching and then run the irreducibility test.
50/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Fully indecomposable matrices
Fully indecomposable matrix
There is no permutation matrices P and Q such that
PAQ =
(A11 A12
O A22
),
with the same condition on the blocks and their sizes as above.
Proof: Skipped the proof in the class.
51/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Cliques and independent sets
Clique
In an undirected graph G = (V ,E), a set of vertices S ⊆ V is a clique if for alls, t ∈ S , we have (s, t) ∈ E .
Maximum clique: A clique of maximum cardinality (finding a maximum cliquein an undirected graph is NP-complete).
Maximal clique: A clique is a maximal clique, if it is not contained in anotherclique.
In a symmetric matrix A, a clique corresponds to a subset of rows R and thecorresponding columns such that the matrix A(R,R) is full.
Independent set
A set of vertices is an independent set if none of the vertices are adjacent toeach other. Can we find the largest one in polynomial time?
In a symmetric matrix A, an independent set corresponds to a subset of rows Rand the corresponding columns such that the matrix A(R,R) is either zero, ordiagonal.
52/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: More on cliques
Clique: In an undirected graph G = (V ,E ), a set of vertices S ⊆ V is aclique if for all s, t ∈ S , we have (s, t) ∈ E .
In a symmetric matrix A corresponds to a subset of rows R and thecorresponding columns such that the matrix A(R,R) is full.
Cliques in bipartite graphs: Bi-cliques
In a bipartite graph G = (U ∪ V ,E ), a pair of sets 〈R,C 〉 where R ⊆ Uand C ⊆ V is a bi-clique if for all a ∈ R and b ∈ C , we have (a, b) ∈ E .
In a matrix A, corresponds to a subset of rows R and a subset ofcolumns C such that the matrix A(R,C ) is full.
The maximum node bi-clique problem asks for a bi-clique of maximumsize (e.g., |R|+ |C |), and it is polynomial time solvable, whereasmaximum edge bi-clique problem (e.g., asks for a maximum |R| × |C |) isNP-complete.
53/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Definitions: Hypergraphs
Hypergraph: A hypergraph H = (V ,N) consists of a finite set V called thevertex set and a set of non-empty subsets of vertices N called the hyperedgeset or the net set. A generalization of graphs.
For a matrix A, define a hypergraph whose vertices correspond to the rows andwhose nets correspond to the columns such that vertex vi is in net nj iff aij 6= 0(the column-net model).
A sample matrix
1 2 3 4
1 × × ×2 × ×3 × × ×
The column-net hypergraph model
43n
1n
1v
2v
3v
n
n2v
2n
4n
3n
1n
1
2
3
v
v
54/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Basic graph algorithms
We assume some familiarity with
Breadth-first search, BFS tree
Depth-first search, DFS tree (in particular edge classifications)
Topological sort of DAGs, finding connected components ofundirected graphs and strongly connected components of directedgraphs.
transitive closure or reduction of a directed graph.
55/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Depth-first search: Undirected graphs
Edge classification
Any DFS on an undirected graph produces only Tree and Back edges.
16/17
a c g
h f
d j
i e b
1/20
2/19
3/12
5/86/7
9/10
4/11
13/14
15/18
i
j
h
g
c f
e d
a
b
56/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Could not get enough of it: Questions
How would you describe the following in the language of graphs
the structure of PAPT for a given square sparse matrix A and apermutation matrix P (Class).
the structure of PAQ for a given square sparse matrix A and twopermutation matrices P and Q.
the structure of Ak , for k > 1, where A has a zero diagonal. Whatabout Ak when A has a zero-free diagonal? (Class).
the structure of AAT (Class).
the structure of the vector b, where b = Ax for a given sparsematrix A, and a sparse vector x (Class).
57/58 CR12
Course presentationSparse matrices
Graphs: Definitions and some problems
Could not get enough of it: Questions
Can you define:
the row-net hypergraph model of a matrix.
a matching in a hypergraph (is it a hard problem?).
Can you relate:
the DFS or BFS on a tree to a topological ordering? postordering?
58/58 CR12