CS2420: Lecture 42

Vladimir Kulyukin

Computer Science Department

Utah State University

Outline

• Graph Algorithms (Chapter 9)

Initial Graph

B C

A DF

E

3

1

6

86 2

44

5 5

Kruskal’s Algorithm: Basic Insight

A cycle is created if and only if the new edge connects two vertices already connected by a path, i.e., belonging to the same tree.

Kruskal’s: Basic Ideas

• Initially, all vertices are in separate trees.

• Subsequently, when an edge (u, v) is examined, we look for the tree containing u and the tree containing v.

• If the trees are NOT the same, we unite them into a larger tree containing both u, v and (u, v).

Kruskal’s Algorithm: UnionFind

• Kruskal’s Algorithm can be implemented with the UnionFind data structure.

• The UnionFind data structure was designed to solve the problem of partitioning a given set into a collection of disjoint subsets.

• The set partitioning problem has many applications in data mining, information retrieval, molecular biology, bioinformatics, etc.

Disjoint Subsets of a Set

• Let S be a set of n elements. Partition S into a collection of k disjoint subsets S1, S2, S3, ..., Sk.

Disjoint Subsets of a Set

Set S of n elements

Disjoint Subsets of a Set

Partitioning of S into 5 disjoint subsets

Example: Partition S into 6 Disjoint Subsets

• S = {1, 2, 3, 4, 5, 6}, k = 6– S1 = {1}

– S2 = {2}

– S3 = {3}

– S4 = {4}

– S5 = {5}

– S6 = {6}

Example: Partition S into 5 Disjoint Subsets

• S = {1, 2, 3, 4, 5, 6}, k = 5– S1 = {1, 2}

– S2 = {3}

– S3 = {4}

– S4 = {5}

– S5 = {6}

UnionFind Data Structure: Operations

• The UnionFind data structure has the following operations:– make a set out of a single element (singleton).– find a subset that contains some element x.– compute the union of two disjoint subsets, the

first of which contains x and the second of which contains y.

UnionFind: Operations

• MakeSet(x) - creates a one-element set {x}.

• Find(x) - returns a subset containing x.

• Union(x, y) - constructs the union of the disjoint sets Sx and Sy such that Sx contains x and Sy contains y.

• Sx U Sy replaces both Sx and Sy in the collection of subsets.

MakeSet: Example

• S = {1, 2, 3, 4, 5, 6}.

• MakeSet(1); MakeSet(2); MakeSet(3); MakeSet(4); MakeSet(5); MakeSet(6);

• The above sequence of MakeSet operations gives us:

• {{1}, {2}, {3}, {4}, {5}, {6}}.

Union: Examples

• {{1}, {2}, {3}, {4}, {5}, {6}}

• Union(1, 4) {{1, 4}, {2}, {3}, {5}, {6}}

• Union(5, 2) {{1, 4}, {5, 2}, {3}, {6}}

Union: Examples

• {{1, 4}, {5, 2}, {3}, {6}}

• Union(4, 5) {{1, 4, 5, 2}, {3}, {6}}

• Union(3, 6) {{1, 4, 5, 2}, {3, 6}}

UnionFind: Implementation

• Use one element of each disjoint set as a representative.

• Two implementation alternatives:– QuickFind: Find = O(1); Union = O(N).– QuickUnion: Union = O(1); Find = O(N).

QuickFind

• Two data structures:– An array of representatives; this array maps

each element into its representative.– An array of subsets; this array contains each

subset implemented as a linked list.

{{1}, {2}, {3}, {4}, {5}, {6}}

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

Representatives Subsets

Union(1, 4) => {{1, 4}, {2}, {3}, {5}, {6}}

1

2

3

1

5

6

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

Representatives Subsets

Null

Union(5, 2) => {{1, 4}, {5, 2}, {3}, {6}}

1

5

3

1

5

6

1

2

3

4

5

6

1

2

3

4

5

6

1

2

3

4

5

6

Representatives Subsets

Null

Null

Union(4, 5) => {{1, 4, 5, 2}, {3}, {6}}

1

1

3

1

1

6

1

2

3

4

5

6

1 2

3

4 5

6

1

2

3

4

5

6

Representatives Subsets

Null

Null

Null

union(3, 6) => {{1, 4, 5, 2}, {3, 6}}

1

1

3

1

1

3

1

2

3

4

5

6

1 2

3

4 5

6

1

2

3

4

5

6

Representatives Subsets

Null

Null

Null

Null

QuickFind: Asymptotic Analysis

• MakeSet(x) = O(1)

• Why?

• All we need to do is to create a list and add x to it.

QuickFind: Asymptotic Analysis

• Find(x) = O(1)

• Why?

• All we need to do is look up x’s representative and then return the list to which the representative points.

QuickFind: Asymptotic Analysis

• Union(x, y) = O(n)

• Here is what we need to do to compute Union(x, y):– Do Find(x) and Find(y); // O(1)– Append Find(y) to the end of Find(x); // O(1) – Set the y-subset in the subsets array to NULL;

// O(1)– Update the representatives for each element

in Find(y); // O(N).

QuickFind: Asymptotic Analysis

• Consider the following sequence of unions:– union(2, 1), union(3, 2), union(4, 3), ...,

union(n, n-1).

• The first union requires 1 step, the second - 2 steps, the third - 3 steps, ..., the n-th - n-1 steps. The sequence of n unions is asymptotically quadratic:

• 1 + 2 + 3 + ... + (n - 1) = O(n2).

QuickFind: Optimizations

• When performing the union operation, always append the shorter list to the longer one (union by size).

• The worst case run time of any legitimate sequence of unions is O(nlogn).

• The worst case run time of a sequence of n-1 unions and m finds is O(nlogn + m).

QuickUnion

• Two data structures:– An array of nodes, each containing exactly

one element.– An array of trees, each containing one

subset.– The root of each tree is the representative for

the subset represented by that tree.– Each node has a pointer to its parent so that

we can get to the root.

{{1}, {2}, {3}, {4}, {5}, {6}}

1

2

3

4

5

6

1

2

3

4

5

6

Nodes Trees

1

2

3

4

5

6

1

2

3

4

5

6

Union(1, 4) => {{1, 4}, {2}, {3}, {5}, {6}}

1

2

3

4

5

6

1

2

3

4

5

6

Nodes Trees

1

2

3

4

5

6

1

2

3

4

5

6

Null

Union(5, 2) => {{1, 4}, {5, 2}, {3}, {6}}

1

2

3

4

5

6

1

2

3

4

5

6

Nodes Trees

1

2

3

4

5

6

1

2

3

4

5

6

Null

Null

Union(4, 5) => {{1, 4, 5, 2}, {3}, {6}}

1

2

3

4

5

6

1

2

3

4

5

6

Nodes Trees

1

2

3

4

5

6

1

2

3

4 5

6

Null

Null

Null

Union(3, 6) => {{1, 4, 5, 2}, {3, 6}}

1

2

3

4

5

6

1

2

3

4

5

6

Nodes Trees

1

2

3

4

5

6

1

2

3

4 5

6

Null

Null

Null

Null

Union(5, 6) => {1, 4, 5, 2, 3, 6}

1

2

3

4

5

6

1

2

3

4

5

6

Nodes Trees

1

2

3

4

5

6

1

2

34 5

6

Null

Null

Null

Null

Null

Quick Union: Asymptotic Analysis

• MakeSet(x) = O(1)

• Why?

• All we need to do is to create a tree node and add x to it.

QuickUnion: Asymptotic Analysis

• Find(x) = O(n)

• Why?

• We need to look up x’s node and then chase the upward pointers to the root of the tree that contains x.

QuickUnion: Asymptotic Analysis

• Union(x, y) = O(1)

• Why?– Do Find(x) and Find(y); // O(1)– Attach the root of Find(y) to the root of

Find(x); // O(1)– Set the pointer in the tree array that used to

point to Find(y) to NULL; // O(1)

QuickUnion: Optimization

• We can optimize the union operation by always attaching the root of a larger tree to the root of the smaller tree.

• Two alternatives:– Union by size - the size of the tree is the

number of nodes in the tree.– Union by rank - the rank of the tree is the

tree’s height.

QuickUnion: Optimization

• If union by size or union by rank is used, the height of the tree is logn.

• The time efficiency of a sequence of n-1 unions and m finds is O(n + mlogn).