1NTUEEAlgorithms

Algorithms演算法

Professor Chien-Mo James Li 李建模

Graduate Institute of Electronics EngineeringNational Taiwan University

Data Structures(focus on Trees)

2NTUEEAlgorithms

Dynamic Set• Dynamic set is a set of elements that can grow or shrink over time

• Each element is represented by an object of two attributes

♦ key: a unique number/character to identify the object

♦ satellite data: carried around but not used in set operation

• Query operations

♦ SEARCH

♦ MINIMUM

♦ MAXIMUM

♦ SUCCESSOR

♦ PREDECESSOR

• Modifying operations

♦ INSERT

♦ DELETE

3NTUEEAlgorithms

Outline• Elementary Data Structures, CH10

• Hash Table CH 11 * (not in exam)

• Binary Search Trees , CH12

• Red Black Trees, CH13

4NTUEEAlgorithms

Stack• Last In First Out (LIFO)

♦ underflow: pop an empty stack

♦ overflow: push a full stack

S.top S.top S.top

5NTUEEAlgorithms

Stack Operations• What are their time complexities?

STACK-EMPTY(S)1 if S.top == 02 return TRUE3 else return FALSE

POP(S)1 if STACK-EMPTY(S)2 error “underflow”3 else S.top = S.top – 14 return S[S.top + 1]

PUSH(S, x)1 S.top = S.top + 12 S[S.top] = x

6NTUEEAlgorithms

Queue• First In First Out (FIFO)

♦ leaves from head and enters from tail

• Fig. 10.2

♦ (b) enqueue

∗ 17,3,5

♦ (c) dequeue

∗ 15Q.head

Q.head

Q.head

Q.tail

Q.tail

Q.tail

7NTUEEAlgorithms

Queue Operations• What are their time complexities?

ENQUEUE(Q, x)1 Q[Q.tail] = x2 if Q.tail == Q.length3 Q.tail = 14 else Q.tail = Q.tail + 1

DEQUEUE(Q)1 x = Q[Q.head]2 if Q.head == Q.length3 Q.head = 14 else Q.head = Q.head + 15 return x

8NTUEEAlgorithms

Doubly Linked List• An object has three attributes

♦ key, prev (pointer) and next (pointer)

• Fig 10.3

L.head

L.head

L.head

9NTUEEAlgorithms

List Operations• What are their time complexities?

LIST-SEARCH(L, k)1 x = L.head2 while x ≠ NIL and x.key ≠ k3 x = x.next4 return x

LIST-INSERT(L, x)1 x.next = L.head2 if L.head ≠ NIL3 L.head.prev = x4 L.head = x5 x.prev = NIL

LIST-DELETE(L, x)1 if x.prev ≠ NIL2 x.prev.next = x.next3 else L.head = x.next4 if x.next ≠ NIL5 x.next.prev = x.prev

10NTUEEAlgorithms

Linked List with Sentinels• Sentinel is a dummy object for boundary condition simplification

♦ sentinel has no key, but has pointers

♦ L.nil points to the sentinel

• Fig 10.4

L.nil

L.nil

L.nil

L.nil

11NTUEEAlgorithms

List Operation w/ Sentinel• Sentinel helps to make code easier

• Does sentinel help to improve complexity?

LIST-INSERT’(L, x)1 x.next = L.nil.next2 L.nil.next.prev = x3 L.nil.next = x4 x.prev = L.nil

LIST-SEARCH’(L, k)1 x = L.nil.next2 while x ≠ L.nil and x.key ≠ k3 x = x.next4 return x

LIST-DELETE’(L, x)1 x.prev.next = x.next2 x.next.prev = x.prev

12NTUEEAlgorithms

Other Lists• Singly linked list

♦ no previous pointer

• Sorted list

♦ elements are sorted by their keys

• Unsorted list

♦ elements are not sorted

• Circular list

♦ previous of head points to the tail

♦ next of tail points to the head

13NTUEEAlgorithms

Outline• Elementary Data Structures, CH10

• Hash Table CH 11 (NOT in exam)

• Binary Search Trees , CH12

• Red Black Trees, CH13

14NTUEEAlgorithms

Hash Table• Use hashing when universe of keys is very large

• Hash function: map key k is to slot h(k)

• Collision: two keys hashed to the same h(k)

15NTUEEAlgorithms

Collision Resolution by Chaining• Chaining: place all elements hashed to the same slot in a linked list

16NTUEEAlgorithms

Outline• Elementary Data Structures, CH10

• Hash Table CH 11

• Binary Search Trees , CH12

• Red Black Trees, CH13

17NTUEEAlgorithms

Representation of a Tree• Each node x has 4 attributes

♦ x.key (and possibly other satellite data)

♦ x.left: points to x’s left child.

♦ x.right: points to x’s right child.

♦ x.p: points to x’s parent.

• T.root points to the root of tree T ; T.root.p = NIL

• A node with no children is a leaf (or external node)

T.root

leaf

18NTUEEAlgorithms

Binary Search Tree (BST)• Binary Search Tree Property

♦ If y is in left subtree of x, then y.key ≤ x.key

♦ If y is in right subtree of x, then y.key ≥ x.key

• Height of node

♦ # of edges on longest downward path from the node to a leaf

• Height of tree = height of root

• Worst running time to search a BST is proportional to its height

• Example: (a) height = 2 (b) height =4, (b) is less efficient to search

19NTUEEAlgorithms

Inorder Tree Walk• Visit nodes in this order

♦ left-subtree,

♦ root,

♦ Right-subtree

• ITW sorts keys increasingly

• Example: 2, 3, 5, 5, 7, 8

• Q1: are the results of (a) and (b) the same?

• Q2: what is time complexity? (see 12.1)

20NTUEEAlgorithms

Other Tree Walks• Preorder tree walk

♦ root, subtree

• Postorder tree walk

♦ subtree, root

• Q1: What is the postorder of the following trees?

• Q2: What is the order of keys?

21NTUEEAlgorithms

Min. and Max.• Always follow the left child

♦ minimum=2

• Complexity = Θ(h)

• Always follow the right child

♦ maximum=20

• Complexity = Θ(h)

22NTUEEAlgorithms

• Path to find 13: 15 6 7 13

• Complexity = Θ(h)

Searching

x.key

23NTUEEAlgorithms

FFT• Which version is better, recursive or iterative?

x.key x.keyx.key

= x.left= x.right

24NTUEEAlgorithms

Successor• Assume all keys are distinct, successor of x is y

♦ if y.key is the smallest key ≥ x.key

♦ Successor of x = next key of x in inorder tree walk

∗ if x is the largest key, successor is NIL

• Example

♦ successor of 15 is 17

♦ successor of 17 is 18

♦ successor of 4 is 6

♦ successor of 13 is ?

25NTUEEAlgorithms

Finding Successor• case 1. If node x has non-empty right subtree,

♦ x’s successor (y) is minimum in x’s right subtree

♦ example: successor of 15 is 17

• case 2. If node x has empty right subtree,

♦ x’s successor (y) is the lowest ancestor of x

∗ y’s left child is x’s ancestor

♦ Example: successor of 4 is 6

// case 1

// case 2

// x==y.left

26NTUEEAlgorithms

Predecessor• Assume all keys are distinct, predecessor of x is y

♦ if y.key is the largest key < x.key

♦ Predecessor of x = previous key of x in inorder tree walk

∗ if x is the smallest key, predecessor is NIL

• Example

♦ predecessor of 15 is 13

♦ predecessor of 7 is 6

♦ predecessor of 17 is ?

27NTUEEAlgorithms

Exercises• Q1: show successor time complexity = O(h)

♦ Hint: Theorem 12.2

• Q2: please write predecessor algorithm

♦ hint: symmetric to successor

28NTUEEAlgorithms

Insertion• Insert z into BST

• Beginning at root, trace downward

♦ x: traces the downward path looking for NIL to insert z

♦ y is trailing pointer, parent of x.

• When z.key < x.key trace left subtree

♦ otherwise, trace right subtree

• When x is NIL, it is the position for z

• Compare z’s value with y’s value,

♦ z.key < y.key: insert z at y’s left

♦ z.key > y.key: insert z at y’s right

• Time complexity = Θ(h)

// empty tree

29NTUEEAlgorithms

Insertion Example• Insert 14

• Insert 16

30NTUEEAlgorithms

Deletion Examples• z is deleted: (a) z has no child, (b) z has 1 child, (c) z has 2 children

31NTUEEAlgorithms

Deletion (case 1, 2)• case 1. If z has no children, just remove z

• case 2. If z has just one child,

♦ child take z’s position , dragging the child’s subtree along.

Fig 12.4 (a) Fig 12.4 (b)

32NTUEEAlgorithms

Deletion (case 3)• case 3. If z has two children, find z’s successor y

♦ 3a: If y is z’s right child, replace z by y (leave y’s right child alone)

♦ 3b. Otherwise, replace y by x (y’s right child), replace z by y

Fig 12.4 (c)

Fig 12.4 (d)

33NTUEEAlgorithms

Tree-Delete*

case 1+2

case 3

*this algorithm is different from 2nd ed.

case 3b: replace y by x

case 3a: replace z by y

34NTUEEAlgorithms

Transplant• Replace the subtree rooted at u

♦ by the subtree rooted at v

• Time complexity = Θ(1)

35NTUEEAlgorithms

Exercise• Case 1: Delete A

• Case 2: Delete G

• Case 3a: Delete K

• Case 3b: Delete B

36NTUEEAlgorithms

Complexity Analysis• TRANSPLANT = Θ(1)

• TREE-MINIMUM = Θ(h)

• other lines = Θ(1)

• over all = Θ(h)

37NTUEEAlgorithms

FFT• Why does y not have left child?

38NTUEEAlgorithms

Summary• BST supports all dynamic set operations in Θ (h)

♦ SEARCH

♦ INSERT

♦ DELETE

♦ MINIMUM

♦ MAXIMUM

♦ SUCCESSOR

♦ PREDECESSOR

• CH 12.4 shows the expected tree height is O(lg n)

♦ but what about the worst case?

♦ is there a smart algorithm that guarantee h=lg(n) in worst case?

39NTUEEAlgorithms

Outline• Elementary Data Structures, CH10

• Hash Table CH 11

• Binary Search Trees , CH12

• Red Black Trees, CH13

40NTUEEAlgorithms

Red-black Tree• A red-black tree is a binary search tree

♦ plus an attribute color

♦ red-black tree is approximately balanced

• Red-black tree must satisfy five properties:

♦ 1. Every node is either red or black

♦ 2. Root is black

♦ 3. Every leaf (T.nil) is black

♦ 4. If a node is red, then both its children are black

∗ Hence, no two consecutive reds on a path from root to a leaf

♦ 5. All paths from a node to descendant leaves

∗ contain the same number of black nodes

∗ i.e. black height of a node is unique

41NTUEEAlgorithms

Black Height• Height of node x, h(x):

♦ number of edges in a longest path from node x to a leaf

• Black height of node x, bh(x):

♦ number of black nodes on the path from node x to leaf,

∗ including T.nil

∗ not counting x itselfT.root

42NTUEEAlgorithms

Red-Black Tree Example• Fig. 13.1

T.nil

43NTUEEAlgorithms

FFT• Why R&B tree is (approx. ) balanced?

44NTUEEAlgorithms

What is T.nil?• Sentinel T.nil is a dummy object that represents NIL

• T.nil has five attributes

♦ p, left, right, and key

♦ color is black

• Only one T.nil for the whole tree to save space

• Use T.nil helps to make boundary condition easier

♦ will see later

• T.nil is usually not plotted in the tree

45NTUEEAlgorithms

Two Claims• Claim 1: For any node x, bh(x) ≥ h(x)/2

♦ By property 4, fewer than h(x)/2 nodes are red

♦ Hence, more than bh(x)/2 are black

• Claim2: Subtree rooted at node x contains more than 2bh(x)-1 nodes

♦ math induction:

♦ 1. x is leaf, bh(x) =0, subtree contains zero nodes

♦ 2. Any child of x has black-height

∗ either bh(x) (if the child is red)

∗ or bh(x)-1 (if the child is black)

♦ By hypothesis, each child contains ≥∗ 2bh(x)-1 -1 internal nodes

♦ Thus, the subtree rooted at x contains ≥∗ 2(2bh(x)-1-1)+1 = 2bh(x) - 1 internal nodes

∗ ‘+1’ is x itself

46NTUEEAlgorithms

Lemma 13.1• A red-black tree with n internal nodes has height h ≤ 2 lg (n +1)

♦ according to above two claims, at root

• Hence, red-black tree is approximately balanced

♦ height is at most twice minimum

• SEARCH, MIN, MAX, PRECEDESSOR, SUCCESSOR

♦ all query operations are O(h)= O(lg (n))

♦ how about insertion and deletion?

∗ must maintain the red-black tree properties

)1lg(2

121212 2/2/)()(

+≤−=−≥−≥

nh

n hroothrootbh

47NTUEEAlgorithms

Rotation• Rotation is a local operation for binary search trees

♦ left/right rotations. they are inverse of each other

♦ preserve BST property

♦ does not change order

• Example: Fig. 13.2

♦ inorder walk: α, x, β, y, γ unchanged after left/right rotation

48NTUEEAlgorithms

Left-Rotate

assume y≠T.nil

time complexity = O(1)

// β

y.p x.p

49NTUEEAlgorithms

Left-Rotate Example

inorder tree walk unchanged

50NTUEEAlgorithms

Exercise• Please write pseudo code for right-rotate

♦ hint: exchange left and right

51NTUEEAlgorithms

Insertion• insert node z

• four differences

52NTUEEAlgorithms

Keep RB Tree Properties• Assign red color to inserted node, z.color = red, any violation?

♦ 1. Every node is either red or black∗ OK

♦ 2. The root is black∗ OK, unless z is root

♦ 3. Every leaf (T.nil) is black∗ OK

♦ 4. If a node is red, then both its children are black ∗ if z.p is red, two reds can be in a row, violation!∗ how to fixup? check uncle’s color

♦ 5. For each node, black height is unique∗ OK, why?

• Color fixup: move the “extra red” from z up the tree ♦ until it reaches the root

∗ simply paint the root black♦ suitable rotation and recoloring is performed to ensure

∗ all properties are preserved below z

53NTUEEAlgorithms

Fixup: Case 1• Case 1: If uncle y is red

♦ paint z’s parent (A) and uncle (D) black

♦ paint z’s gradparent (C) red

♦ move new z up two levels

♦ keep checking upwards

• Fig. 13.5

54NTUEEAlgorithms

Fixup: Case 2, 3• Case 2: If uncle y is black and z is a right child

♦ left rotate around z.p (A)

♦ turn into case 3

• Case 3: If uncle y is black and z is a left child

♦ paint parent z.p (B) black and grandparent z.p.p (C) red

♦ right rotate on z.p.p (C)

♦ no more iterations

• Fig. 13.6

55NTUEEAlgorithms

Exercise• Try to insert 10 85 15 70 20 60 30 50 65 80 90 40 5 55

• Too Many Cases?

• How can I remember so many cases?

56NTUEEAlgorithms

Fixup Algorithm

// y is uncle

57NTUEEAlgorithms

Loop Invariant• At the start of each iteration of the while loop,

♦ z is red

♦ There is at most one red-black violation:

∗ Property 2: z is a red root, or

∗ Property 4: z and z.p are both red.

• Initialization: We already seen why the loop invariant holds initially.

• Termination: The loop terminates because z.p is black.

♦ Hence, property 4 is OK. Only property 2 might be violated,

∗ the last line fixes it.

• Maintenance:

♦ When we start the loop body, the only violation is property 4

♦ need to consider 6 cases

∗ 3 cases are symmetric

58NTUEEAlgorithms

Complexity of RB-Insert• RB-Insert-Fixup

♦ O(lg n)

♦ no more than two rotations

• RB-Insert

♦ O(lg n)

59NTUEEAlgorithms

RB-Insert-Fixup Example• Fig 13.4

60NTUEEAlgorithms

Exercise• Please finish the other 3 cases of RB-INSERT-FIXUP algorithm

61NTUEEAlgorithms

Tree-Delete (Review)• case 1 (z has no child): remove z

• case 2 (z has one child): replace z by its own successor

• case 3 (z has two child): y = z’s successor

♦ 3a: If y is z’s right child, replace z by y

♦ 3b. Otherwise, replace y by x

∗ then replace z by y

62NTUEEAlgorithms

RB-Delete case1,2

• z has < 2 child

♦ y=z

♦ x=y’s child

• Because y(=z) is removed, y’soriginal color is gone

♦ color fixup from node x

y

x

x

x

y x

*different from 2nd ed.

63NTUEEAlgorithms

RB-Delete case 3

• z has 2 children

♦ y = z’s successor

♦ x = y’s successor

• y inherits z’s color, y’s original color is gone

♦ color fixup from node x

replace z by y

replace y by x

y inherits z’s color

64NTUEEAlgorithms

Delete Fixup• If y originally was red, no violation

♦ why?

• If y originally was black, push the black onto its only child x

♦ what if x is already black? it becomes doubly black

∗ violation!

• Color fixup: move the “doubly black” from x up the tree until

♦ 1. x points to a red node,

∗ simply color it black

♦ 2. x points to the root,

∗ simply remove doubly black

♦ suitable rotation and recoloring is performed to ensure

∗ all properties are preserved below x

65NTUEEAlgorithms

Fixup Case 1• Case 1: x’s sibling w is red

♦ w must have black children.

♦ Make w (D) black and x.p (B) red.

♦ Left rotate on x.p (B).

♦ x’s new sibling (C) is black

∗ new w is now C

♦ Go immediately to case 2, 3, or 4

• Fig 13.7

66NTUEEAlgorithms

Fixup Case 2• Case 2: w is black and both w’s children are black

♦ Take “one black” off x (doubly black singly black) and

♦ also take “one black” off w (black red).

♦ Move that “one black” up to x.p (B)

∗ new x is now B

♦ If B is red, color it black and terminates

∗ otherwise, keep going up and do the next iteration

67NTUEEAlgorithms

Fixup Case 3• Case 3: w is black, w’s left child (C) is red and right child (E) is black

♦ Make w (D) red and w’s left child (C) black

♦ Then right rotate on w (D).

♦ New sibling of x (C) is black with a red right child (D)

∗ new w is now C

♦ becomes case 4

68NTUEEAlgorithms

Fixup Case 4• Case 4: w is black, w’s right child (E) is red, w’s left child (C) is don’t

care

♦ Make w (D) be x.p’s (B) color (D.color = B.color)

♦ Make x.p (B) black and w’s right child (E) black

♦ Left rotate on x.p (B)

♦ Remove the extra black on x

♦ Setting x to root causes the loop to terminate

69NTUEEAlgorithms

Too Many Cases?• How can I remember so many cases?

• Any real life example ?

70NTUEEAlgorithms

71NTUEEAlgorithms

Complexity of RB-Delete• O(lg n) time to get through RB-DELETE up to RB-DELETE-FIXUP.

• Within RB-DELETE-FIXUP:

♦ Case 2 is the only case in which more iterations occur.

∗ x moves up 1 level, O(lg n) iterations.

♦ Each of cases 1, 3, and 4 has 1 rotation

∗ less than 3 rotations in all.

• Totally,O(lg n) time.

72NTUEEAlgorithms

FFT• in case 3a,

• since x = y.right

• x.p=y is redundant?

redundant ???

73NTUEEAlgorithms

RB-Transplant• Make two differences for RB tree

♦ RB-TRANSPLANT references T.nil

♦ v.p = u.p , even if v is the sentinel T.nil

∗ can trace from T.nil up to its parent correctly

∗ fixup exploits this feature

74NTUEEAlgorithms

Other Self-balancing BST• The idea of balancing a BST is due to Adel’son Verl’skii and Landis

♦ AVL tree

• AA-tree

• Treaps

• Splay tree

75NTUEEAlgorithms

Reading• CH10 (optional)

• CH 12, CH 13

• Appendix B.5

• RB-tree的Demo程式http://www.ece.uc.edu/~franco/C321/html/RedBlack/redblack.html

♦ 可以一步一步demo 如何加入及删除

♦ 好玩！請試試看