Trees

Part of this lecture was organized by the instructors at the University of Manitoba in Canada and has been modified by Dr. Ahmad Reza Hadaegh

Trees• A tree is a linked data structure which reflects some

hierarchical structure

• There are several varieties of trees

• Each with its own characteristics• A family of related data structures

Trees• A tree gets its name from its structure which is tree-like

• A tree has a root, branches, leaves

A

JI

HGFE

DCB

Root node

Leaf nodes

Branches

Trees• A tree consists of a number of connected nodes

• The nodes are organized in a top-down hierarchical manner

• A tree is an instance of a general structure called a graph• A graph consists of vertices (nodes) and edges

(branches), but is not necessarily hierarchical in nature• Sometimes tree and graph terminology is

interchanged

Trees

• This tree has • 10 nodes, 9 branches, 1 root node, and 6 leaf nodes

A

JI

HGFE

DCB

Trees

• Other defining characteristics• If node A points to node B, then A is the parent of B

• Furthermore, B is the child of A

• Children of the same parent are siblings

• All nodes “beneath” a given node (children, children of children, …) are descendants of that node

• All nodes on the path between the root and a given node are ancestors of that node

Trees

• The parent of G is D and the parent of E is B• The children of A are B, C, and D• B, C, and D are sibling• The ancestors of J are G, D, and A• The descendants of D are G, H, I and J

A

JI

HGFE

DCB

Trees• Some typical operations upon a tree

• Insert• Add a new node into a tree

• Delete• Remove a node from a tree

• Search• Find a node (if it exists) that contains a given key

• Traverse• Systematically examine (an operate on) all nodes in a tree

Binary Trees• Binary trees

• A binary tree is a tree in which no node has more than two children

A

HG

FED

CB

Ordered Binary Trees

• Ordered binary trees• An ordered binary tree is a binary tree in which

left descendants < given node < right descendants

• Applied recursively throughout the tree structure

Ordered Binary Trees

• The following binary tree is not ordered

A

HG

FED

CB

Ordered Binary Trees

• The following binary tree is ordered

D

HF

GCA

EB

Ordered Binary Trees

• The following binary tree is also ordered

D

H

E G

B

A

FC

Ordered Binary Trees

• The following binary tree is also ordered

B

E

A

D

H

F

C

G

Ordered Binary Trees

• Ordered binary trees are often referred to as binary search trees (BST)

• They tend to be a “co-operative” structure on which to work

Searching a BST

• Searching a BST

• Algorithm• Start with root node• if search key = node key then stop• else if search key < node key then go left• else go right• Stop if at any time during the search, you fall off the

tree, stop

Searching a BST

• Find node with key C

`

D

HF

GCA

EB

Searching a BST

• Find node with key X

D

HF

GCA

EB

Searching a BST

• Given an existing BST, searching for a key is simple

• Effectively a binary search• Is it as efficient as a binary search?

• Algorithm is recursive, but can be trivially implemented iteratively

• Could be implemented as follows:

Searching a BSTbool Search(nodeptr curr, char key) { bool found; nodeptr curr;

curr = root; found = false; while ((curr != NULL) && (! found)) if (curr->data == key) found = true; else if (key < curr->data) curr = curr->left; else curr = curr->right; return found;}

// We can also do the search recursively

Traversing a Tree• Tree traversal is a common operation

• Traversal of a tree can be divided into two category:• Depth-First Traversal and• Breadth-First Traversal

• Three common types of depth-first traversal are:• Pre-order, • post-order, and • in-order traversal

• Depth-First Traversal can be easily done recursively

• We can also use stack operation to print a traverse a tree

• Breath-First traversal can be done with Queue

Recursive Depth-First Traversal

• In-order traversal● Perform an In-order traversal of the left sub-tree ● Visit the current node● Perform an In-order traversal of the right sub-tree

• Pre-order traversal● Visit the current node● Perform a pre-order traversal of the left sub-tree● Perform a pre-order traversal of the right sub-tree

• Post-order traversal● Perform a Post-order traversal of the left sub-tree● Perform a Post-order traversal of the right sub-tree● Visit the current node

void InOrderTraversal (NodePtr root) { if (root = = NULL) return; InOrder(root->left); cout << root ->data << endl; InOrder(root->right);}

void PreOrderTraversal (NodePtr root) { if (root = = NULL) return; cout << root ->data << endl; PreOrder(root->left); PreOrder(root->right);}

void PostOrderTraversal (NodePtr root) { if (root = = NULL) return; PostOrder(root->left); PostOrder(root->right); cout << root ->data << endl;}

Depth-First Traversal with StackIn Order - Traversalvoid Tree::PrintInOrderTreeWithStack(TreeStructPtr Root){ Stack stk; TreeStructPtr p = Root; while (p!= NULL) { while (p != NULL) { if (p->Right != NULL)

stk.push(p->Right); stk.push(p); p=p->Left; } p = stk.pop(); while (!stk.empty() && p->Right == NULL) { cout << p->Number << "-->"; p = stk.pop(); } cout << p->Number << "-->"; if (!stk.empty()) p = stk.pop(); else p = NULL; }}

void Tree::PrintPreOrderTreeWithStack(TreeStructPtr Root){

Stack stk;TreeStructPtr p = Root;if (p != NULL){

stk.push(p);while (!stk.empty()){

p = stk.pop();cout << p->Number << "-->";if (p->Right != NULL)

stk.push(p->Right);if (p->Left != NULL)

stk.push(p->Left);}

}}

Depth-First Traversal with StackPre Order - Traversal

void Tree::PrintPostOrderTreeWithStack(TreeStructPtr Root){

Stack stk;TreeStructPtr p = Root, q = Root;while (p!= NULL){

for ( ; p->Left != NULL; p = p->Left)stk.push(p);

while (p != NULL && (p->Right == NULL || p->Right == q)) {

cout << p->Number << "-->";q = p;if (stk.empty())

return;p = stk.pop();

}stk.push(p);p = p->Right;

}}

Depth-First Traversal with StackPost Order - Traversal

Breadth-First Traversal

`

D

HF

GCA

EB

• In Breadth-First traversal, all the nodes in the first level are printed, followed by the nodes in second level, followed by the nodes in the third level and so on.

• For example, the breadth-first traversal of the above graph gives: D B E A C G F H

• Queue can be used to implement the Breadth-first traversal

void Tree::PrintBreadthFirstWithQueue(TreeStructPtr Root){

Queue que;TreeStructPtr p = Root;if (p != NULL){

que.push(p);while (!que.empty()){

p = que.pop();cout << p->Number << "-->";if (p ->Left != NULL)

que.push(p->Left);if (p->Right != NULL)

que.push(p->Right);}

}}

Breadth-First Traversal with Queue

Stackless Depth-First Traversal• There are also algorithms that can be used to do depth-first

traversal of a tree without using stack or recursion.

• One of these methods is Threaded Trees

• Threads are pointers to the successors and predecessors of the nodes according to a particular traversal.

• For example, if we want to do in-order traversal, threads become pointers to the successors and predecessors of the nodes according to the in-order traversal.

• In this case every node’s left and right pointer is pointing to something. Even leaves of the tree point to something.

• If a node has a left child, its left pointer points to its left child; otherwise its left pointer is pointing to its predecessor

• If a node has a right child, its right pointer points to its right child; otherwise it points to its successor

• An example of threaded tree for in-Order traversal is shown below

• The threads shown in broken blue color pointing to the successors and the threads shown in solid red color are pointing to the predecessors

A

G

FED

CB

JIH

• In order Traversal gives us : G, D, B, H, E, I, A, F, J, C

• To do the in-order traversal, it is sufficient to keep the successor threads only. The broken red line shows the path of in-order traversal

• If a node has a right child, its successor pointer is NULL (Example: node B in the following tree)

• However, if the node does not have a right child, its right pointer points to its successor node (Example: Node A in the following tree)

A

B

TreeStructPtr p = Root; prev;

If (p != NULL){ while (p->Left != NULL) p = p->Left; while (p ! = NULL) { cout << p->Number << "-->"; } prev = p; p = p->Right; if (p != NULL && prev->Successor == NULL) while (p->Left != NULL) p = p->Left; }}

• Here is the algorithm to do the stackless in-order traversal of a binary tree

Another method of stackless in-order traversal

• Using threaded tree is not the only way to do stackless in-order traversal of a binary search tree.

• Another stackless algorithm is called Morris method

• In this case, we change the structure of the tree such that the tree has no left descendants

• Then we traverse the tree, print the content

• Next, we charge the structure of the tree to its original shape. • For in-order traversal of the tree, this algorithm is called MorrisInOrder

MorrisInOrder(){

P = Rootwhile not finished

if P has no left descendant{

visit it;P = P->Right;

}else{

Temp = the right child of the rightmost node in its left descendant of P

Temp->Right = P;

P = Root}

}

P

Temp

P

P

10

73

205

P

Temp 10

73

20

5

Temp

P

10

73

20

5

P

Temp

10

7

3

20

5

P Temp

• Now we can print the linked list.

• The nodes are organized in in-order form like a singular linked list

Step 1 Step 2 Step 3

Step 4

10

73

20

5

P

10

7

3

20

5P

Temp

10

73

20

5

P

10

73

20

5

Temp

P

10

73

205

P

Temp

10

73

205

P=NULL

Temp

Step 5 Step 6 Step 7

Step 8 Step 9 Step 10

• Now the tree is back to its original shape after the traversal is done

TempTemp

Insertions

• Binary trees are normally created by inserting nodes into an initially empty tree

● If tree empty, first insert will create the root node

● Otherwise, tack new node onto the (unique) appropriate node that has less than 2 children

● This will require a search routine

• Insert 15, 8, 12, 19, 5, and 23 into an empty BST

8

1515

8

15

12

198

15

12 23

198

15

125

Step 1 Step 2 Step 3

Step 4 Step 5 Step 6

198

15

125

// This procedure implements a recursive insert. // it can also be done iteratively.

void insert (nodeptr& root, nodeptr newnode) { if (root == null) root = newnode; else if (newnode->data < root->data) insert (root->left, newnode); else insert (root->right, newnode); }

void Tree::InsertWithNoRecursion(TreeStructPtr& Root, int x){ TreeStructPtr p = Root, prev=Root;

while (p != NULL){ prev = p;

if (x < p->Number) p = p->Left;else if (x > p->Number) p = p->Right;else{ cout << "\n!!!! " << x << " is already in the tree " << endl; return;}

}TreeStructPtr NewNode;NewNode = new (TreeStruct);NewNode->Number = x;NewNode->Left = NULL;NewNode->Right = NULL;if (Root = = NULL)

Root = NewNode;else if (prev->Number < x)

prev->Right = NewNode;else

prev->Left = NewNode;return;

}

This is the implementation of the insert function without using Recursion

Deletions

• Deleting is a little more difficult. Why?

• When deleting from a BST, the algorithm must often reorganize tree while maintaining the inherent ordering

• Three cases to consider

23

198

15

125

`

23

198

15

125

`

23

198

15

5

• Case 1: Deleting a leaf node● This is easy -- detach leaf node from tree by setting its

parent’s pointer to nil● For example, suppose we want to delete 12

• Case 2: Deleting a node with a single child● Easy -- replace the node to be deleted with its child

• For example, suppose we want to delete 19

23

198

15

125 23

198

15

125

238

15

125

12

2622

23107

218

17

1914

These nodes must be properly linked back into the tree

• Case 3: Deleting a node with two Children

• For example, suppose we want to delete 21

There are Two possible algorithms for delete

• Method 1:• Replace the deleted node with one of its children, say

the left child (i.e. node 17)

• The remaining right child (i.e. node 23) must now be linked back into the tree

• Place it on the right of the largest node of the new left subtree

12

2622

23107

218

17

1914

• Replace the deleted node with one of its children, say the left child (i.e. node 17)

12

2622

23107

8

17

1914

12

2622

23107

8

17

1914

• Place it (node 23) on the right of the largest node of the new left subtree

12

23

19107

178

14

• Done!

2322

Deletions• Method 2:

• Locate the in-order successor of the node to be deleted (the node that contains the next highest value)

• To locate the in-order successor, go right, then go left as far as possible

• Replace the contents of the node to be deleted with the contents of its in-order successor

• Delete the successor node• This algorithm is possible since the in-order successor

node will never have more than one child

12

2622

23107

218

17

1914

• Delete 21

12

2622

23107

218

17

1914

• Replace the contents of the node to be deleted with the contents of its in-order successor

12

2621

23107

228

17

1914

• Now swap the value of the node with the value of its successor node

• Delete the successor node

12

2621

23107

228

17

1914

Deletions

12

26

23107

228

17

1914

• Done!

• An alternative approach to the second method is to find predecessor of a node instead of the successor node.

• This mainly works the same as the second method

• Another issue to consider is that the second method does not change the structure of the tree as much as the first method does.

● The height of the tree does not change as much in the second method compare to the first method

• However, if we keep applying the second method for deletion and choose successor node to replace with the node, the tree eventually becomes unbalanced

• One way to improve the second method is we can write the algorithm to alternatively delete the predecessor of the node from the left subtree and delete the successor from the right subtree

Balancing a Binary Tree

• A binary tree is height-balanced or simply balanced if the difference between the left subtree and right subtree of any node is either 0 or 1

• For example, consider the following binary tree:

B

R

P

K

D

M

330B

121K

011P

DifferenceHeight of the right subtree

Height of the left subtree

Node

• Because there is at least one node (node B) with the difference value of 3, this tree is not balanced

• Suppose we receive a set f data all in one shot and we want to insert them into a binary tree such that the end result is a balanced binary search tree.

• If we insert the data one by one randomly, the tree may not turn to be a balanced binary search tree.

• One method to solve this problem is to ● First sort the data using the best sort algorithm available● Designate the middle element to be the root of the binary tree● The array would consist of two sub arrays:

● one from the first element to the middle element (the root) but not including the middle element

● Another consists of middle + 1 till the last element

● Now the left child of the tree is taken from the middle of the first sub-array and its right child from the middle of the second sub-array

● Now divide the first sub-array into two other sub-arrays and repeat the same process

● Similarly divide the other sub-array into two other sub-arrays and repeat the same process

void Tree:: Balance (T data[ ], int first, int last){ if (first <=last)

{ int middle = (first + last) /2;

insert (data[middle]);Balance (data, first, middle-1)Balance(data, middle+1, last);

}}

• The algorithm is:

• Example:● Stream of data: 5 1 9 8 7 0 2 3 4 6● Sorted Data: 0 1 2 3 4 5 6 7 8 9

4

Step 1

Step 2

0 1 2 3 4 5 6 7 8 9

1

4

0 1 2 3 4 5 6 7 8 9

Step 3

1

4

0

0 1 2 3 4 5 6 7 8 9

1

4

20

Step 4 0 1 2 3 4 5 6 7 8 9

Step 5 0 1 2 3 4 5 6 7 8 9

1

4

20

3

Step 6 0 1 2 3 4 5 6 7 8 9

1

4

20

3

7

Step 7 0 1 2 3 4 5 6 7 8 9

1

4

20

3

7

5

Step 8 0 1 2 3 4 5 6 7 8 9

1

4

20

3

7

5

6

Step 9 0 1 2 3 4 5 6 7 8 9

1

4

20

3

7

5

6

8

Step 10 0 1 2 3 4 5 6 7 8 9

1

4

20

3

7

5

6

8

9

• What if we already have a balanced binary search tree and we want to insert another element such that after insertion the tree remains balanced

• One solution is to sort all elements including the new one again and then re-insert the elements again to the tree as we discussed before.

• Another solution proposed by C. Day and later improved by Q. Stout and B. Warren. Their solution is known as DSW algorithm

• DSW algorithm avoids sorting. It acquires deconstructing and reconstructing of the tree

• The building block for the tree transformation in this algorithm is the rotation

• There are two types of rotations: Left rotation and Right Rotation which are symmetrical to one another

• The right rotation of a node called “Child” around its parent node called “Parent” is performed according to the following algorithm

RotateRight (Grandparent, Parent, Child){

- If Parent is not the root of the tree Grandparent of child becomes child’s parent by

replacing the parent

- Right subtree of child becomes left subtree of child’s parent

- Node child acquire parent as its right child}

Gr

c

a

Par

Ch

b

Gr

c a

Par

Ch

b

• The following is an example of right rotation

Gr

Ch

Par

Right rotation of Ch around Par

Left rotation of Ch around Par

Gr

Ch

Par

a

b ca b

c

• DSW algorithm transforms an arbitrary binary search tree into a linked list like tree called a backbone tree

• Then the backbone tree is transformed in a series of passes into a perfectly balanced tree by repeatedly rotating every second node of the backbone around its parent

5

30

10

20

40

15

2823

25

5

20

10

15

23

25

30

40

28

25

23

20 30

4010

155

28

• In the first phase, the backbone is created using the following algorithm:

createBackbone( root, n){

tmp = root;

while (tmp != NULL)

if tmp has a left child{

rotate this child around tmp;set tmp to the child which just became parent

}else

set tmp to its right child}

5

30

10

20

40

15

2823

25

tmp

5

30

10

20

40

15

2823

25

tmp

5

30

10

20

40

15

2823

25

tmp

5

30

10

20

40

15

2823

25

tmp

5

30

10

20

40

15

2823

25

tmp

5

30

10

20

40

15

28

23

25

tmp

5

30

10

20

40

15

28

23

25

tmp

5

30

10

20

40

15

28

23

25

tmp

5

30

10

20

40

15

28

23

25

tmp

5

30

10

20

40

15

28

23

25

tmp

5

30

10

20

40

15

28

23

25 tmp

5

30

10

20

40

15

28

23

25

tmp=NULL

5

30

10

20

40

15

28

23

25tmp

• In the second phase, the Backbone is transformed into a tree but this time the tree is perfectly balanced by having leaves on two adjacent levels

• In each pass down the backbone, every second node down to a certain point is rotated around its parent

• The algorithm is as follows:

createPerfectTree(n){

m = 2 lg(n+1) -1 Make n – m rotations starting from the top of the backbone

while (m >1){

m = m/2make m rotations starting from the top of the backbone

}}

5

20

10

15

23

25

30

40

28

n = 9m = 7

So we need to do n – m = 2 rotations

5 20

10

15 23

25

30

40

28

m = 7/2 = 3

So we need to do three rotations

20

25

30

28

235

10

15

40

25

20

23

30

28 40

5

10

15

m = 3/2 = 1

So we need to do one rotation

m = 1/2 = 0No more rotation is necessary

Step 1 Step 2 Step 3

Step 4

• Can you do this example now. 1

9

2

4

5

3

86

7 10

Some of the code in this lecture is placed in example 1 for the tree lecture in the web