Download - Abstract Data Types Stack, Queue Amortized analysis

1

Abstract Data TypesStack, Queue

Amortized analysis

2

ADT is an interface

• It defines– the type of the data stored– operations, what each operation does

(not how)– parameters of each operation

3

ADT

Application

Implementation of theData structure

חוזה בין מתכנת האפליקציה

ומיישם מבנה הנתונים

ממשק

4

Example: Stacks

• Push(x,S) : Insert element x into S• Pop(S) : Delete the last element inserted into

S• Empty?(S): Return yes if S is empty• Top(S): Return the last element inserted into

S• Size(S)• Make-stack()

5

The Stack Data Abstraction

push

push

push

6

The Stack Data Abstraction

push

push

push

pop

push

Last in,First out.

7

• Evaluate an expression in postfix or Reverse Polish Notation

Infix Postfix

(2+ 3) * 5 2 3 + 5 *

( (5 * (7 / 3) ) – (2 * 7) ) 5 7 3 / * 2 7 *-

A stack application

8

2 3 + 5 *

A stack application

23

9

2 3 + 5 *

A stack application

55

10

2 3 + 5 *

A stack application

25

11

S ← make-stack()while ( not eof ) do B ← read the next data;

if B is an operand then push(B,S) else X ← pop(S)

Y ← pop(S) Z ← Apply the operation B on X and Y push(Z,S)return(top(S))

Pseudo-code

12

Implementation

• We will be interested in algorithms to implement the ADT..

• And their efficiency..

13

Using an array

12 1 3A

A[0] A[1]

t

The stack is represented by the array A and variable t

A[2] A[N-1]

1213

14

Using an array

12 1 3A

A[0] A[1]

t

make-stack(): Allocates the array A, which is of some fixed size N, sets t ← -1

The stack is represented by the array A and variable t

A[2] A[N-1]

15

Operations

12 1 3A

t

size(S): return (t+1)

empty?(S): return (t < 0)

top(S): if empty?(S) then error else return A[t]

A[N-1]A[0] A[1]

A[2]

16

Pop

12 1 3A

t

pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)

A[N-1]A[0] A[1]

A[2]

pop(S)

17

Pop

12 1 3A

t

A[N-1]A[0] A[1]

A[2]

pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)

pop(S)

18

Push

12 1 3A

t

push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x

A[N-1]A[0] A[1]

A[2]

push(5,S)

19

Push

12 1 5A

t

push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x

A[N-1]A[0] A[1]

A[2]

push(5,S)

20

Implementation with lists

12 1 5

top

size=3

x.element

x.nextx

21


12 1 5

top

make-stack(): top ← null size ← 0

size=3

22

Operations

12 1 5

top

size(S): return (size)

empty?(S): return (top = null)

top(S): if empty?(S) then error else return top.element

size=3

23

Pop

12 1 5

top

pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)

pop(S)

size=3

24

Pop

12 1 5

top


pop(S)

size=2

25

Garbage collection

1 5

top


pop(S)

size=2

26

Push

1 5

topsize=2

push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)

27

Push

1 5

topsize=2


5

28

Push

1 5

topsize=2


5

29

Push

1 5

topsize=3


5

30

Analysis

• Bound the running time of an operation on the worst-case

• As a function of the “size”, n, of the data structure

• T(n) < 4n+7• Too detailed, we are just interested

in the order of growth

31

Big-O

) ו- כך ש:c - קיים ) ( ( ))T n O f n0n

0)()( nnnfcnT

דוגמא:2

( ) ( 1)T n n

)(4)1(222nOnn

32

Big-O

))(()( ngOnf

cg(n)

f(n)

n0

33

• 4n O(n2)

• 4n2 O(n2)

• 2n O(n10)

• 10 O(1)

• 100n3+10n O(n3)

• log2(n) O(log10(n))

More examples

34

The running time of our stack and queue operations

• Each operation takes O(1) time

35

Stacks via extendable arrays

• We do not want our implementation using arrays to be limited to only N elements

• When the array is full we will double its size

36

Push

push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x

12 1A

t

A[N-1]A[0] A[1]

37

Push


12 1 3 3A 4 5 7 3 2 8 1

A[N-1]A[0] A[1]

t

push(5,S)

38

Push


push(5,S)

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

39

Push

push(x,S): if size(S) = N then allocate a new array of size 2N copy the old array to the new one t ←t+1 A[t] ← x push(5,S)

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

12 1 3 3 4 5 7 3 2 8 1

t

40

Push


t

A[2N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

push(5,S)

41

Push


5

A[2N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

push(5,S)

42

Analysis

• An operation may take O(n) worst case time !

• But that cannot happen often..

43

Amortized Analysis

• How long it takes to do m operations ?

• Well, O(nm)

• Yes, but can it really take that long ?

46

x x

47

x x

x x x

48

x x

x x x x

49

x x

x x x x

x x x x x

50

x x

x x x x

x x x x x x

51

x x

x x x x

x x x x x x x

52

x x

x x x x

x x x x x x x x

53

x x

x x x x

x x x x x x x x

x x x x x x x x x

54

x x

x x x x

x x x x x x x x

x x x x x x x x x x

55

x x

x x x x

x x x x x x x x

x x x x x x x x x x x

56

x x

x x x x

x x x x x x x x


Only after z-1 pushes that cost 1 we will have a push that costs 2z+1

Let z be the size of the array we just copied

57

x x

x x x x

x x x x x x x x


3+(2∙4+1)=3∙4

7+(2∙8+1)= 3∙8

Total cost O(m) !!

Start from the second row: z=2

1+(2∙2+1)=3∙2

z=4 z=8

58

Theorem: A sequence of m operations on a stack takes O(m) time

proof.

59

Amortized Analysis (The bank’s view)

• We will have a bank

• An operation can either pay a token for a unit of work by itself, or take a token from the bank to pay for it

• An operation can put tokens in the bank

• If the bank never has a negative balance then the total # of tokens spent by the operations bounds the total work

60

For a proof:

• Define exactly how each operation pays for its work, and how many tokens it puts in the bank

• Prove that the balance in the bank is always non-negative

• Count the total # of tokens – that’s your bound

61

“Easy” push

• when we push an item, we also put ≤ two tokens, one on the item which we insert, and one on another item if there is some item without a token

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

62

5

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

“Easy” push

• when we push an item, we also put ≤ two tokens, one on the item which we insert, and one on another item if there is some item without a token

63

5 6

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

64

5 6 6 7 1 10 4 67 2 5 7

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

65

“hard” push

• Tokens from the bank “pay” for copying the array into the larger array

5 6 6 7 1 10 4 67 2 5 7

A[N-1]

12 1 3 3A 4 5 7 3 2 8 1

A[0] A[1]

t

66

• Then you pay a token and put two in the bank as an “easy” push

5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 A

t

5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4

t

“hard” push

67

Need to prove

• The balance is never negative:

• When we get to an expensive push there are enough tokens to pay for copying

• By induction: prove that after the i-th push following a copying there are 2i tokens in the bank

68

How many tokens we spent ?

• Each operation spent 3 tokens

69

Summary

• So the total # of tokens is 3mTHM: A sequence of m operations takes

O(m) time !! (we finished the proof)

• Each operation takes O(1) amortized time

70

Theorem: A sequence of m operations on a stack takes O(m) time

proof (2) .

• We formalize the bank as a potential function

71

Amortized(op) = actual(op) +

Define

Define: a potential function

72

Amortized(op1) = actual(op1) + 1- 0


Amortized(opn) = actual(opn) + n- (n-1)

……

+

iAmortized(opi) = iactual(opi) + n- 0

iAmortized(opi) iactual(opi) if n- 0 0

73

Example: Our extendable arrays

Define a potential of the stack

2( 1) 2( 1) 0( )

0

t N if t NS

Otherwise

74

Amortized(push) = actual(push) + =

1 + 2(t+2) – N - (2(t+1) – N) = 3

Amortized(push) = N + 1 + =

N + 1 + (2 - N) = 3

Amortized(pop) = 1 + =

1 + 2(t-1) – N - (2t – N) = -1

With copying:

Without copying:

75



Amortized(opn) = actual(opn) + n- (n-1)

……

+

iAmortized(opi) = iactual(opi) + n- 0

iAmortized(opi) iactual(opi) if n- 0 03m ≥

76

Summary

• So the total # of tokens is 3m THM: A sequence of m operations

starting from an empty stack takes O(m) time !!

• Each operations take O(1) amortized time

77

Queue

• Inject(x,Q) : Insert last element x into Q• Pop(Q) : Delete the first element in Q• Empty?(Q): Return yes if Q is empty• Front(Q): Return the first element in Q• Size(Q)• Make-queue()

78

The Queue Data Abstraction

inject

inject

79

The Queue Data Abstraction

inject

inject

inject

inject

popFirst in,First out (FIFO).

80

Using an array

12 1 4 2A 5

A[0] A[1]

t

pop(Q)

A[2] A[N-1]

81

Using an array

1 4 2 5A

A[0] A[1]

t

pop(Q)

A[2] A[N-1]

82

Using an array

1 4 2 5A

A[0] A[1]

t

A[2] A[N-1]

This would be inefficient if we insist that elements span a prefix of the array

83

Using an array

12 1 4 2A 5

A[0] A[1]

r

A[2] A[N-1]

f

A

A[0] A[1]

r

A[2]

f

Empty queue f=r

84

Using an array

12 1 4 2A 5

A[0] A[1]

r

pop(Q)

A[2] A[N-1]

f

85

Using an array

1 4 2A 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)

86

Using an array

1 4 2A 5 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)inject(5,Q)

87

Using an array

1 4 2A 5 5 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)

88

Using an array

2A 5 5 5

A[0] A[1]

A[2] A[N-1]

f r

pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

89

Using an array

A 5 5 5 5

A[0] A[1]

r

A[2] A[N-1]

f

pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

90

Make the array “circular”

5 5 A 5 5

A[0] A[1]

r

A[2] A[N-1]

f

Pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

91

Operations

empty?(Q): return (f = r)

top(Q): if empty?(Q) then error else return A[f]

1 4 2A 5

A[0] A[1]

A[2] A[N-1]

f r

92

Operations

size(Q): if (r >= f) then return (r-f) else return N-(f-r)

1 4 2A 5

A[0] A[1]

A[2] A[N-1]

f r

93

Operations

size(Q): if (r >= f) then return (r-f) else return N-(f-r)

5 5 A 5 5

A[0] A[1]

r

A[2] A[N-1]

f

94

Pop

pop(Q)

1 4 2A 5

A[0] A[1]

A[2]

f r

pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)

95

Pop

pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)

pop(Q)

1 4 2A 5

A[0] A[1]

A[2]

f r

96

Inject

inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N

inject(5,Q)

4 2A 5

A[0] A[1]

A[2]

f r

97

Inject

inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N

inject(5,Q)

4 2A 5 5

A[0] A[1]

A[2]

f r

98


12 1 5

headsize=3

tail

inject(4,Q)

99


12 1 5

headsize=3

tail

inject(4,Q)

4

100


12 1 5

headsize=3

tail

inject(4,Q)

4

Complete the details by yourself

101

Double ended queue (deque)

• Push(x,D) : Insert x as the first in D• Pop(D) : Delete the first element of D• Inject(x,D): Insert x as the last in D• Eject(D): Delete the last element of D• Size(D)• Empty?(D)• Make-deque()

102

Implementation with doubly linked lists

5

head

size=2

tail

13

x.next

x.element

x.prev

x

103

Empty list

head

size=0

tail

We use two sentinels here to make the code simpler

104

Push

push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1

5

head

size=1

tail

105


5

head

size=1

tail

push(4,D)

4

106


5

head

size=1

tail

push(4,D)

4

107


5

head

size=1

tail

push(4,D)

4

108


5

head

size=2

tail

push(4,D)

4

109

Implementations of the other operations are similar

• Try by yourself