1. (8 points) Goal: Practice analysis of algorithms. Consider the algorithm represented by the following program fragment.

MYSTERY(int x, int n)

int sum=0; if (x<=100) then

while( x > 10 ) x= x­1;

else for(int i=1; i<=n; i++)

sum=sum+i+x;

return x + sum;

a) (2 points) What value (as a function of x and n) does MYSTERY compute? b) (2 points) Assume that the value of x is constant, but unknown. Use only arithmetic operations as primitive operations. Compute the worst­case running time of MYSTERY as a function of x and n. c) (2 points) Assume that x remains constant while n goes to infinity. Derive a tight, big­Oh expression (dependent on x) for the running time of MYSTERY. Justify your solution. d) (2 points) Assume that n remains constant while x goes to infinity. Derive a tight, big­Oh expression (dependent on n) for the running time of MYSTERY. Justify your solution. Solution: a) (x, ) x for x 0 10 for 10 x 100 (n ) otherwise f n = < 1 2

n(n+1) + x + 1 b.) We count the following arithmetic operations:

i. x = x 1 ii. i++ iii. sum = sum + i + x iv. x + sum

Then the worst case running time would be: (f (x, )) 1 for x 0 x for 10 x 100 2n otherwise T n = < 1 9 + 1

c) (f (x, )) O(1) for x 100 O(n) otherwise T n = d) (f (x, )) (1)T n = O

Question 2: Solution: a. The running time is determined by the for loop from lines H2 to line H4.

Asymptotically we get a running time of Θ(n).

H1 y = 0;

H2 for (i = n; i >= 0; i­­)

H3 y *= x;

H4 y += a[i];

b. More naive pseudocode.

N1 y = 0;

N2 for (i = n; i >= 0; i­­)

N3 z = a[i];

N4 for (j = 0; j < i; j++)

N5 z *= x;

N6 y += z;

The running time is dominated by the nested for loop in lines N4 and N5. Asymptotically,

we get a running time of Θ(n 2 ). This is a larger asymptotic running time than Horner’s rule.

c.

Initialization: Prior to the first loop iteration, we have i = n, so that the sum goes from k=0 to k=n­(n+1)=­1, and it does not contain any term and so equals to 0. This is

identical to the initialization value of y.

Maintenance: Assume that before the iteration i ­> i­1 we have

In line H3 we multiply y by x, resulting in

In line H4 we add a[i] to y, resulting in

This equates to

Hence the loop invariant is preserved at the beginning of the next iteration with i­1.

Termination: The algorithm terminates when i = ­1. The loop invariant guarantees that at this time we have

d. Firstly, line H1 guarantees that the loop invariant is initialized correctly for i = n. When

the loop invariant is maintained while the loop in line H2 simply counts down until the

algorithm terminates with i = ­1. At this time, the loop invariant guarantees that y is

identical to the value of the polynome we had to evaluate, i.e.

3. Purpose: practice working with asymptotic notation. Please solve (12 points)

3­2 on page 61 , (8 points) 3­4 [a­c] on page 62. For full marks justify your

solutions.

3­2

Solution: sorry for the bad display of formulas in the table.

A B O O Ω Ω Θ

A Lgkn Nεk Yes No No No No

B Nk Cn Yes Yes No No No

C sqrt(n

)

nsin(n

)

No No No No No

D 2n 2n/2 No No Yes Yes No

E nlgc Clgn Yes No Yes No Yes

F lg(n!) lg(nn) Yes No Yes No Yes

a) pply LHospital s ruleA ′ ′

nlgk = (log n)2k = ( )ln 2

ln n k

lim lim limn→∞ nεlg nk = ( )1

ln2k

n→∞ nε(lnn)k = ( )1

ln2k

n→∞ εnεk(lnn)k 1

lim lim = ( )1ln2

kn→∞

k!ε nk ε = k!

(ln2) εk k n→∞1nε = 0

b) pply LHospital s ruleA ′ ′

lim limn→∞ cnnk

= limn→∞ c lncnknk 1

= limn→∞k!

c (lnc)n k = k!(lnc)k n→∞

1cn = 0

c) The value of sin n fluctuates between ­1 and 1. Therefore, this function is not

comparable.

d) pply LHospital s ruleA ′ ′

2 limn→∞2n

2n/2 = limn→∞n/2 = ∞

e) nlgc = clgn

f) , lg(g(n!) g(n)l =n

i=1l ) lgnnn = n

, lg(n!) is Big­oh of lg( .ecause g(n) nlgnBn

i=1l )nn

Because , lg(n!) is Big­omega of lg( .g(i) g(i) g ( ) )lg( )n

i=1l

n

i= +12n

ln

i= +12n

l 2n = ( 2

n 1 2n )nn

3­4

Solution:

a) f(n)=O(g(n)) implies g(n)=O(f(n))

Let f(n)=n and g(n)= , then we have n=O( but O(n).n2 )n2 ≠n2

=> False

b) f(n)+g(n)= (min(f(n), g(n)))θ

Let f(n)=n and g(n)= , then min(f(n), g(n))= .n 2 n , n n, n 1 2 < 1

Therefore, for , we have min(f(n), g(n))=n= .1n θ(n)

However, f(n)+g(n)=n+ .≠ θ(n)n2

=> False

c) f(n)=O(g(n)) implies lg(f(n))=O(lg(g(n))), where lg(g(n)) and f(n) 1 for all sufficiently a

large n.

f(n)=O(g(n)) implies that for some c>0 and we have n 0,n 0 f (n) cg(n).0

If is true, then ) is true and also is f (n) cg(n)0 g(f (n)) lg(cg(n)l g (f (n)) lgc g(g(n))l + l

true.

Since g (g(n)) 1,l gc g (g(n)) lgc.l * l

g (f (n)) lgc g (g(n)) lgc g (g(n)) g (g(n)) 1 gc)lg(g(n))l + l * l + l = ( + l

Therefore, .g(f (n)) (lg (g(n)) )l = O

=> True

4. Purpose: more practice. The following 10 functions are comparable by

asymptotic growth.

(Here, lg := logarithm base 2)

4 lg(n), n3 , n2 +nlg5 (n), n2 ­n+5, n2 lg(n), n3 lg3 (n), sqrt( n), n/2, lg( n), 2 ­ n a) (5 points) Put them in increasing order of asymptotic growth.

Solution:

The order would be: 2 ­ n , lg( n ), sqrt( n ), n /2, n 2 ­ n +5, 4 lg(n ), n 2 +n lg 5 ( n ), n 2 lg( n ), n 3 ,

n 3 lg 3 ( n ) b) (2 point) The order is NOT unique. Which functions can be exchanged?

Solution:

4 lg(n )=2 2*lg(n) = 2 lg(n)*2 =n 2 , So, 4 lg( n ) , n 2 +n lg 5 ( n ), n 2 ­ n +5 can be exchanged.

5. Purpose: reinforce your understanding of sorting algorithms and lowerbounds. Please analyze the number of comparisons needed by InsertionSort(2bonus points), Merge Sort (2 bonus points), and Heap Sort (2 bonuspoints) in the special case when all the keys are 0’s and 1’s. Assume thatthe algorithm in question does not know that the keys are 0’s and 1’s, thatit behaves just as it would for any other set of keys. In each case, give abig-theta bound for the worst-case number of comparisons for n → ∞ ineach of three situations:

• The number of 0’s is a constant c > 0;

• The number of 1’s is a constant c > 0; and

• The number of 0’s is ⌊n/2⌋ and the number of 1’s is ⌈n/2⌉

(a) Insertion Sort

The number of comparisons for insertion sort is less or equal to n−1+I and greater or equal to I, where I is the number of inversions in thearray. Each inversion forces a comparison: ifa[i] > a[j] and i < j,then a[i] and a[j] will be compared when a[j] is inserted. And thereare at most n − 1 additional comparisons that are performed thenan element has found it correct place and is not the first element.When there are only 0’s and 1’s, the maximum number of inversionsoccurs when all the 1’s precede all the 0’s. In this case, the numberof inversions is k ∗ (n − k), where k is the number of 0’s. Take thefollowing example :

1 1 1 1 1 0

The zero on the right has to be compared with all the existing 1s inthe worst case. Here to get the sequence 0 1 1 1 1 1 it would require5 comparisons in total.

If the sequence was :

1 1 1 1 1 0 0 the total number of comparisons would be 5+6 (for eachzero) = 11 with number of inversions= 10.

• The number of 0’s is a constant c > 0;c ∗ (n− c) ∈ θ(n)

• The number of 1’s is a constant c > 0;(n− c) ∗ c ∈ θ(n)

• The number of 0’s is n/2 and the number of 1’s is n/2bn/2 ∗ cdn/2 ∈ θ(n2)

(b) Merge SortIn our implementation, merging two lists with a total of n elements,requires n−1 compares, independent of the specific elements. Hence,the number of comparisons at each level of the recursion tree is θ(n)for a total of θ(nlgn) comparisons.

1

(c) Heap SortThe MakeHeap phase of Heapsort does not have any impact on theasymptotic number of comparisons. The sorting phase accounts forthe following scenarios.

• The number of 0’s is a constant c > 0;Each time a 0 becomes the root, it moves down one step ata time towards the leftmost position that did not have a 0 be-fore. In other words, it continues to have only 0’s to the leftof it on its level. This also means that every time it tricklesdown its depth decreases by 1: the 0’s to the left of it keep itfrom coming back to the same level, and if it is the last 0 onits level, the depth of the tree decreases. So each 0 can move tothe root at most lg(n) times (the number of times its depth candecrease) and take at most 2 ∗ lg(n) comparisons heapify. Anytime a 1 moves to the root there are 2 comparisons. Thus tothe total number is bounded by: 2 ∗ c ∗ lg2n + (n − c) ∈ θ(n).

• The number of 1’s is a constant c > 0;The first c iterations of Heapsort’s main loop remove the 1’sfrom the heap, now consisting of only 0’s. A heap with all 0’stakes 2 comparisons per iteration. Therefore the answer is θ(n)

• The number of 0’s is n/2 and the number of 1’s is n/2 .The upper bound is O(nlg(n)).To find the lower bound:Let us take an example where the last row has all 0’s and everynon leaf node is a 1. MakeHeap will not change this in anyway. Assume n = 2x − 1. After the first 2x−2 iterations, halfof the bottom row which consisted of 0’s will end up as partof the subheap on the far left side. Almost half of these willbe in the last but one row of the subheap: the subheap having2x−2 leaves and 2x−3 parents of leaves. Each of the 2x−30′sthat end up as parents of leaves in the subheap traverses pastx − 3 pairs of 1’s. As a result, just these 2x−3 0’s account for2(x−2)2x−3 = (x−3)2x−2 comparisons, which is Ω(nlg(n)) witha suitable small constant. If n 6= 2x−1 we can simply apply thisargument to the largest x for which 2x − 1 < n. Any additional0’s will only increase the relative number of comparisons. Thetight bound is therefore θ(nlg(n)).

6. (1 point) Purpose: even more practice. Show en ∈ ω(nt)

Using L’Hospital’s Rule:

limn→+∞

en

nt= lim

n→+∞

en

t!=∞.

2

7. (8 points) Purpose: practice algorithm design and algorithm analysis.Give pseudocode for a θ(logn) algorithm which computes an, given a > 0and integer n. Justify the asymptotic running time of your algorithm. Donot assume that n is a power of 2.

Algorithm 1 Iterative algorithm to calculate power(a,n)

1: procedure FastPower(a,n)2: result← 13: if n == 0 then

4: return result5: power of 2← a6: k← ⌊lg(n)⌋+ 17: for i=1 to k do

8: if n is even then

9: n← n/210: else

11: n← (n− 1)/212: result← result ∗ power of 2

13: power of 2← power of 2 ∗ power of 2return result

This algorithm calculates an in ⌊lg(n)⌋ + 1 iterations of the for loop.Since the number of operations in each iteration is bound by a constant,the algorithm has (log(n)) worst-case running time. Assume n = ak2

k +ak−12

k−1 + . . . + a020 with aj ∈ 0, 1

The algorithm calculates a(a020),a(a02

0+a121),... a(ak2

k+ak−12k−1+. . .+a02

0)

and stores these values in the variable result. For example,for n = 5 =1 ∗ 22 + 0 ∗ 21 + 1 ∗ 20, algorithm FastPower(a, 5) computes successively:Initialization:

result = 1, n = 5, power of 2 = afor-loop:i = 1, result = a, n = 2, power of 2 = a2

i = 2, result = a3, n = 1, power of 2 = a4

i = 3, result = a5, n = 0, power of 2 = a8 The final value is in the result

variable as shown above.

In an alternate recursive version the recurrence could be formulated as:

T (n) =

θ(1), if n = 1 or n = 0

T (n2 ) + θ(1), otherwise(1)

This can be solved using Master’s Theorem with a = 1, b = 2–Case 2.

3