Algorithm Design and Analysis

Summer 2018 Amo G. Tong 1

Lecture 4Randomized Algorithms• Randomized Quick Sort

• Randomized Selection

Summer 2018 Amo G. Tong 2

• Quick Sort:• Take an element 𝑥 = 𝐴[𝑞] in 𝐴[1,… , 𝑛].

• Partition into two subarrays 𝐴[1,… , 𝑞 − 1] and 𝐴[𝑞 + 1,… , 𝑛] such that each element of 𝐴[1,… , 𝑞 − 1] is less than or equal to 𝑥 which is less than or equal to each element of 𝐴[𝑞 + 1,… , 𝑛].

• Sort the subarrays recursively.

• No need to combine.

Randomized Algorithms

Summer 2018 Amo G. Tong 3

Input: an array 𝐴[1,… , 𝑛] of numbers

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 4

Input: an array 𝐴[1,… , 𝑛] of numbers

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 5

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 1 𝑖 = 0, exchange 5 with 1

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 6

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 10 9 1 2 5

𝑖 = 0, exchange 5 with 1

𝑖 = 1, exchange 0 with 0

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 7

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 10 9 1 2 50 9 1 2 5

𝑖 = 0, exchange 5 with 1

𝑖 = 1, exchange 0 with 0

𝑖 = 1

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 8

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 5

𝑖 = 0, exchange 5 with 1

𝑖 = 1, exchange 0 with 0

𝑖 = 1

𝑖 = 2, exchange 9 with 1

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 9

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 50 1 9 2 5

𝑖 = 0, exchange 5 with 1

𝑖 = 1, exchange 0 with 0

𝑖 = 1

𝑖 = 2, exchange 9 with 1

𝑖 = 3, exchange 9 with 2

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 10

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 50 1 9 2 50 1 2 9 5

𝑖 = 0, exchange 5 with 1

𝑖 = 1, exchange 0 with 0

𝑖 = 1

𝑖 = 2, exchange 9 with 1

𝑖 = 3, exchange 9 with 2

exchange 9 with 5

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 11

Input: an array 𝐴[1,… , 𝑛] of numbers

0 9 5 2 10 9 1 2 50 9 1 2 50 9 1 2 50 1 9 2 50 1 2 9 50 1 2 5 9

𝑖 = 0, exchange 5 with 1

𝑖 = 1, exchange 0 with 0

𝑖 = 1

𝑖 = 2, exchange 9 with 1

𝑖 = 3, exchange 9 with 2

exchange 9 with 5

Return 4

0 1 2 5 9

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 12

Input: an array 𝐴[1,… , 𝑛] of numbers

QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =PARTITION(A, p, r)3 QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

0 9 5 2 1

0 1 2 5 9

• Quick Sort:

Randomized Algorithms

Summer 2018 Amo G. Tong 13

Input: an array 𝐴[1,… , 𝑛] of numbers

QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =PARTITION(A, p, r)3 QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end

PARTITION (𝐴, 𝑝, 𝑟)1 select an element 𝑥 = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟]2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

Which element should we take?

• Quick Sort:• Take an element 𝑥 in 𝐴[1,… , 𝑛].

• Partition into two subarrays 𝐴1[1, … , 𝑎] and 𝐴2[1, … , 𝑏] such that each element of 𝐴1 is less than or equal to x which is less than or equal to each element of 𝐴2. (𝑎 + 𝑏 = 𝑛 − 1)

• Sort the subarrays recursively.

• No need to combine.

• 𝑥 decides the size of the two subarrays.• 𝑇(𝑛) = 𝑇(𝑎) + 𝑇(𝑏) + Θ(𝑛);

Randomized Algorithms

Summer 2018 Amo G. Tong 14

• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .

• If the array is always evenly partitioned:• 𝑥=median

Randomized Algorithms

Summer 2018 Amo G. Tong 15

• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .

• If the array is always evenly partitioned:• 𝑥=median

• 𝑇 𝑛 = 𝑇 𝑛/2 + 𝑇 𝑛/2 + Θ 𝑛 .

• 𝑇(𝑛) = Θ(𝑛 log 𝑛)

Randomized Algorithms

Summer 2018 Amo G. Tong 16

• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .

• If the array is always evenly partitioned:• 𝑥=median

• 𝑇 𝑛 = 𝑇 𝑛/2 + 𝑇 𝑛/2 + Θ 𝑛 .

• 𝑇(𝑛) = Θ(𝑛 log 𝑛)

• If the array is always partitioned with a fixed proportion 𝒂:• x= the 𝑎𝑛 -th smallest element

• 𝑇(𝑛) = 𝑇(𝑎𝑛) + 𝑇((1 − 𝑎)𝑛) + Θ(𝑛) where 0 < 𝑎 < 1;

• 𝑇(𝑛) = Θ(𝑛 log 𝑛)

Randomized Algorithms

Summer 2018 Amo G. Tong 17

• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .

• If one of the subarray is always empty:• 𝑥= the maximum or the minimum

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = Θ(𝑛2)

Randomized Algorithms

Summer 2018 Amo G. Tong 18

• 𝑥 decides the size of the two subarrays.• 𝑇 𝑛 = 𝑇 𝑎 + 𝑇 𝑏 + Θ 𝑛 .

• If one of the subarray is always empty:• 𝑥= the maximum or the minimum

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = Θ(𝑛2)

• If one of the subarray has a constant length 𝒂:• x= the a-th smallest element

• 𝑇(𝑛) = 𝑇(𝑛 − 1 − 𝑎) + 𝑇(𝑎) + Θ(𝑛) where 0 < 𝑎 < 𝑛;

• 𝑇(𝑛) =? ? (try it yourself)

Randomized Algorithms

Summer 2018 Amo G. Tong 19

• If we always select the last element as the pivot:

Randomized Algorithms

Summer 2018 Amo G. Tong 20

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

Randomized Algorithms

Summer 2018 Amo G. Tong 21

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

• Expected running time:• Assume each of the possible permutations appears with the same

probability.

Randomized Algorithms

Summer 2018 Amo G. Tong 22

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

• Expected running time:• Assume each of the possible permutations appears with the same

probability.

• With probability 1/n, the last element is the 𝒊-th smallest.

Randomized Algorithms

Summer 2018 Amo G. Tong 23

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

• Expected running time:• Assume each of the possible permutations appears with the same

probability.

• With probability 1/n, the last element is the 𝑖-th smallest.

• If the last element is the 𝑖-th smallest, 𝑇 𝑛 = T(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛

Randomized Algorithms

Summer 2018 Amo G. Tong 24

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

• Expected running time:• Assume each of the possible permutations appears with the same

probability.

• With probability 1/n, the last element is the 𝑖-th smallest.

• If the last element is the 𝑖-th smallest, 𝑇 𝑛 = T(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛

• The expected value is

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑛 − 𝑖 ] + 𝑐𝑛)

Randomized Algorithms

Summer 2018 Amo G. Tong 25

Randomized Algorithms

Summer 2018 Amo G. Tong 26

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.

Randomized Algorithms

Summer 2018 Amo G. Tong 27

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Randomized Algorithms

Summer 2018 Amo G. Tong 28

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.

Randomized Algorithms

Summer 2018 Amo G. Tong 29

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

Randomized Algorithms

Summer 2018 Amo G. Tong 30

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

𝑬[𝑻(𝒌)] = σ𝒊=𝟏𝒌 𝟏

𝒌(𝑬[𝑻 𝒊 − 𝟏 ] + 𝑬[𝑻 𝒌 − 𝒊 ] + 𝒄𝒌)

Randomized Algorithms

Summer 2018 Amo G. Tong 31

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1

𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)

=𝟏

𝒌(𝟐σ𝒊=𝟎

𝒌−𝟏𝑬[𝑻 𝒊 ] + 𝒄𝒌𝟐)

Randomized Algorithms

Summer 2018 Amo G. Tong 32

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1

𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)

=1

𝑘(2σ𝑖=0

𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)

=𝟏

𝒌(𝟐σ𝒊=𝟐

𝒌−𝟏𝑬[𝑻 𝒊 ] + 𝟐𝑻(𝟎) + 𝟐𝑻(𝟏) + 𝒄𝒌𝟐)

Randomized Algorithms

Summer 2018 Amo G. Tong 33

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1

𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)

=1

𝑘(2σ𝑖=0

𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)

=1

𝑘(2σ𝑖=2

𝑘−1𝐸[𝑇 𝑖 ] + 2𝑇(0) + 2𝑇(1) + 𝑐𝑘2)

≤𝟏

𝒌(𝟐σ𝒊=𝟐

𝒌−𝟏𝒃 𝒊 𝐥𝐨𝐠 𝒊) + (𝟒𝒂 + 𝒄𝒌𝟐)/𝒌

Randomized Algorithms

Summer 2018 Amo G. Tong 34

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1

𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)

=1

𝑘(2σ𝑖=0

𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)

=1

𝑘(2σ𝑖=2

𝑘−1𝐸[𝑇 𝑖 ] + 2𝑇(0) + 2𝑇(1) + 𝑐𝑘2)

≤2𝑏

𝑘(σ𝑖=2

𝑘−1 𝑖 log 𝑖) + (4𝑎 + 𝑐𝑘2)/𝑘

(see additional reading material)

≤2b

𝑘(

𝑘2

2log 𝑘 −

𝑘2

4+

1

4) + (4𝑎 + 𝑐𝑘2)/𝑘

Randomized Algorithms

Summer 2018 Amo G. Tong 35

We need this to be ≤ 𝑏 𝑘 log 𝑘.

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝑇(𝑖 − 1) + 𝑇(𝑛 − 𝑖) + 𝑐𝑛)⇒ 𝐸 𝑇 𝑛 = O(n log n)

Proof: Let 𝑇 𝑘 ≤ 𝑎 for 𝑘 < 2.Inductive hypothesis: 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log𝑛

Basic: 𝐸[𝑇(2)] =1

2(𝑇(0) + 𝑇(1) + 𝑐 ⋅ 2) +

1

2(𝑇(1) + 𝑇(0) + 𝑐 ⋅ 2) = 2𝑎 + 2𝑐.

We need 2𝑎 + 2𝑐 < 2𝑏.Induction: Suppose 𝐸 𝑇 𝑛 ≤ 𝑏 𝑛 log 𝑛 for all 2 < 𝑛 < 𝑘.

𝐸[𝑇(𝑘)] = σ𝑖=1𝑘 1

𝑘(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑘 − 𝑖 ] + 𝑐𝑘)

=1

𝑘(2σ𝑖=0

𝑘−1𝐸[𝑇 𝑖 ] + 𝑐𝑘2)

=1

𝑘(2σ𝑖=2

𝑘−1𝐸[𝑇 𝑖 ] + 2𝑇(0) + 2𝑇(1) + 𝑐𝑘2)

≤2𝑏

𝑘(σ𝑖=2

𝑘−1 𝑖 log 𝑖) + (4𝑎 + 𝑐𝑘2)/𝑘

(see additional reading material)

≤2b

𝑘(

𝑘2

2log 𝑘 −

𝑘2

4+

1

4) + (4𝑎 + 𝑐𝑘2)/𝑘

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

• Expected running time:• Assume each of the possible permutations appears with the same

probability.

• The expected running time is

𝐸 𝑇 𝑛 = Θ(𝑛 log 𝑛)

Randomized Algorithms

Summer 2018 Amo G. Tong 36

• If we always select the last element as the pivot:• Worst-case running time:

• 𝑇(𝑛) = 𝑇(𝑛 − 1) + 𝑇(0) + Θ(𝑛)

• 𝑇(𝑛) = 𝑂(𝑛2)

• Expected running time:• Assume each of the possible permutations appears with the same

probability.

• The expected running time is

𝐸 𝑇 𝑛 = Θ(𝑛 log 𝑛)

Randomized Algorithms

Summer 2018 Amo G. Tong 37

This is not always true.We want a Θ(𝑛 log 𝑛)algorithm which does not rely on this assumption

• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.

Randomized Algorithms

Summer 2018 Amo G. Tong 38

• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.

• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.

• With probability 1/𝑛, the 𝑖-th smallest element is selected.

Randomized Algorithms

Summer 2018 Amo G. Tong 39

• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.

• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.

• With probability 1/𝑛, the 𝑖-th smallest element is selected.

Randomized Algorithms

Summer 2018 Amo G. Tong 40

RANDOM-PARTITION (𝐴, 𝑝, 𝑟)1 randomly select an element x = 𝐴[𝑘] in 𝐴[𝑝. . 𝑟],

where each element has the same probability to be selected.2 exchange A[k] with A[r];3 𝑖 = 𝑝 − 14 for 𝑗 = 𝑝 to 𝑟 − 15 if 𝐴 𝑗 ≤ 𝑥6 𝑖 = 𝑖 + 17 exchange 𝐴[𝑖] with 𝐴[𝑗]8 end9 end10 exchange 𝐴[𝑖 + 1] with 𝐴[𝑟]11 return 𝑖 + 1;

• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.

• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.

• With probability 1/𝑛, the 𝑖-th smallest element is selected.

Randomized Algorithms

Summer 2018 Amo G. Tong 41

QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =PARTITION(A, p, r)3 QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end

RANDOM-QUICKSORT (𝐴, 𝑝, 𝑟)1 if 𝑝 < 𝑟2 𝑞 =RANDOM-PARTITION(A, p, r)3 RANDOM-QUICKSORT (𝐴, 𝑝, 𝑞 − 1);4 RANDOM-QUICKSORT (𝐴, 𝑞 + 1, 𝑟);5 end

• Randomized Quick Sort;• Select the pivot uniformly at random from the given array.

• With probability 1/𝑛, 𝐴[𝑖] is selected as the pivot.

• With probability 1/𝑛, the 𝑖-th smallest element is selected.

• Expected running time:• No assumption on the input.

• The expected running time is

𝐸 𝑇 𝑛 = σ𝑖=1𝑛 1

𝑛(𝐸[𝑇 𝑖 − 1 ] + 𝐸[𝑇 𝑛 − 𝑖 ] + 𝑐𝑛)

Randomized Algorithms

Summer 2018 Amo G. Tong 42

The same as that in page 24.

• Selection:

Randomized Algorithms

Summer 2018 Amo G. Tong 43

Input: an array 𝐴[1,… , 𝑛] of distinct numbers and an integer 𝑖, 1 ≤ 𝑖 ≤ 𝑛.

Output: the 𝑖-th smallest element in 𝐴[1,… , 𝑛] .

• Selection:

Randomized Algorithms

Summer 2018 Amo G. Tong 44

Pseudocode:RANDOMIZED-SELECT(𝐴, 𝑝, 𝑟, 𝑖)1 if 𝑝 == 𝑟2 return A[p]3 end4 q=RANDOM-PARTITION(𝐴, 𝑝, 𝑟)5 𝑘 = 𝑝 − 𝑞 + 1;6 if 𝑖 == 𝑘7 return 𝐴[𝑞]8 end9 if 𝑖 < 𝑘10 return RANDOM-SELECT(𝐴, 𝑝, 𝑞 − 1, 𝑖)11 else12 return RANDOM-SELECT(𝐴, 𝑞 + 1, 𝑟, 𝑖 − 𝑘)13 end

• Selection:

Randomized Algorithms

Summer 2018 Amo G. Tong 45

Pseudocode:RANDOMIZED-SELECT(𝐴, 𝑝, 𝑟, 𝑖)1 if 𝑝 == 𝑟2 return A[p]3 end4 𝑞 =RANDOM-PARTITION(𝐴, 𝑝, 𝑟)5 𝑘 = 𝑝 − 𝑞 + 1;6 if 𝑖 == 𝑘7 return 𝐴[𝑞]8 end9 if 𝑖 < 𝑘10 return RANDOMIZED-SELECT(𝐴, 𝑝, 𝑞 − 1, 𝑖)11 else12 return RANDOMIZED-SELECT(𝐴, 𝑞 + 1, 𝑟, 𝑖 − 𝑘)13 end

Θ(𝑛)

𝑇(max(𝑞 − 𝑝, 𝑟 − 𝑞))

T(𝑛)

• Randomized selection;• Select the pivot uniformly at random from the given array.

• With probability1/𝑛, 𝐴[𝑖] is selected as the pivot.

• With probability1/𝑛, the 𝑖-th smallest element is selected.

• Running time:• If the 𝑖-th smallest element is selected as the pivot, the running time is

𝑇 𝑛 = 𝑇 max 𝑖 − 1, 𝑛 − 𝑖 + Θ(𝑛)

• So the expected running time is:

𝐸 𝑇 𝑛 =

𝑖=0

𝑛−11

𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) )

Randomized Algorithms

Summer 2018 Amo G. Tong 46

Randomized Algorithms

Summer 2018 Amo G. Tong 47

𝐸 𝑇 𝑛

=

𝑖=1

𝑛1

𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) )

=

𝑖=1

𝑛/2 −11

𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) ) +

𝑖= 𝑛/2 −1

𝑛1

𝑛(𝐸[𝑇 max 𝑖 − 1, 𝑛 − 𝑖 ] + Θ(𝑛) )

=

𝑖=1

𝑛/2 −11

𝑛(𝐸[𝑇 𝑛 − 𝑖 ] + Θ(𝑛) ) +

𝑖= 𝑛/2

𝑛1

𝑛(𝐸[𝑇 𝑖 − 1 ] + Θ(𝑛) )

=

𝑖= 𝑛/2

𝑛2

𝑛(𝐸[𝑇 𝑖 − 1 ] + Θ(𝑛) )

𝐸 𝑇 𝑛 =

𝑖= 𝑛/2

𝑛2

𝑛(𝐸[𝑇 𝑖 − 1 ] + Θ(𝑛) )

Prove 𝐸[𝑇(𝑛)] = 𝑂(𝑛)

• Expected running time:• The algorithm is not randomized. The expectation is taken over the

distribution of the inputs.

Randomized Algorithms

Summer 2018 Amo G. Tong 48

• Expected running time:• The algorithm is not randomized. The expectation is taken over the

distribution of the inputs.

• The algorithm is randomized. The expected running time is 𝑶(𝒇(𝒏)) for any given input.

Randomized Algorithms

Summer 2018 Amo G. Tong 49

• Expected running time:• The algorithm is not randomized. The expectation is taken over the

distribution of the inputs.

• The algorithm is randomized. The expected running time is 𝑂(𝑓(𝑛)) for any given input.

• The algorithm is randomized and the input is also sampled.

Randomized Algorithms

Summer 2018 Amo G. Tong 50

• Expected running time:• The algorithm is not randomized. The expectation is taken over the

distribution of the inputs.

• The algorithm is randomized. The expected running time is 𝑂(𝑓(𝑛)) for any given input.

• The algorithm is randomized and the input is also sampled.• Consider the Randomized Quick Sort with an input uniformly sampled

from {all the permutations over {1, … , 𝑛}}.

• What is the running time? (exercise)

Randomized Algorithms

Summer 2018 Amo G. Tong 51