Randomized Algorithms - ict.iitk.ac.inย ยท Randomized Quick Sort from perspective of ๐ and ๐...
Transcript of Randomized Algorithms - ict.iitk.ac.inย ยท Randomized Quick Sort from perspective of ๐ and ๐...
Randomized Algorithms CS648
Lecture 2 โข Three interesting problems
โข Balls into bins โข Balls out of bin โข Randomized Quick Sort
โข Random Variable and Expected value
1
Recap from last lecture
โข Probability space (๐,P)
โข Events
โข Probability of union of events
โข Partition theorem :
If events ๐1,โฆ, ๐๐partition the sample space ๐ then
โข Conditional probability
2
P[A|B] = ๐[๐โฉ๐]
๐[๐]
P(AUB) = P(A) + P(B) โ P(AโฉB)
P(B) = P(๐๐โฉB๐ )
BALLS INTO BINS
3
Balls into Bins
Ball-bin Experiment: There are ๐ balls and ๐ bins.
Each ball selects its bin randomly uniformly and independent of other balls
and falls into it.
Applications: โข Hashing
โข Load balancing in distributed environment 4
1 2 3 โฆ ๐ โฆ ๐
1 2 3 4 5 โฆ ๐ โ 1 ๐
Balls into Bins
Question : What is the probability that there is at least one empty bin ?
5
1 2 3 โฆ ๐ โฆ ๐
1 2 3 4 5 โฆ ๐ โ 1 ๐
Balls into Bins
What is the probability space (ฮฉ,P) ?
โข | ฮฉ | = ๐๐
โข P(ฯ) = 1/ ๐๐ for each ฯฯต ฮฉ
6
1 2 3 โฆ ๐ โฆ ๐
1 2 3 4 5 โฆ ๐ โ 1 ๐
Balls into Bins
Let us define event ๐ฌ๐๐
โข Events ๐ฌ๐๐ and ๐ฌ๐
๐ are ??
โข Events ๐ฌ๐๐ and ๐ฌ๐
๐ are ??
โข Events ๐ฌ๐๐ and ๐ฌ๐
๐ are ??
7
1 2 3 โฆ ๐ โฆ ๐
1 2 3 4 5 โฆ ๐ โฆ ๐ โ 1 ๐
disjoint
Independent
Independent
: ๐th ball falls into ๐th bin.
Balls into Bins
๐ฌ๐๐ : ๐th ball enters ๐th bin.
โข Pr[๐ฌ๐๐] = ??
โข Pr[๐ฌ๐๐] = ??
โข Pr[๐th bin is empty] = ??
=(1 โ 1
๐ )๐
8
1๐
1 โ1
๐
Pr[๐ฌ1๐ โฉ๐ฌ2
๐ โฆโฉ๐ฌ๐๐] = Pr[๐ฌ1
๐ ] โจฏ โฆPr[๐ฌ๐๐]
1 2 3 4 5 โฆ ๐ โฆ ๐ โ 1 ๐
1 2 3 โฆ ๐ โฆ ๐
Balls into Bins
โข Pr[๐th bin is empty] = (1 โ 1
๐ )๐
โข Pr[๐th and ๐th bin are empty] = ??
โข Pr[a specified set of ๐ bins are empty] = ??
9
(1 โ 2๐
)๐
(1 โ ๐๐ )๐
1 2 3 โฆ ๐ โฆ ๐
1 2 3 4 5 โฆ ๐ โฆ ๐ โ 1 ๐
Balls into Bins
Question: What is the probability that there is at least one empty bin ?
Attempt 1: Explore the sample space associated with the โballs into binsโ.
Attempt 2: ??
Define ๐๐ : โ๐th bin is emptyโ
Event โthere is at least one empty binโ = ๐๐๐๐=1
10
Express the event as union of some events โฆ
Balls into Bins
Theorem: For events ๐1,โฆ, ๐๐defined over a probability space (๐,P), then
P( ๐๐๐๐=1 ) = P(๐๐)๐
โ P(๐๐ ๐๐ )๐<๐
+ P(๐๐ ๐๐ ๐๐ )๐<๐<๐
โ โฆ
(โ1)๐+1 P(๐1 ๐2 โฆ ๐๐)
--------------------------------------------------------------------
= (๐1)(1 โ 1
๐ )๐
โ (๐2)(1 โ 2
๐ )๐
+ (๐3)(1 โ 3
๐ )๐
โ โฆ
+ โ1 ๐+1(๐๐) (1 โ ๐
๐ )๐
โฆ 11
๐(๐,๐)
Balls into Bins
Homework Exercise:
What is the probability that there are exactly ๐ empty bins ?
Hint: You will need to use ๐(๐,๐) with suitable values of ๐.
12
BALLS OUT OF BIN
13
Balls Out of Bin
There is a bin containing ๐ red balls and ๐ blue balls.
Balls are taken out from the bag
Question: What is the probability that appears ahead of all blue balls. ?
14
uniformly randomly and without replacement.
What if ๐ = 1 ?
Balls Out of Bin
There is a bin containing 1 red ball and ๐ blue balls.
Balls are taken out from the bag
Question: What is the probability that appears ahead of all blue balls ?
Question: What is the probability that appears at ๐th place ? 15
uniformly randomly and without replacement.
1
๐ + 1
1
๐ + 1
Balls Out of Bin
There is a bin containing ๐ red balls and ๐ blue balls.
Balls are taken out from the bag
Question: What is the probability that appears ahead of all blue balls ?
Homework: Give formal arguments in support of your answer. 16
uniformly randomly and without replacement.
1
๐ + 1
RANDOMIZED QUICK SORT
17
18
How to solve a problem ?
Perspective
19
does matter
Randomized Quick Sort
Input: ๐จ[0..๐ โ 1]
RandomizedQuickSort(๐จ,๐, ๐) //For the first call, ๐ =0, ๐=๐ โ 1
{ If (๐ < ๐)
๐ an element selected randomly uniformly from ๐จ[๐..๐];
๐ Partition(๐จ,๐,๐,x);
RandomizedQuickSort(๐จ,๐, ๐ โ ๐);
RandomizedQuickSort(๐จ,๐ + ๐, ๐)
}
Assumption : All elements are distinct (if not, break the ties arbitrarily)
Notation : ๐๐ : ๐th smallest element of array ๐จ.
Question: What is the probability that ๐๐ is compared with ๐๐ ?
20
Recall that the execution of RandomizedQuickSort is totally immune to the permutation of ๐จ.
Randomized Quick Sort
Input: ๐จ[0..๐ โ 1]
RandomizedQuickSort(๐จ,๐, ๐) //For the first call, ๐ =0, ๐=๐ โ 1
{ If (๐ < ๐)
๐ an element selected randomly uniformly from ๐จ[๐..๐];
๐ Partition(๐จ,๐,๐,x);
RandomizedQuickSort(๐จ,๐, ๐ โ ๐);
RandomizedQuickSort(๐จ,๐ + ๐, ๐)
}
The sample space : all recursion trees (rooted binary trees on ๐ nodes).
21
Randomized Quick Sort
Input: ๐จ[๐..๐ โ ๐]
RandomizedQuickSort(๐จ,๐, ๐) //For the first call, ๐ =0, ๐=๐ โ 1
{ If (๐ < ๐)
๐ an element selected randomly uniformly from ๐จ[๐..๐];
๐ Partition(๐จ,๐,๐,x);
RandomizedQuickSort(๐จ,๐, ๐ โ ๐);
RandomizedQuickSort(๐จ,๐ + ๐, ๐)
}
22
๐4
๐2
๐1 ๐3
๐6
๐5 ๐7
๐ = 7 1
7โ 1
3โ 1
3
Randomized Quick Sort
Input: ๐จ[๐..๐ โ ๐]
RandomizedQuickSort(๐จ,๐, ๐) //For the first call, ๐ =0, ๐=๐ โ 1
{ If (๐ < ๐)
๐ an element selected randomly uniformly from ๐จ[๐..๐];
๐ Partition(๐จ,๐,๐,x);
RandomizedQuickSort(๐จ,๐, ๐ โ ๐);
RandomizedQuickSort(๐จ,๐ + ๐, ๐)
}
23
๐ = 7 ๐5
๐6
๐7
๐2
๐1 ๐4
๐3
1
7โ 1
4โ 1
2โ 1
2
Randomized Quick Sort
Question: What is the probability that ๐๐ is compared with ๐๐ ?
Attempt 1: Explore the sample space associated with Randomized Quick Sort.
add the probability of each recursion tree in which ๐๐ is compared with ๐๐.
Attempt 2: ??
24
View the execution of RandomizedQuickSort from perspective of ๐๐ and ๐๐
Not a feasible way to calculate the probability
Randomized Quick Sort from perspective of ๐๐ and ๐๐
In order to analyze the Randomized Quick Sort algorithm from the perspective of ๐๐ and ๐๐, we do the following:
โข We visualize elements ๐1, โฆ , ๐๐ arranged from left to right in increasing order of values.
โข This visualization ensures that the two subarrays which we sort recursively lie to left and right of the pivot element. In this way we can focus on the subarray containing ๐๐ and ๐๐easily.
โข Note that this visualization is just for the sake of analysis. It will be grossly wrong if you interpret it as if we are sorting an already sorted array.
25
Go through the next few slides slowly and patiently, pondering at each step. Never accept anything until and unless you can see the underlying truth yourself.
Randomized Quick Sort from perspective of ๐๐ and ๐๐
26
Elements of A arranged in Increasing order of values
๐๐ ๐๐
Randomized Quick Sort from perspective of ๐๐ and ๐๐
Observation:
๐๐ and ๐๐ get compared during an instance of Randomized Quick Sort iff
the first pivot element from ๐บ๐๐ is either ๐๐ or ๐๐.
Let us define two events.
๐ต๐ : first pivot element selected from ๐บ๐๐ during Randomized Quick Sort is ๐๐.
๐ต๐ : first pivot element selected from ๐บ๐๐ during Randomized Quick Sort is ๐๐.
Pr[๐๐ and ๐๐ get compared] = ??
27
๐๐ ๐๐
๐บ๐๐
Pr[๐ต๐U๐ต๐]
Randomized Quick Sort from perspective of ๐๐ and ๐๐
Pr[๐๐ and ๐๐ get compared] = Pr[๐ต๐U๐ต๐]
= Pr[๐ต๐] + Pr[๐ต๐] - Pr[๐ต๐ โฉ๐ต๐]
= Pr[๐ต๐] + Pr[๐ต๐]
= 1
๐โ๐+1 + 1
๐โ๐+1
= 2
๐โ๐+1
28
๐๐ ๐๐
๐บ๐๐
What relation exists between ๐ต๐ and ๐ต๐ ?
๐ต๐ and ๐ต๐ are disjoint events.
What is Pr[๐ต๐] ? Pr[๐ต๐] =
1
|๐บ๐๐|.
Randomized Quick Sort from perspective of ๐๐ and ๐๐
Theorem: During Randomized-Quick-Sort on ๐ elements,
the probability ๐๐ and ๐๐ are compared with probability 2
๐โ๐+1 .
Inferences: โข Probability depends upon the rank separation ๐ โ ๐ + 1
โข Probability ----------?----------- the size of the array.
โข Probability ๐๐ and ๐๐+1 are compared = ? .
โข Probability of comparison of ๐0 and ๐๐โ1 = ? .
29
is independent of
1 2
๐
PROBABILITY THEORY
โข Random variable
and
โข expected value
30
Random variable
31
Randomized-Quick-Sort
on array of size n
Number of HEADS in 5 tosses
Sum of numbers in 4 throws
Number of comparisons
Random variable
Definition: A random variable defined over a probability space (ฮฉ,P)
is a mapping ฮฉ R.
โข ๐ฟ(ฯ) : the value of ๐ฟ on elementary event ฯ โ ๐ .
Notations for random variables :
โข ๐ฟ, ๐, ๐ผ, โฆ(capital letters)
32
ฮฉ
R
Set of real numbers
๐ฟ
Many Random Variables for the same Probability space
Random Experiment: Throwing a dice two times
โข X : the largest number seen
โข Y : sum of the two numbers seen
33
X(ฯ) = 6 ฯ
Y(ฯ) = 9
Expected Value of a random variable (average value)
Definition: Expected value of a random variable X defined over a probability space (ฮฉ,P) is
E[X] = X(ฯ) โจฏ P(ฯ)ฯฯต ฮฉ
34
ฮฉ
X= a X= b
X= c
E[X] = aโจฏ P(X = a)aฯต X
Examples
Random experiment 1: A fair coin is tossed n times
Random Variable X: The number of HEADS
E[X] = ๐ โจฏ P(X =๐)๐
= ๐ โจฏ๐๐๐ (1 2 )
๐ (1 2 )๐โ๐
= ๐ 2
Random Experiment 2: 4 balls into 3 bins
Random Variable X: The number of empty bins
E[X] = ๐ โจฏ P(X =๐)๐
35
Can we solve these problems ? Random Experiment 1 : There are ๐ balls and ๐ bins.
Each ball falls into a bin selected randomly uniformly and independently.
Random Variable X: The number of empty bins
E[X]= ??
Random Experiment 2 : A bin contains ๐ red balls and ๐ blue balls in a bin.
The balls are taken out randomly uniformly without replacement.
Random Variable X: The number of red balls preceding all blue balls.
E[X]= ??
Random Experiment 3 Randomized Quick sort on ๐ elements Random Variable X: The number of comparisons
E[X]= ??
36
โข Spend at least 2 hours today trying to solve the 3 problems given in the previous slide.
โข These 2 hours will be very valuable for you in the long run.
I really mean it.
37
Fun with probability
38
Stick problem
We break the stick at ๐ points selected randomly uniformly.
39
What is the probability that the ๐ + ๐ pieces can be joined to form a polygon ?