MapReduce-Simulator: Matchmaking and Scheduling Algorithm Norman Lim Dept. of Systems and Computer...
-
Upload
roxanne-underwood -
Category
Documents
-
view
216 -
download
0
Transcript of MapReduce-Simulator: Matchmaking and Scheduling Algorithm Norman Lim Dept. of Systems and Computer...
MapReduce-Simulator: Matchmaking and
Scheduling Algorithm
Norman LimDept. of Systems and Computer Engineering
Carleton UniversityOttawa, ON, [email protected]
SYSC 5807: Resource Management on Distributed Systems
Overview of Matchmaking and Scheduling (Mapping) Algorithm
1. When a job j arrives, it is placed in a queue depending on the scheduling policy selected.o FIFO: uses first-in first-out queueo EDF: uses earliest deadline first queue
2. Job/Task Mapping algorithm is invoked. o Retrieves the first job in the queue and maps each of the job’s
tasks onto the resources.o Maps each task t onto a resource r that can execute t at its
earliest possible time (at or after its earliest start time).o For a particular job, the tasks in that job with higher execution time
are mapped first.
2
Overview of Matchmaking and Scheduling (Mapping) Algorithm Cont.
EDF Policy Only 3. If a job j is not able to be scheduled to complete
executing before its deadline, then the Job/Task Re-mapping algorithm is invoked.o Remaps j and the set of jobs (called S) that have caused j to miss
its deadline. • S includes jobs that are scheduled to start at, or complete within
the interval: [start time of job j, deadline of job j].o j and the jobs in S are re-ordered according to earliest deadline
first.o The Job/Task Mapping algorithm (see previous slide) is invoked to
map j and the jobs in S.
3
Example: Jobs and Resources
4
Job 1 (j1)Arrival time = 0sEST = 2sDeadline = 30s
t1
2t2
5t3
8
t4
10t5
11
Job 2 (j2)Arrival time = 1sEST = 1sDeadline = 20s
t1
4t2
5
t3
10
Resource 1 (r1)Map Capacity = 1Reduce Capacity = 1
Resource 2 (r2)Map Capacity = 1Reduce Capacity = 1
EST = earliest start time
Example: FIFO
5
FIFO Queue
r1
r2
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
j1, t3
j1, t2 j1, t1
j1, t5
j1, t4
j2, t2
j2, t1
j2, t3
j2 will miss its deadline.
reduce
map
reduce
map
Job 1 (j1)Arrival time = 0sEST = 2sDeadline = 30s
Job 2 (j2)Arrival time = 1sEST = 1sDeadline = 20s
Example: EDF (Case 1)
6
EDF Queue
Job 1 (j1)Arrival time = 0sEST = 2sDeadline = 30s
r1
r2
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
j1, t3
j1, t2 j1, t1
j1, t5
j1, t4
Job 2 (j2)Arrival time = 1sEST = 1sDeadline = 20s
j2, t2
j2, t1
j2, t3
reduce
map
reduce
map
Example: EDF (Case 2)
7
EDF Queue
r1
r2
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
j1, t3
j1, t2 j1, t1
j1, t5
j1, t4
j2, t2
j2, t1
j2, t3reduce
map
reduce
map
Job 1 (j1)Arrival time = 0sEST = 2sDeadline = 30s
Job 2 (j2)Arrival time = 1sEST = 1sDeadline = 20s
j2 will miss its deadline.
Job 2 (j2)Arrival time = 1sEST = 1sDeadline = 20s
Job 1 (j1)Arrival time = 0sEST = 2sDeadline = 30s
j1, t3
j1, t2 j1, t1
j1, t5
j1, t4
j2, t2
j2, t1
j2, t3