Scheduling in Cloud Presented by: Abdullah Al Mahmud Course: Cloud Computing(Fall 2012)
-
Upload
jared-pitts -
Category
Documents
-
view
219 -
download
1
Transcript of Scheduling in Cloud Presented by: Abdullah Al Mahmud Course: Cloud Computing(Fall 2012)
Quincy: Fair Scheduling for Distributed Cluster
Scheduling in CloudPresented by: Abdullah Al MahmudCourse: Cloud Computing(Fall 2012)
PapersQuincy: Fair Scheduling for Distributed Computing Clusters Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, Andrew Goldberg @ MSR Silicon Valley
Optimized Resource Allocation & Task Scheduling Challenges in Cloud Computing EnvironmentsDominique A. Heger, DHTechnologies (DHT)
Quincy: Fair Scheduling for Distributed Computing ClustersMichael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, and Andrew GoldbergModified version of www.sigops.org/sosp/sosp09/slides/quincy/QuincyTestPage.htmlProblem SettingHomogenous ClusterFine grain resource sharing (multiplex all computers in the cluster between all jobs)Independent tasks(less costly to kill a task and restart the task)Goal of QuincyFair Sharing and Data LocalityN computers, J concurrent jobs-Each job gets at least N/J computers-Place tasks near data to avoid network bottlenecks-Joint optimization of fairness and data localityCluster Architecture
6CS-> Core SwitchRS-> Rack SwitchCi->CoresBaseline: Queue Based Scheduler
7Baseline: Queue Based SchedulerGreedy: Running the first available job in the queueSimple Greedy Fairness: Starving a job that submits large number of workersFairness with preemption: Killing workers from a job that already have submitted large number of workers.Flow Based Scheduler: QuincyConstruct a graph based on scheduling constraint and cluster architectureFinding a matching in the graph is equivalent to finding a feasible schedule.Can assign a cost to any matchingFairness constraints: number of tasks that are scheduledGoal: Minimize matching cost while obeying fairness constraints
Graph ConstructionStart with a directed graph representation of the cluster architecture
10Graph Construction (2)
11Graph Construction (3)
12A Feasible Matching
13Final Graph
14Result: Makespan when network is bottleneck(s)
Result: Data Transfer (TB)
ConclusionNew computational model for data intensive computingElegant mapping of scheduling to min-cost flow/matching problem
17Optimized Resource Allocation & Task Scheduling Challenges in Cloud Computing EnvironmentsDominique A. HegerResource Allocation in the CloudEach task's resource demand can be described via a multi-dimensional vector such as that the task i requires x processing cores, y GB of memory, and z GB of storage.Classical Bin Packing instance(Three Dimensional) which is a well known NP Complete problem
ANN Based Task Scheduling
ConclusionThis paper discusses some theoretical aspects of Task Scheduling and Resource AllocationQuestion?Thank You