April 18, 20231

Grenchmark:A workload generator for Grid schedulers

First Demo at TU Delft

A. Iosup, D.H.J. EpemaPDS Group, ST/EWI, TU Delft

April 18, 20232

Evaluating Grid schedulers performance• Grid schedulers performance

• Qualitative metricssupported application types, advanced fault tolerance, advanced ...

• Quantitative metricsresource consumption, system performance, scheduler functionality

• Other metricscost, single number

• Needs• Applications, workloads, more workloads…

• Benefits • Configuration testing and tuning• Functionality testing• Comparing design alternatives• Runtime estimation• Systems procurement support

April 18, 20233

Grenchmark

• A systematic approach to testing Grid schedulers• A set of metrics for comparing schedulers• A set of representative applications

• Both real and synthetic• Single tasks, MPI tasks, Java, IBIS/Satin tasks, batch

tasks, filters/streaming tasks, chains of tasks, DAGs, general graphs, ...(?)

• Easy tools to create synthetic workloads [TODAY]

• Testing, testing, testing… [EVERY DAY]

April 18, 20234

Grenchmark process

$ ./wl-gen.py wl-desc.in $ ./wl-submit.py out/wl-to-submit.wl

12

1

2

TODO

• Sample run:• 95 jobs / 411 components• 95 files / 32 directories• 300KB data

• Sample run:• multi-threaded – submission delay +/- 0.01s• 95 JDFs

April 18, 20235

Outline

• Introduction• Grenchmark• Using Grenchmark• Customizing Grenchmark• Extending Grenchmark• Conclusions

[done][done][here]

April 18, 20236

wl-desc.in

$ cat wl-desc.in# File-type: text/wl-spec# WL Unit ID<tab>Times this unit<tab>Job Type<Tab>Site structure

type<Tab>Total # of jobs per unit<tab>Detailed Site structure<Tab>Arrival time distribution<Tab>Other info

#ID Times UnitType SiteType Total SiteInfo ArrivalTimeDistr OtherInfo 0 5 sser single 1 *:? U(0,1000) - 1 10 sser unordered2 - N(500,200)

NSites=2,DifferentSites=false ? 1 sser ordered ? fs3:2,fs2:1,fs1:1 Poisson(500) - ? 1 sser semi-ordered 5 fs3:2,*:1 LogNorm(10,5)

NSites=3,DifferentSites=true ? 1 fake semi-ordered 5 fs3:2,*:1 Weibull(2000,0.5)

NSites=3,DifferentSites=true ? 5 wl:poisson - - - Poisson(500)

RandomWorkload=true,NJobs=10,NComponents=2;8,ComponentSize=4;8 ? 2 wl:poisson - - - Poisson(500)

RandomWorkload=false,NComponents=4-8,ComponentSize=8;16 ? 1 wl:poisson - - - Poisson(500)

RandomWorkload=false,NComponents=4;8,ComponentSize=8;16,Submitter=drunner,Application=Poisson-g2-gm

? 2 wl:poisson - - - Poisson(500)RandomWorkload=false,NComponents=4;8,ComponentSize=8;16,Submitter=krunner,Application=findpi-gm

• Describes the workload to be generated; input file for wl-gen.py

April 18, 20237

Preliminary notions

• Job, workload, workload unit• Job = Set of components (co-allocation support)

[Job = one program execution / the basic scheduling unit / … ]

• Workload = Set of Jobs• Workload Unit = Set of jobs with the same

property, generated in the same step (definition useful only for workload generator)

• Other• JDF = Job description file• Inter arrival time = time between two consecutive

jobs arrive in the system

April 18, 20238

Some more notions

• Job site type / per site description• Single – run at one site• Unordered – run at several sites, all unspecified• Ordered – run at several site, all specified• Semi-ordered – run at several site, some

specified• Other (?)

• Inter-arrival time distributions Constant, Uniform, Normal, Exponential(λ), Poisson(Mean), HExp2, HPoisson2, Weibull, LogNormal, Gamma (~)

April 18, 20239

Describing the workload: Jobs wl-desc.in

ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) -

• Generate Times x Jobs or Times x Sets of Jobs• One Job = k Components (here: 1 -- single)

April 18, 202310

Demo 1: Generating one job

• Single site, single instance

April 18, 202311

Demo 2: Generating n jobs of the same type• 250 jobs in 100s, distribution Uniform(0,1000)

April 18, 202312

Demo 3: Generating mixes of jobs

• 10 single jobs, 2 instances per job (count=2)Q: what happens with the running/output sandbox?

• 10 unordered jobs, 5 instances at 3 sites (3 components) eachQ: what about ensuring DifferentSites=true? I can do it from the submitter, but then ordered… [Jan David’s P2P simulation must include controlled wide-area/single-area mixes]

• 10 ordered jobs, inter-arrival time (IAT) IAT 5s

• 10 semi-ordered jobs, 5 instances, 2 ordered sites (1 random location), IAT LogNorm(10,3)

April 18, 202313

Describing the workload: Jobs wl-desc.in

ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) -

• Generate Times x Jobs or Times x Sets of Jobs• One Job = k Components (here: 1 -- single)

April 18, 202314

Describing the workload: Units wl-desc.in

ID Times UnitType SiteType TotalComps PerSiteDesc [I]ArrivalTimeDistr OtherInfo 0 1 sser single 1 *:? U(0,1000) - ? 1 wl:poisson unordered - - Poisson(500) \

RandomWorkload=false,NComponents=4;8,ComponentSize=8;16,\Submitter=drunner,Application=Poisson-g2-gm

• Generate Times x Jobs or Times x Sets of Jobs• wl: prefix – self-aware workload unit generator [Hashim’s jobs]

April 18, 202315

Demo 4: Generating one workload unit

• Hashim’s sample workload:• Poisson-g2-gm (Globus MPICH-G2)• Components: 4 and 8• Component Sizes: 8 and 16• IAT Poisson(5s)• Run with drunner

• Total: 4 jobs, 24 components

April 18, 202316

Demo 5:Generating a full workload• Hashim’s full workload, 8 times:

• Poisson-g2-gm (Globus MPICH-G2) and findpi-gm• Components: 4 and 8• Component Sizes: 4/8 and 16• IAT Poisson(5s)• Run with drunner (poisson) / krunner (pi)• Submit time 1 day

• Generate:

• Submit:

• Total: 3200 jobs, 19200 components / 3k files, 4k dirs

• Timing: 30s generate / 86,400s submit (1 day, eh?)

$ wl-gen.py --duration=86400000 wl-desc.in

$ wl-submit.py --onefile out/wl-desc.in

April 18, 202317

Outline

• Introduction• Grenchmark• Using Grenchmark• Customizing Grenchmark• Extending Grenchmark• Conclusions

[done][done]

[here]

[done]

April 18, 202318

Customizing the workload generator Command-line parameters• wl-gen.py –help … displays a comprehensive help screen

• wl-gen.py –o outdir … creates the workload in directory outdir

• wl-gen.py –d 50000 … sets the total submission duration to 50s

• wl-gen.py –j my-jdf … uses my-jdf to write JDFs

• wl-gen.py –o out-1 wl-desc.in my favourite command

[seen]

April 18, 202319

Customizing the submission tool Command-line parameters

• wl-submit.py –help … displays a comprehensive help screen

• wl-submit.py –t 10 … uses 10 threads to submit

• wl-submit.py --nosubmit … just prints the submission commands

• wl-submit.py --onefile … output job submit (runner) stderr and stdout to one file each (concatenated)

• wl-submit.py out/wl-to-submit.wl my favourite command

[seen]

April 18, 202320

Outline

• Introduction• Grenchmark• Using Grenchmark• Customizing Grenchmark• Extending Grenchmark• Conclusions

[done][done]

[here]

[done]

[done]

April 18, 202321

Extending the workload generator (1) Write your own Job Generators

1. Create your-job-generator.py in apps/

2. Place a Generator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example:Generator=""" Name=your-job-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“

3. Create a function generateWorkloadUnit, which takes in

UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom

4. Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each

5. Use the generator in wl-desc.in by putting your-job-type in the UnitType field

April 18, 202322

Extending the workload generator (2) Write your own Unit Generators

1. Create your-job-generator.py in apps/

2. Place a WLGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example:WLGenerator=""" Name=your-workload-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“

3. Create a function generateWorkload, which takes in UnitDir, UnitID, WLUnit, SubmitDurationMS, bGenerateRandom

4. Make generateWorkloadUnit return a dictionary with keys ‘info’ and ‘jobs’, with appropriate data each

5. Use the generator in wl-desc.in by putting your-job-type in the UnitType field

April 18, 202323

Extending the workload generator (3) Write your own JDF Writers

1. Create your-jdf-writer.py in jdfprint/

2. Place a JDFGenerator attribute, organized as a Key=Value list, one per line, with at least Name=your-job-type defined. Example:JDFGenerator=""" Name=your-jdf-writer-type Author=You R. Name Copyright=copyright © 2005 You R. Name. All rights reserved. ""“

3. Create a function generateJDF, which takes in OutFileName, ListOfComponents

4. Make generateJDF write the components to the specified file, with appropriate data each

5. Use the generator by calling wl-gen.py with the –j parameter

April 18, 202324

Conclusions and future work

• Grenchmark easily generates diverse workloads of Grid applications

• KOALA developersWould you like to use Grenchmark to test KOALA?

• EverybodyDo you have specific Grid applications types that you would like to test? Try Grenchmark!

April 18, 202325

Thank you!

Questions? Remarks? Observations? All welcome!

Grenchmark http://grenchmark.st.ewi.tudelft.nl/ [10x Paulo]

Alexandru IOSUPTU Delft

A.Iosup@ewi.tudelft.nl

http://www.pds.ewi.tudelft.nl/~iosup/index.html [google: “iosup”]