Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R....
-
date post
22-Dec-2015 -
Category
Documents
-
view
227 -
download
0
Transcript of Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R....
![Page 1: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/1.jpg)
Realistic CPU Workloads Through Host Load Trace Playback
http://www.cs.cmu.edu/~pdinda/LoadTraces
Peter A. Dinda
David R. O’Hallaron
Carnegie Mellon University
![Page 2: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/2.jpg)
2
Talk in a Nutshell
• Workloads to evaluate distributed systems• Prediction-based systems• Shared benchmarks
• Reconstruct time-varying CPU contention behavior from traces of Unix load average
• Real (non-parametric)• Reproducible• Comparable
• Artifacts (http://www.cs.cmu.edu/~pdinda/LoadTraces)
• Playload tool• Collection of host load traces
![Page 3: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/3.jpg)
3
Outline
• Evaluation of distributed systems• Adaptive applications• Prediction systems
• Host load traces• Synthetic Vs trace-based workloads
• Host load trace playback
• Evaluation
• Conclusion
![Page 4: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/4.jpg)
4
Evaluating Distributed Systems
• Workloads critical in evaluation
• Shared benchmarks desperately needed• SPEC?
• Synthetic versus trace-based workloads• Parametric versus non-parametric
• Ideally:• Real workloads• Reproducible• Comparable• Sharable
![Page 5: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/5.jpg)
5
Evaluating Prediction Systems• Prediction systems model workload
• RPS, NWS, adaptive applications, etc.
• Synthetic workloads assume a model• Model may be wrong or incomplete• Easy to use in simulation or in a testbed
• Trace-based workloads assume no model• However, traces may not be representative• Harder to use, especially in a testbed
• How do we use traces in a testbed?• Host load traces
![Page 6: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/6.jpg)
6
Host Load Traces
• Periodically sampled Unix load average• Exponentially averaged run queue length
• Measure of contention for CPU
• Complex statistical properties [SciProg99]• High variability, strong autocorrelation function, self-
similar, epochal behavior, …• Difficult to synthesize
![Page 7: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/7.jpg)
7
Host Load and Running Time
1 3 5 7Measured Load
0
5
10
15
20
25
Exe
cutio
n T
Ime
(Sec
onds
)
42,000 pointsCoefficient of Correlation = 0.998
nomtt
texec
exec t
dttzt
texecnow
now
)(1
1
![Page 8: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/8.jpg)
8
Available Host Load Traces• DEC Unix 5 second exponential average
• Full bandwidth captured (1 Hz sample rate)• Long durations• Available! http://www.cs.cmu.edu/~pdinda/LoadTraces
Machines Duration
August 1997 13 production cluster8 research cluster2 compute servers
15 desktops
~ one week(over onemillionsamples)
March 1998 13 production cluster8 research cluster2 compute servers
11 desktops
~ one week(over onemillionsamples)
![Page 9: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/9.jpg)
9
Host Load Measurement
Sample1 h
{
Ready Queue
Sample2
unknownsample ratef=2 Hz estimated
exponentialaverage, tau=5 s
f=1 Hz
Kernel User
TraceFile
![Page 10: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/10.jpg)
10
Host Load Trace Playback
h-1Load
GeneratorTraceFile
LoadMeasure
Sample1h
{
Sample2error
-
applied load
measuredload
target load
![Page 11: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/11.jpg)
11
What are h and h-1?
iii xezez recordrecord )1()( //1
record
record
e
zezx iii
/
/1
1
)(
iii xezez playbackplayback )1()( //1
run queue lengthtrace filetime constant
for recorded host
applied load (recovered run queue length)
h:
h-1:
measured load
time constantfor playback host
![Page 12: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/12.jpg)
12
Load Generator
“1.5 load for 1 second”
“1.0 load for 1 second”
...
“0.5 load for 1 second”
Master
...
Worker Processes
“0.0 load for 1 second”
![Page 13: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/13.jpg)
13
Load Generator// Split w into n cycles
while (!done) {if (uniformrand(1.0) < p)
compute for w/n seconds;else sleep for w/n seconds;
}
“p load for w seconds”
done=“w seconds have elapsed” Time-based playback
done=“w*p CPU seconds have been used” Work-based playback
(simplified)
![Page 14: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/14.jpg)
14
Time-based Playback
0
0.5
1
1.5
2
2.5
0 50 100 150 200 250Time (seconds)
Target Load
Measured Load
External continuous 1.0 load
External load amplitude modulates applied load
![Page 15: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/15.jpg)
15
Work-based Playback
External load amplitude and frequency modulates applied load
0
0.5
1
1.5
2
2.5
0 50 100 150 200 250Time (seconds)
Target Load
Measured Load
External continuous 1.0 load
![Page 16: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/16.jpg)
16
Evaluation
• Traces described earlier• Example uses one hour 1997 axp0 trace
• Characterization of signals and errors• Summary stats• Distributions• Autocorrelation
• Multiple platforms• Digital Unix• Solaris• Linux• FreeBSD
![Page 17: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/17.jpg)
17
Evaluation Summary StatsTarget MeasuredEnvironment
Mean Std Mean StdAlpha/DUX 4.0 1.065 0.465 1.047 0.442Sparc/Solaris 2.5 1.047 0.376 1.112 0.356PII/FreeBSD 2.2 1.047 0.376 1.123 0.361PII/RH Linux 5.2 1.047 0.376 1.131 0.360
ErrorEnvironmentMean Std
Alpha/DUX 4.0 -0.018 0.127Sparc/Solaris 2.5 0.076 0.061PII/FreeBSD 2.2 0.076 0.124PII/RH Linux 5.2 0.084 0.164
![Page 18: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/18.jpg)
18
Evaluation on Alpha/DUXTitle:manch-7.good.compare.3600.out.targettrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:manch-7.good.compare.3600.out.errhist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:manch-7.good.compare.3600.out.measuredtrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:manch-7.good.compare.3600.out.erracf.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Target
Measured Error ACF
Error Histogram
![Page 19: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/19.jpg)
19
Evaluation on Sparc/SolarisTarget
Measured Error ACF
Error HistogramTitle:federation.good.compare.3600.out.targettrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:federation.good.compare.3600.out.errhist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:federation.good.compare.3600.out.measuredtrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:federation.good.compare.3600.out.erracf.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 20: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/20.jpg)
20
Evaluation on P2/FreeBSDTarget
Measured Error ACF
Error HistogramTitle:greenfield.good.compare.3600.out.targettrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:greenfield.good.compare.3600.out.errhist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:greenfield.good.compare.3600.out.measuredtrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:greenfield.good.compare.3600.out.erracf.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 21: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/21.jpg)
21
Evaluation on P2/LinuxTarget
Measured Error ACF
Error HistogramTitle:infocom.good.compare.3600.out.targettrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:infocom.good.compare.3600.out.errhist.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:infocom.good.compare.3600.out.measuredtrace.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Title:infocom.good.compare.3600.out.erracf.epsCreator:MATLAB, The Mathworks, Inc.Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
![Page 22: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/22.jpg)
22
Conclusion• CPU Workloads from traces of
Unix load average• Reproduce contention behavior (ignoring priorities)• Real, non-parametric workloads• Reproducible
• Artifacts (http://www.cs.cmu.edu/~pdinda/LoadTraces)
• Playload tool• Collection of host load traces
• Future• Benchmarks• Priorities, memory, disk, etc.• In-kernel?
![Page 23: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/23.jpg)
23
Feedback?
Use error signal to better track the load trace
h-1Load
GeneratorLoad
Measure
error- h-1
+
z
TraceFile
x level
![Page 24: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/24.jpg)
24
The Problem With Feedback
Feedback would try to make SUM of applied load and external load in system track the load trace
ExternalLoad
h-1Load
GeneratorLoad
Measure
error- h-1
+
z
TraceFile
x level
AppliedLoad
Effect ofCombined
Load
![Page 25: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/25.jpg)
25
Making Feedback Work
ExternalLoad
h-1Load
GeneratorLoad
Measure
error- h-1
+
z
TraceFile
x level
SignalSeparation
AppliedLoad
Effect ofCombined
Load
EstimatedEffect of
Applied Load
EstimatedEffect of
External Load
Load SourceModels
![Page 26: Realistic CPU Workloads Through Host Load Trace Playback pdinda/LoadTraces Peter A. Dinda David R. O’Hallaron Carnegie Mellon University.](https://reader035.fdocuments.net/reader035/viewer/2022081515/56649d805503460f94a64819/html5/thumbnails/26.jpg)
26
Why Host Load Traces for Evaluating Distributed Systems?
• Real
• Comparable and reproducible• Analogous to a SPEC benchmark• Usable in simulation and experimentation
• Non-parametric and non-synthetic• Especially important for prediction systems