Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and...
Transcript of Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and...
![Page 1: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/1.jpg)
© 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Data Mining for Sustainable Data Centers
Manish Marwah
Senior Research Scientist
Hewlett Packard Laboratories
![Page 2: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/2.jpg)
2
Overview
• Sustainability and Data Centers
• Data mining applications
−Chiller operation characterization
−PV prediction
−Anomaly detection
![Page 3: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/3.jpg)
3 3 June 2013
Motivation Industry challenge: Create technologies, IT infrastructure and business models for the low-carbon economy
2%
Aviation Total carbon emissions
2% IT industry
The footprint of IT will need to be reduced quite significantly in a low-carbon economy.
![Page 4: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/4.jpg)
Sustainability
“sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs”
the Brundtland Commission of the United Nations, 1987
4 3 June 2013
![Page 5: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/5.jpg)
5 4 June 2013
Sustainability What do I mean by “sustainability”?
Social (“People”)
Economic (“Profit”)
Environmental (“Planet”)
Risk: Ecological Damage
Sustainable
Risk: Limited Adoption
Risk: Commercially unfeasibility
Figure Credit: A. Agogino, UC Berkeley
![Page 6: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/6.jpg)
6 3 June 2013
Environmental Sustainability
• Impact factors (e.g., carbon, water, toxicity, etc.)
• Life Cycle View
![Page 7: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/7.jpg)
7 3 June 2013
Sustainable Data Centers Lifecycle Assessment
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Handheld
Note
book
Desk
top
Bla
de
Serv
er
Data
Cente
r
Fra
ctio
n o
f Li
fecy
cle E
nerg
y
Operational
Embedded
Results are illustrative only. Actual footprint may differ.
Ref: IEEE Computer 2009
![Page 8: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/8.jpg)
8 © Copyright 2011 Hewlett-Packard Company
Chandrakant D. Patel; [email protected]
Cloud Data Center Supply and Demand Side
Chilled Water loop
Cooling
Tower loop
CHILLER
Warm Water
Air
Mixture
In
QCond
Return Water
QEvap
Makeup
Water
Wp
Wp
UPS
PDU
Qdata center
Switch Gear
Data Center
Power
Computing
Cooling
![Page 9: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/9.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9 U
Onsite Power Grid
Ecosystem of Clients
Biogas PV
Outside Air
Ground Loops
Mechanical Cooling
Local Cooling Grid
6 12 18 240
0.2
0.4
0.6
0.8
1
1.2
Po
we
r(kW
)
Time(hour)
PV Supply
6 12 18 240
0.5
1
po
we
r (K
W)
time (hour)
critical workload
non-critical workload
cooling power
IT Services
Supply
Demand
IT Infrastructure
Supply and Demand in a Data Center
![Page 10: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/10.jpg)
10
Sustainable Operation and Management of Chillers using Temporal Data Mining (KDD ‘09)
• Data Centers
−Cooling Infrastructure
• Problem Statement
• Prior Work
• Our Approach
−Symbolic representation
−Event encoding
−Motif mining
−Sustainability characterization
• Experimental Results
• Summary
![Page 11: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/11.jpg)
11
Data Center Cooling Infrastructure
Computer room air-conditioner (CRAC)
Chiller Unit
Cooling Towers
Water Return (Tin)
Water Supply (Tout)
Consumes from 1/3 up to 1/2 of total power consumption
![Page 12: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/12.jpg)
12
Ensemble of Chillers
• Challenging to operate efficiently
−Complex physical system
• Dynamic
• Heterogeneous
• Inter-dependencies
• Many constraints
−Accurate models not available
−Rapid cycles undesirable – reduce lifespan
• Domain experts determine settings based on heuristics
• Can it be automated through a data-driven approach?
• Which unit to turn ON/OFF?
• At what utilization?
• How to handle increase/decrease in cooling load?
Chiller Ensemble
![Page 13: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/13.jpg)
13
Problem Statement
• Given the following chiller time series
−utilization levels
−power consumption
−cooling loads
• Is it possible to determine which operational settings are more energy efficient?
• And then use this information to advise data center facility operators
![Page 14: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/14.jpg)
14
Some Terminology
• IT cooling load
• Chiller utilization
• Chiller power consumption
• Coefficient of performance (COP)
Cooling Load
Power consumption
![Page 15: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/15.jpg)
15
Prior Work
• Classical approaches to model time series data
−Principal component analysis
−Discrete Fourier transforms
• Discrete representations: SAX [Keogh et al.]
• Motifs: Repeating subsequences [Yankov et al.]
![Page 16: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/16.jpg)
16
Our approach • Goal: Sustainability
characterization of multi- variate time series data
− Chiller utilization data
• Four Main Steps
− Symbolic representation
− Event encoding
− Motif mining
− Sustainability Characterization
Cluster Analysis
Multivariate Time Series Data
Event Encoding
Frequent Motif Mining
Symbolic representation
Transition-event sequence
Frequent motifs
Sustainability characterization
of frequent motifs
Other discrete data sources can be integrated
![Page 17: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/17.jpg)
17
Clustering
• Individual vector: Utilization across all chiller units
• Raw Data: Sequence of such vectors
• Perform k-means clustering
• Use cluster labels to encode multi-variate time series
![Page 18: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/18.jpg)
18
• Event Sequence
Ei = Event type ti = Time of occurrence
• Episode − Ordered collection of events occurring together
• Episode occurrence − Events same ordering as episode in the data.
• Motifs − Frequently occurring episodes
),(),...,,(),,( 2211 NN tEtEtE
)21,(),20,(),17,(),15,(),14,(),12,(),6,(),4,(),3,(),1,( ACDBEACDBA
CBA
<(A,1), (B,3), (D,4), (C,6), (E,12), (A,14), (B,15), (C,17)>
Some Definitions
![Page 19: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/19.jpg)
19
Redescribing time series data
• Perform run-length encoding:
− Note transitions from one symbol to another
• Higher level of abstraction
− Transition events
![Page 20: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/20.jpg)
Level-wise (Apriori-based) motif mining
20 3 June 2013
Candidate generation followed by counting
![Page 21: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/21.jpg)
21
aabbbbbaaaxaaacccccaaaaabbbbbbaaeaaaaaacccccbggaaa
Discrete representation of chiller ensemble time-series Clustering
aabbbbbaaaxaaacccccaaaaabbbbbbaaeaaaaaacccccbggaaa
Occurrence #1 Occurrence #2 ab->ba->ac Motif
Transition
Encoding
Frequent
Episode
Mining
Methodology Summary
Multi
-variate
tim
e-series
Vector representation
![Page 22: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/22.jpg)
22
Sustainability characterization of Motifs
• Average motif COP (coefficient of performance)
− Indicates cooling efficiency of a chiller unit
• COP = IT Cooling Load
Power consumed
• Frequency of oscillations of a motif
− Impacts chiller lifespan
−Normalized number of mean-crossings
![Page 23: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/23.jpg)
23
Experimental Results • Data
−From HP R&D data center in Bangalore • 70,000 sq ft
• 2000 racks of IT equipments
−Ensemble of five chiller units • 3 air cooled chillers
• 2 water cooled chillers
−480 hours of data • July 2 – 7, Nov 27 – 30, Dec 16 – 26, 2008
• 22 motifs found in the data
![Page 24: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/24.jpg)
24
11,11,11,8,8,8,8,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,13,14,12,12,11,11,11
11-8,6637 8-10,6641 10-13,6656 13-14,6657 14-12,6658 12-11,6660
[ 11-8 , 8-10 , 10-13 , 13-14 , 14-12, 12-11 ]
4 15 1 1 2
Symbol seq:
Encoded seq:
Time Series
Transition Motif:
Inter-transition gap constraint = 20 min
A Motif – Detailed Example (1/3)
![Page 25: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/25.jpg)
25
11,11,11,8,8,8,10,10,10,10,10,10,10,10,10,10,10,10,10,13,13,14,12,12,11,11,11
11-8,6698 8-10,6701 10-13,6714 13-14,6716 14-12,6717 12-11,6719
3 13 2 1 2
[ 11-8 , 8-10 , 10-13 , 13-14 , 14-12, 12-11 ]
Symbol seq:
Encoded seq:
Time Series
Transition Motif:
Inter-transition gap constraint = 20 min
A Motif – Detailed Example (2/3)
![Page 26: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/26.jpg)
26
11,11,11,8,8,8,10,10,10,10,10,10,10,10,10,10,10,10,10,10,13,14,12,12,12,11,11,11
11-8,6758 8-10,6761 10-13,6775 13-14,6776 14-12,6777 12-11,6780
3 14 1 1 3
[ 11-8 , 8-10 , 10-13 , 13-14 , 14-12, 12-11 ]
Symbol seq:
Encoded seq:
Time Series
Transition Motif:
Inter-transition gap constraint = 20 min
A Motif – Detailed Example (3/3)
![Page 27: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/27.jpg)
27
Motif 5
Two Interesting Motifs
C1, C2, C3 → Air cooled
C4, C5 → Water cooled
Motif 8 Time (min) →
Chiller
C1
C2
C3
C4
C5
18%
49%
44%
0%
0%
34%
11%
0%
66%
0%
27
Motif 8 Motif 5
COP 4.87 5.40
Units operating
3 air-cooled 2 air-cooled, 1 water cooled
![Page 28: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/28.jpg)
28
Potential Savings
• Annual saving from operating in Motif 5 instead of Motif 8
−Cost savings = $40,000 (~10%)
−Carbon footprint savings = 287,328 kg of CO2
![Page 29: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/29.jpg)
29
Summary • Data center chillers consume substantial power
−Ensemble of chillers – part of data center cooling infrastructure – are challenging to operate energy efficiently
• Mine and characterize motifs
−Symbolic representation
−Event encoding
−Motif mining
−Sustainability characterization
• Demonstrated our approach on data from a real data center – indicates significant potential energy savings
![Page 30: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/30.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30
The Net-Zero Energy Data Center Implementation in Palo Alto
Data center
Outside air
PV micro grid
Cooling infrastructure power demand
Data center supply side Data center demand side
Chill
er
CO
P
Outside Air Temperature (°C) Load (kW)
![Page 31: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/31.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31
Net-Zero Energy Methodology and Integration
Execution
Supply-side Prediction
•Renewable power prediction •Cooling capacity prediction
IT Demand Prediction
IT Workload Planning • Integrate Supply and Demand Side • IT Demand Shaping • Power capping
Measurement Verification
DC Operation Objectives:
• Net-zero energy operation • Maximize use of renewable energy • Minimize dependability on Grid
Dynamic IT Provisioning
Dynamic Cooling Provisioning
Prediction Planning
Verification and Reporting
6 12 18 240
0.2
0.4
0.6
0.8
1
1.2
Po
we
r(kW
)
Time(hour)
PV Supply
6 12 18 240
0.5
1
po
we
r (K
W)
time (hour)
critical workload
non-critical workload
cooling power
0 5 10 15 200.1
0.15
0.2
0.25
0.3
0.35
Time (hour)
Po
we
r (k
W)
Outside Air Cooling Available Capacity
6 12 18 240
0.2
0.4
0.6
0.8
1
1.2
po
we
r (
KW
)
time (hour)
critical workload
non-critical workload
cooling power
renewable supply
Ref: Z. Liu, Y. Chen, C. Bash, A. Wierman, D. Gmach, Z. Wang, M. Marwah, C. Hyser, "Renewable and Cooling Aware Workload Management for Sustainable Data Centers", ACM SIGMETRICS/Performance, June 11-15 2012, London, UK.
![Page 32: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/32.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32
Prediction: Summary
PV Supply Prediction
• Search for most “similar” days in the recent past • Hourly generation estimated from corresponding hours of “similar” days
5 10 15 200
0.2
0.4
0.6
0.8
1
time (hour)
PV
Po
we
r (k
W)
actual power
predicted power
IT Workload Prediction
• Perform a periodicity analysis (e.g., Fast Fourier Transform)
• Use an auto-regressive model to predict workload from historical data
6 12 18 240
5
10
15
20
time (hour)C
PU
De
ma
nd
(n
um
be
r o
f C
PU
s)
predicted workload
actual workload
Cooling Capacity Prediction
• End-to-End Energy Modeling
0 5 10 15 200.1
0.15
0.2
0.25
0.3
0.35
Time (hour)
Po
we
r (k
W)
Outside Air Cooling Available Capacity
Ref: Breen, T.J. et. al. “From Chip to Cooling Tower Data Center Modeling: Validation of Multi-Scale Energy Management Model”, Proceedings of Itherm, June 2012
Ref: P. Chakraborty, M. Marwah, M. Arlitt, N. Ramakrishnan, Fine-grained Photovoltaic Output Prediction Using a Bayesian Ensemble, in Proceedings of the 26th Conference on Artificial Intelligence (AAAI'12), Toronto, Canada, July 2012
![Page 33: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/33.jpg)
Fine grained PV Prediction using Bayesian Ensemble
• Motivation
• Integration of renewable sources is an important goal of the smart grid effort
• PV output is variable and intermittent
• Knowledge of future PV output enables demand-side management and “shaping” in data centers
• Problem addressed
• Predict PV output for the next day
• Data
• Historical PV output data for about 9 months from the HPL Palo Alto site
• Weather data
6 12 18 240
0.2
0.4
0.6
0.8
1
1.2
Po
we
r(kW
)
Time(hour)
PV Supply
(AAAI 2012)
![Page 34: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/34.jpg)
Fine grained PV Prediction using Bayesian Ensemble
• Approach
• Extract daily profiles from training data
• Use ensemble of predictors
• Naïve Bayes
• K-NN
• Motif based
• Perform Bayesian model averaging
• Results
(AAAI 2012)
![Page 35: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/35.jpg)
Fine grained PV Prediction using Bayesian Ensemble
• Results
Error by weather condition Actual versus predicted
(AAAI 2012)
![Page 36: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/36.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 36
Planning: Supply-Side Aware IT Workload Planning
6 12 18 240
0.5
1
po
we
r (
KW
)
time (hour)
critical workload
non-critical workload
cooling power
6 12 18 240
0.2
0.4
0.6
0.8
1
1.2
po
we
r (
KW
)
time (hour)
critical workload
non-critical workload
cooling power
renewable supply
DemandOptimal Net
Zero Plandemand
shaping
Overall demand is “shaped” according to input constraints and operation objectives
Demand Shaping
Satisfy critical workload resource requirements
Planning Flow
A detailed workload scheduling and capacity allocation plan
• Workload scheduling plan
• IT resource and power capacity allocation
• Cooling micro-grid capacity allocation
electricity price
goals cooling capacity efficiency
energy storage
IT workload & SLAs
renewable supply
Workload Planning
Ref: Z. Liu, Y. Chen, C. Bash, A. Wierman, D. Gmach, Z. Wang, M. Marwah, C. Hyser, "Renewable and Cooling Aware Workload Management for Sustainable Data Centers", ACM SIGMETRICS/Performance, June 11-15 2012, London, UK.
![Page 37: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/37.jpg)
Power and Workload Visualization
low
medium
Before optimization After optimization
![Page 38: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/38.jpg)
Some other projects
• Anomaly detection (SensorKDD 2010)
• Energy Disaggregation (SDM 2011, AAAI 2013)
• Automating Life Cycle Assessment (IEEE Computer 2011)
• Building Energy Management (BuildSys 2011)
38 3 June 2013
![Page 39: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/39.jpg)
©2009 39
Anomalous Thermal Behavior Detection using PCA
– Example: Event (Anomaly) Detection
Start: 2009-09-28 16:44:34 End: 2009-09-28 23:58:34
Network Switch
Rack D5
Period of increased energy consumption (17 % increase)
Normal energy consumption
Switch turned on
(SensorKDD 2010)
![Page 40: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/40.jpg)
©2009 40
Energy Disaggregation
![Page 41: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/41.jpg)
©2009 41
Proposed Variant of Factorial HMM’s (SDM 2011)
![Page 42: Data Mining for Sustainable Data Centers€¦ · Create technologies, IT infrastructure and business models for the low-carbon economy 2% Total ... •Maximize use of renewable energy](https://reader034.fdocuments.net/reader034/viewer/2022042916/5f57ca8d988960796b1f1114/html5/thumbnails/42.jpg)
References • P. Chakraborty, M. Marwah, M. Arlitt, and N. Ramakrishnan. Fine-grained Photovoltaic Output Prediction using a Bayesian Ensemble, in Proceedings of the 26th Conference on Artificial Intelligence (AAAI'12), Toronto, Canada, 7 pages, July 2012, To appear.
• Z. Liu, Y. Chen, C. Bash, A. Wierman, D. Gmach, Z. Wang, M. Marwah, C. Hyser, "Renewable and Cooling Aware Workload Management for Sustainable Data Centers", ACM SIGMETRICS/Performance, June 11-15 2012, London, UK.
•Manish Marwah, Amip Shah, Cullen Bash, Chandrakant Patel, Naren Ramakrishnan, "Using Data Mining to Help Design Sustainable Products," IEEE Computer, August 2011
• Hyungsul Kim, Manish Marwah, Martin Arlitt, Geoff Lyon and Jiawei Han, "Unsupervised Disaggregation of Low Frequency Power Measurements", SIAM International Conference on Data Mining (SDM 11), Mesa, Arizona, April 28-30, 2011.
• Gowtham Bellala, Manish Marwah, Martin Arlitt, Geoff Lyon, Cullen Bash, "Towards an understanding of campus-scale power consumption." In ACM BuildSys, November 1, 2011, Seattle, WA.
• Manish Marwah, Ratnesh Sharma, Wilfredo Lugo, Lola Bautista, "Anomalous Thermal Behavior Detection in Data Centers using Hierarchical PCA," in SensorKDD in conjunction with KDD 2010.
• D. Patnaik, M. Marwah, Sharma, Ramakrishna, "Sustainable Operation and Management of Data Center Chillers using Temporal Data Mining," In ACM KDD, June 27 - July 1, 2009, Paris, France.
• Amip Shah, Tom Christian, Chandrakant D. Patel, Cullen Bash, Ratnesh K. Sharma: Assessing ICT's Environmental Impact. IEEE Computer 42(7): 91-93, July 2009.