#watitis2015 TOWARD A GREENER HORIZON: PROPOSED ENERGY SAVING CHANGES TO MFCF DATA CENTERS Naji Alamrony #watitis2015 AGENDA Objectives and Motivation Energy and Peak power reduction Reducing IT load Reducing Data Center energy waste Current State of MFCF Data Centers Proposed actions Conclusion #watitis2015 OBJECTIVES Propose changes to current data centers Reduce energy consumption Minimal implementation cost Significant rewards Getting UW support/involvement #watitis2015 PEAK POWER VS ENERGY Energy determines Size of energy bill Battery life Peak power determines Capacity Electrical Cooling #watitis2015 MOTIVATION Monetary Energy bills Environmental greenhouse gases emission Capacity Efficient use of the datacenter building Cost of building new data center #watitis2015 ENERGY CONSUMPTION COMPARISON Ref: Data Center Efficiency Assessment, NRDC, August 2014 #watitis2015 MFCF ENERGY CONSUMPTION Estimated Energy cost from $10,000 to $15,000/Month #watitis2015 RESTRICTIONS Cost start cheap but effective High availability Performance Virtualizability Capacity (Data center) Cooling Electricity Spatial using existing data center #watitis2015 REDUCING PEAK POWER/ENERGY #watitis2015 BACKGROUND IT LOAD POWER, EFFICIENCY VS. UTILIZATION Reference: The case for energy-proportional computing, Barroso, 2007 #watitis2015 BACKGROUND IT LOAD #watitis2015 BACKGROUND - IT MACHINES UTILIZATION 1 DAY #watitis2015 BACKGROUND - IT MACHINES UTILIZATION 2 WEEKS #watitis2015 REDUCING IT LOAD New machines: Use properly sized machines RAM CPU Storage (SSD vs. HDD) Power supply Use efficient class machines Blade server Micro server #watitis2015 REDUCING IT LOAD (CONT.) When using blades consider cooling capacities (up to 30kW/rack) SSD drives is recommended More blades on a chassis more efficient #watitis2015 REDUCING IT LOAD (CONT.) Improve utilization External power management policies ON/OFF policy Suspend polices Internal power policies Enable c-states (sleep modes) Enable p-states (DFVS) Virtualization Provides elasticity for cluster Consolidate light weight services #watitis2015 INCREASE DATA CENTER EFFICIENCY REDUCING COOLING LOAD Measured air flow Input (vertical) = output (horizontal) Tight hot/cold aisle containment Open aisle (currently used) Cold aisle containment (better) Hot aisle containment (recommended) Higher cold aisle temperature ASHRAE recommends up to 27 #watitis2015 HOT AISLE CONTAINMENT #watitis2015 ASHRAE PSYCHROMETRIC CHART #watitis2015 SAFE TEMPERATURE LIMITS Device Temperature Specs CPU Memory GPU Hard drive I/O devices #watitis2015 FREE COOLING Use of free cooling Air-side economizer Water-side economizer #watitis2015 COOLING DEGREE-DAY Cooling degree-days (CDD) A measure of how much (in degrees), and for how long (in days), the outside air temperature was above a certain level (base temperature) CDD is proportional to cooling energy Waterloo CDDs FromBase temp CDD 27 352 #watitis2015 POWER DISTRIBUTION WASTE Require infrastructure change: Higher voltage distribution (480, 220v vs. 120) DC distribution Reduce electricity conversion stages #watitis2015 MFCF SERVER ROOMS M (~ 50kW)MC-3015 (~ 100 kW) #watitis2015 CURRENT STATE OF MFCF DATA CENTER Lack of power saving measures No air flow Management/Analysis No efficiency measure (PUE/DCiE) No power-usage monitoring No use of machines utilization logs #watitis2015 PROPOSED ACTION "To measure is to know - if you cannot measure it, you cannot improve it - Kelvin #watitis2015 PROPOSED ACTIONS (CONT.) Determine data center capacity Start with CFD study Monitor energy utilization to Get baseline efficiency PUE, DCiE, or DCP Determine highest Energy consumers Plan load distribution across datacenter Implement Hot aisle containment Increase cold aisle temperature (23 ) #watitis2015 PROPOSED ACTIONS (CONT.) Monitor machine utilization pattern Determine: Comatose servers ON/OFF and suspend policies Virtualizable machines High availability clusters (elasticity) Consolidating services #watitis2015 PROPOSED ACTION (CONT.) Utilize machines remote management To apply ON/OFF and suspend policy To provide machines owners remote control Investigate free cooling options #watitis2015 SUGGESTION FOR THE UNIVERSITY OF WATERLOO Move electricity bill payment down to faculty level Incentive Free cooling worth investigation #watitis2015 CONCLUSION Many ways to reduce energy Start with cheaper, easier, most rewarding Weed out comatose server Define and implement ON/OFF policies Use Virtualization Tighten hot aisle containment Increase cold aisle temperature (23 to 27 ) Utilize economizers (air-side, water-side) Utilize measures for comparison: PUE, CDD #watitis2015 TO BE CONTINUED . I am hoping that by next year, I will be able to present a different kind of presentation. One that includes data, results and actual changes to improve energy efficiency Ultimately leading to a greener future #watitis2015 Questions? Ext #watitis2015 This is an optional area for you to put a message or caption for your photo. Delete it if you dont need to use it.