Energy Efficient Enterprise Computing and Green Datacenters

MassoudPedramUniversityofSouthernCaliforniaDept.ofElectricalEngineering

SmartEnergyDay(Dec.14,2010)IntegratedSystemsCenter,EPFL,Switzerland

Outline

•  ICTEcosystemandICTSustainability

•  LifecycleAnalysisandEnvironmentalFootprintofICT•  CloudComputingasToday’sDrivingPlatform

•  EnergyUseandPeakPowerCrisis•  PowerMinimizationandManagementStrategiesand

Solutions

•  EmergenceofSmartGridandDynamicEnergyPricing•  Conclusion

2

InternetUsageTrend

•  People,organizations,andgovernmentsarebecomingincreasinglydependentontheInternetandinformationandcommunicationstechnologies(ICT)

•  AsICTbecomesmoredeeplyintegratedwithamyriadofsocietalinfrastructure,ICTfailuresandinefficienciescascadethroughmanyothercriticalinfrastructures,makingsuchproblemsuntenablycostly

M. Pedram, USC 3

TheInternetofThings

4 M. Pedram, USC

Audio/Video Distribution Network

Core Network Metro/Edge

Network

Data Center

Access Network

PC Laptop iPhone Medical devices Sensors Actuators Home appliances …

TheICTEcosystem

TheICTecosystemencompassesthepolicies,strategies,processes,information,technologies,applicationsandstakeholdersthattogethermakeupatechnologyenvironmentforacountry,governmentoranenterprise.

‐‐Source:HarvardLaw

Internet

Cloud computing and switching centers

Data and service centers

Servers and thin mobile clients

5 M. Pedram, USC

ICTSustainability•  ICTsustainabilitygoesbeyondgainsinenergyefficiency;

Instead,theICTmustoffsetitsown(growing)environmentalimpacts(e.g.,carbonfootprint)onanabsolutebasis,byreducingitstotalpollutionandconsumptionofresources

6 M. Pedram, USC

–  TheICTcanplayanimportantroleinoffsettingenvironmentalimpactscreatedbyothermuchlargerportionsofoverallelectricitydemandinthenation

–  Notehowevera“reboundeffect”inwhichseveralindirecteffectsstemmingfromincreasedefficienciesincreaseconsumption,thusoverwhelmingsavings

–  Watchforthe“freerider”problem

LifeCycle(Cradle‐to‐Grave)Analysis•  Theanalysisofthefullrangeofenvironmentalimpactsofaproduct

requirestheassessmentofrawmaterialproduction,manufacture,distribution,use,anddisposal

•  Lifecycleenergyanalysis(LCEA)isanapproachinwhichallenergyinputstoaproductareaccountedfor,notonlydirectenergyinputsduringmanufacturing,butalsoallenergyinputsneededtoproducecomponents,materialsandservicesneededforthemanufacturingprocess

•  Netenergycontentistheenergycontentoftheproductminusenergyinputusedduringextractionandconversion,directlyorindirectly–  Differentenergyforms(heat,electricity,chemicalenergyetc.)havedifferent

qualityandvalue–  Ajouleofelectricityis2.6timesmorevaluablethanajouleoffuel

We must find a way to meet the increasing demand for energy without adding catastrophically to greenhouse gases. —Ray Orbach, Undersecretary for Science, U.S. Department of Energy

7 M. Pedram, USC

CO2FootprintofSomeConsumerDevices

•  TheCO2EmissionpercentageofBladeserverseasilyrisesto>70%oftotalCO2emissions

•  Needdifferentstrategiesforeachmarketsegment8 M. Pedram, USC

GreenHouseGases:ReductionPotentialwithICT

Commercial

2009 U.S. Greenhouse Gas Inventory Report, April 2009 http://www.epa.gov/climatechange/emissions/usinventoryreport.html

9 M. Pedram, USC

•  Risingenergyconsumptionandgrowingconcernaboutglobalwarming–  WorldwideGHGemissionsin2006:27,225MMTofCO2Equivalent

•  Majorcontributors:Transport,Industrial,Residential,Commercial•  ImprovedefficiencywithInformationTechnology(IT)usage

–  29%expectedreductioninGHG–  Equaltogrossenergyandfuelsavingsof$315billiondollars

PowerDissipationinModernComputingSystems

•  Highendservers/desktopononeendofthespectrumwithembeddedsystemsontheother

•  Powerdissipationbroadlycategorizedin:•  Digital/Analogcomponents:Microprocessor,memorychips,baseband

processorsetc.•  Mechanicalcomponents:harddrives,fansetc.•  Embeddedsystems:dominatedlargelybydigital/analogcomponents

unlikeservers

Blade server/Desktops Embedded Systems: e.g., iphone

10 M. Pedram, USC

DesigningEnergyEfficientSystems

•  Approachestoenergyefficiencycanbeclassifiedintwobroadcategories:

–  Static(designtime)solutions:•  Oblivioustotheusecase/workloadtheyarerunningunder•  Examplesincludeoptimalplacementandrouting,optimizedcircuitdesign/architecture,staticcompileroptimizationgeneratingenergyefficientcode,codeoptimization

–  Dynamic(runtime)solutions:•  Makeuseoftheworkload/environmentunderwhichthesystemisemployed

•  Examplesincludedynamicvoltageandfrequencyscaling,runtimearchitecturaladaptationlikeissuewidthreduction,reducingtheavailableregisterfilesize,memorymanagement,taskscheduling,resourcemanagement,etc.

11 M. Pedram, USC

Idleness,StateTransitions,andPowerSavings

12 M. Pedram, USC

Pow

er S

avin

gs

Id

lene

ss

Source: W-D. Weber, Google

Time scale ns ms sec min hour day

Components sleeping Machines sleeping

Better active states

More sleep states

Thread packing Job packing (to cores) (to machines)

GQ

Task

A

ssig

nmen

t

PMU Core

Task

LQ

MinimumEnergyCMPDesignThroughDVFS

•  ChipMultiprocessorsystemwithMidenticalcores–  Per‐coreDVFSwithN(voltage‐frequency)configurations–  LocalQueue

•  PowerManagementUnit•  GlobalQueue•  TaskDispatcher•  Tasks

–  Expectedexecutiontime,τ–  Expectedinstructionspercycle

•  Objective:MinimizethetotalEnergyconsumptionofcoresinamulticoresystem

•  Constraints:Minimumrequiredthroughput,IPSreq•  Variables:•  Numberofactivecores–totalnumberofcores:M•  v‐fsettingsofcores–totalsettings:J•  Taskdistribution–totalnumberoftasks:K

13 M. Pedram, USC

AHierarchicalSolution•  DeterminethenumberofONcores

–  ONcoresareinC0whenactiveor C2(haltorsleep)whenidle

•  DetermineoperatingfrequencyofONcores–  Afeedback‐basedcontrolmethodisadoptedforDVFSsetting

•  Thisisneededduetoinherentuncertaintyandvariabilityoftaskcharacteristics

•  PIController:controlleradjuststhev‐fsettingtomatchtherequiredthroughputbasedontheobservederror

•  FeedbackcontrolloopdeterminesasingleoptimumfrequencyforallcoresandthentheQuantizerwouldtranslatefopttoavailableDVFSsteps

•  FindafeasibleassignmentoftaskstotheONcores

14 M. Pedram, USC

EvaluationResults•  Comparisontoarelativelyenergy‐efficientbaselinePM

–  Itimplementsthesamepowerreductiontechniquesasourmethod–  ItutilizesopenloopDVFS,anddoesnotsupportsmartwakeupandshutdown

•  Thefigurecomparesthefrequencysettingandthroughput–  ThebaselinePMalwaysrunswithallcoresON–  Theclosedloopfrequencyisalwayslowerforsamethroughputconstraint–  Inthesecond100ms,thebaselinechooseslowerfrequencywithm=4cores

runningwhileourproposedmethodusesm=3cores•  TheproposedPMgainsanoverallenergyconsumptionof17%

15 M. Pedram, USC

CloudComputingasToday’sDrivingPlatform

16 M. Pedram, USC

•  Widespreadacceptanceandrapidgrowthofcloudcomputingareduetomanyfactorsincluding–  Moore’sLawincomputing;storagebecominglessexpensive;increaseinwide‐

areanetworkbandwidth;web‐basedapplicationdeliveryframeworks;andthehighcostofITinsourcingtoensuresecurity,scalability,failover,andsoftwareupdates

•  Weexpectanexplosioninthenumberofcloudservicesinthecomingdecades

•  Cloudcomputingprovidesamodelforenablingconvenient,on‐demandnetworkaccesstoasharedpoolofconfigurablecomputingresourcesthatcanberapidlyprovisionedandreleasedwithminimalmanagementeffortorservice‐providerinteraction

CDNISP

Client

Datacenter

CloudInfrastructure:ServicePlane

17 M. Pedram, USC

PaaS User

IaaS Provider ◦  Platform(knownasPlatformasaService)◦  Asetofsoftwareandproductdevelopmenttoolshostedontheprovider'shardwareinfrastructure.DeveloperscreatecustomizedapplicationsonthisinfrastructureovertheInternet◦  Applicationdevelopmentframeworkse.g.,GoogleAppEngineandAzureServicesPlatform

◦  Software(knownasSoftwareasaService)◦  Theprovidersuppliesthehardwareinfrastructureandthesoftwareproduct,andinteractswiththeuserthroughafront‐endportal◦  Basicservicesareprovidede.g.,Authentication,Billing&metering,Search,Web‐basedemail,inventorycontrol,databaseprocessinge.g.,AmazonEC2,MicrosoftAzure

•  Aserviceplanecomprisedofthefollowing:◦  Infrastructure(knownasInfrastructureasaService)◦  Theprovideronlysuppliesthehardwareinfrastructurewhiletheconsumersprovisionprocessing,storage,networks,etc.,andrunarbitrarysoftwareandapplications

ExamplesofKeyChallenges

•  HowdoesoneguaranteeahighdegreeofreliabilityandavailabilityofcriticalInformationandCommunicationsInfrastructure(ICI)resourcesthatreliesonheterogeneityandreplicationatmultiplelevels,rangingfromhardwaretoserviceproviders?

•  HowdoesoneequiptheICItohandleordersofmagnitudemoredigitaldataandservicerequestssecurelyandcosteffectively?

•  HowshouldICTcomponentsbearchitectedsoastoachievetrueenergyproportionalityintheamountofworkdelivered?

•  WhataretheeconomicandpublicpolicyimplicationsofthedramaticchangesinsocietalaccesstoresourcesbroughtaboutbytheICI?

18 M. Pedram, USC

InternetGrowthisDrivingDataCenterUsage

Source: ADC Source: Gartner

Blade Server

Corporate data growing 50 fold in three years. —2007 Computerworld

19 M. Pedram, USC

EmergenceofMegaDataCenters

Microsoft’s Chicago Datacenter: Entire first floor is full of containers; each container houses 1,000 to 2,000 systems; 150 - 220 containers on the first floor.

20 M. Pedram, USC

• Threedrivershaveleda“datacentercrisis”•  Demandfordigitalservices

•  IncreaseinenergydissipationofIT

•  Environmentalconcerns

• Datacentersconsume~2%ofallUSelectricity

• Datacenterannualgrowthof15%isunsustainable

• Datacenterpowerprojectedtobe>8%ofUSpowerby2020

•  Datacentercarbonemissionsareprojectedto

exceedthoseoftheairlinesby2020

• Needaparadigmshiftindatacenter

computingtoputusonamoresustainable

andscalableICTenergyefficiencycurve

GreenDataCenters:AStrategicNecessity

02

46

810

121416

Exab

tes/Mon

th

0500100015002000250030003500

Thou

sand

s

InternetTrafficServerShipments

02

46

810

121416

Exab

tes/Mon

th

0500100015002000250030003500

Thou

sand

s

InternetTrafficServerShipments

Source:Gartner

2006 07 08 09 10 11 122006 07 08 09 10 11 12

0

20

40

60

80

100

120

140

160

200001 02 03 04 05 06 07 08 09 10 11 12 13 14

Power(B

KW

h)

CurrentTrendsImprovedOperationsGreenDataCenterGoal

0

20

40

60

80

100

120

140

160

200001 02 03 04 05 06 07 08 09 10 11 12 13 14

Power(B

KW

h)

CurrentTrendsImprovedOperations

Source:USEPA2007

21 M. Pedram, USC

PeakPowerDemandsofDataCenters

•  Apartfromthetotalenergyconsumption,anothercriticalcomponentisthepeakpower;thepeakloadonthepowergridfromdatacentersiscurrentlyestimatedtobeapproximately7gigawatts(GW),

equivalenttotheoutputofabout15baseloadpowerplants–  Thisloadisincreasingasshipmentsofhigh‐endserversusedindata

centers(e.g.,bladeservers)areincreasingata20‐30percentCAGR.

–  Ifcurrenttrendscontinue,powerdemandsareexpectedtoriseto12GWby2011

–  Indeed,accordingtoa2008Gartnerreport,50percentofdatacenterswillsoonhaveinsufficientpowerandcoolingcapacitytomeetthedemandsofhigh‐densityequipment

22 M. Pedram, USC

EcologicalFootprintofDataCenters

•  AlthoughcarbonemissionduetoeverykWhofelectricalenergyconsumedintheU.S.variesdependingonthetypeofpowergeneratorusedtosupplypowerintothegrid,anaverageconversionrateof0.433kgCO2emissionperkWhofelectricalenergycanbeassumed

•  Datacenterscurrentlygenerate23%ofallemissionsproducedbytheICTindustry,afigurethatcontinuestotrendupward(>79MMTofCO2in2011)–  TheIntergovernmental

PanelonClimateChange(IPCC)hascalledforanoverallgreenhousegasemissionsreductionof60‐80%below2000levelsby2050

23 M. Pedram, USC

PowerProfile

24 M. Pedram, USC

ServerUtilizationinaDataCenter

Servers are rarely completely idle and seldom operate near their maximum utilization, instead operating most of the time at between 10 and 50 percent of their maximum

25 M. Pedram, USC

Workload intensity profile for a typical week day

PDF of CPU utilization measured over a 6-month period

ARIMA, M. Martonosi Princeton

Reality:Non‐Energy‐ProportionalServers

26 M. Pedram, USC

Power dissipation of Intel Xeon 5400 series server at high (2.3GHz) and low (2GHz) settings

12

17

22

27

32

37

42

0 100 200 300 400

pow

er(W

)

total util(%)

Here P1 denotes server power dissipation at 100% utilization, whereas PU is power at utilization level of U An energy proportional system would have an Energy Inefficiency Factor (EIF) of one at all utilization levels

  VirtualizationisdisassociatingthetightbondbetweensoftwareandhardwarebyintroducingahypervisorbetweentheOSandhardware  Onecanthenusethesamehardwaretoserve

uptheneedsofthedifferentsoftwareservers:Oracle,MSSQLServer,Exchange,DynamicsCRM,etc.

  ItisalsopossibletorundifferentoperatingsystemssoonecouldrunMSSQLServer2008onWindows2008ServerandrunOracleonLinuxallrunningonthesamehardware

  Bydoingthis,theresourceswillbebetterutilizedsincewecaneasilyadd/migratevirtualmachines  ExamplesincludeMicrosoftHyper‐V,VMware

ESXServer3.5,LinuxXenhypervisor

27 M. Pedram, USC

Virtualization

No Server Consolidation

Server Consolidation

12

17

22

27

32

37

42

47

0 50 100 150 200

pow

er(W

)

Total CPU util (%)

Consolidation:PowerAnalysis

•  Coreconsolidationisineffectivefromapowersavingperspective–  Processorpowerdissipationis

nearlyindependentofwhichsubsetofCPUsisusedbytherunningdomain

•  Processorconsolidationisquitehelpfulinreducingthetotalpowerdissipation–  Reportedasafunctionoftotal

utilizationofallCPUsinthesystem

28 M. Pedram, USC

12

17

22

27

32

0 100 200 300 400

pow

er(W

)

Total CPU util (%)

Differentcoreconsolidationcases

•  ForagivennumberofvirtualCPUs,theresponsetimedecreasesasthenumberofactiveCPUsincreases–  NormalizedCPUutil=TotalCPU

util/(#ofactiveCPUs)

–  TruewhenthenormalizedCPUutilizationisfixed

•  ForagivennumberofphysicalCPUs,theresponsetimeincreasesasthenumberofvirtualCPU’sincreases–  TrueevenwhenthetotalCPU

utilizationisfixed29 M. Pedram, USC

Consolidation:PerformanceAnalysis

0.035

0.085

0.135

0.185

0.235

0.285

0 20 40 60 80 100

resp

onse

tim

e (s

)

Normalized CPU util (%)

Increasing # of active CPU’s

0.035

0.085

0.135

0.185

0.235

0.285

30 50 70 90

resp

onse

tim

e (s

)

Total CPU util (%)

Increasing # of virtual CPU’s

Power‐PerformanceTradeoff

•  Pareto‐optimalsurfaceofEnergy‐DelayProductvs.therequestarrivalrate–  Properselectionofthe

virtualizationconfiguration(#ofvirtualCPUsvs.physicalCPUsandtheirmappings)ofanenterprisecomputingsystemcanimproveenergyefficiencysubjecttogivenSLAs

30 M. Pedram, USC

Energy * response time product per request as a function of

virtualization configuration

20

25

30

35

40

45

50

55

60

0 20 40 60 80 100 120

ener

gy *

res

pons

e tim

e pr

oduc

t per

req

uest

(J *

m

s)

# of requests / s

SchedulingandResourceAssignment

•  Givenasetofresourcesandasetofclientswithrespectiveworkloads,howtoallocate/assignresourcestothesegivenclientworkloadssuchthat:–  Energyconsumptionisminimizedorprofitismaximized

Subjectto:

– Maximum/averageresponsetimeconstraint,minimumthroughputconstraint,resourcecapacityconstraint,maximumpowerbudget,networkbandwidthconstraint,etc.

31 M. Pedram, USC

Client 1

Client m

Task Distribution, Scheduling, and

Resource Assignment

Cluster 1

Cluster n

Server1

Server p . . .

. . .

. . .

ChallengesinResourceAllocation

•  Alarge‐scaledistributedcomputingsystem:–  ClientstypicallyimposesomeServiceLevelAgreements(SLAs)while

thecloudowneristryingtomaximizeitstotalprofit

–  Serverclustersareusuallyheterogeneousinthenumberandtypeofresourcestheycontrol

–  Assumeastepwiseutilityfunctionfortheclientsbasedontheirresponsetimerequirement

–  Mustuseathreedimensionalmodeloftheresourcesintheclusters,i.e.,computational,storageandnetworkingcapabilities

•  ImportantChallenges:–  Energyproportionality–  Resourceheterogeneity–  Workloadpredictionandmodeling

–  Infrastructuremodeling32 M. Pedram, USC

•  Scalingofenergyandexecutiontime:–  Foragivenapplication:

•  Itisnotnecessarythatasenergyperrequestreducesexecutiontimeperrequestincreases

–  Acrossapplication:•  Executiontimeaswellasenergymaynotscaleinasimilarmanneracrossdifferentapplications

•  HP_A:{3.2GHz,96GB,2TB}•  HP_B:{3.2GHz,48GB,1.5TBFlash}•  HP_C:{2.3GHz,48GB,1.9TB}•  HP_D:{2.3GHz,48GB,1.1TBFlash}•  Solidline:executiontime,i.e.,

servicetimeperrequest•  Dottedline:energyconsumption

perrequest

ImpactofResourceHeterogeneity

ADistributedAlgorithm•  Generateaninitialsolutionforassignmentofclientstoclustersand

allocationofprocessinganddatastorageresourcestothem–  Foreachclient,thebestpossibleclustertoexecutetheapplicationisfoundby

findingthehighestapproximatedprofitfortheclientoneachclusterwithrespecttothecurrentstateoftheclusters

–  Thisprocesscontinuesuntilallclientsareassignedtotheclusters•  Allocatecommunicationresourcestoassignedclients

–  AninstanceofMultiChoiceKnapsackProblem,solvedbyapseudopolynomialdynamicprogramming(DP)method

•  Maximizethetotalutilitybyimprovingtheresourceallocationwithinaclusterwithoutchangingtheassignedsetofclientstoit

–  AninstanceofMultidimensionalKnapsackproblem,solvedbyDP

•  IterativelyImproveclientassignmenttoclusters–  Ineachstepaclientispickedrandomlyandthebestclusterischosenwith

respecttothecurrentstateofeachclusterandthecharacteristicsoftheclient

34

SomeResults•  Thenumberofclusters,thenumber

ofdifferentserverclasses,andthenumberofdifferentutilityclassesaresetto5,10and5,respectively

•  Foreachutilityclass,meansoftheserviceratesfortheclientsaregeneratedwith(uniformlydistributed)randomvariablesbetween0.4and1

•  Thevalueofrequestarrivalrateforeachclientissetwitharandomvariablebetween0.5and4.5

•  Theutilityclassofeachclientissetwitharandomassignmentformtheexistingutilityclasses

35

PS: Proportional Share Scheduling

Premise:‐  HotspotsexistthatimpactHVACefficiencyandenergyconsumption

Approach:‐  Placeworkloadinmorepower‐efficientlocations

‐  PerformserverconsolidationandDVFSsetting

DynamicWorkloadPlacementbasedonCoolingEfficiency

1

3 5 0

10

20

30

A B C D E E D C B A Chassis number

Rac

k nu

mbe

r

Tem

pera

ture

num

ber

(°C

)

36 M. Pedram, USC

SmartGridandDynamicEnergy

Pricing

•  Day‐aheadpricepredictionisusefullyaccurate,althoughsomedeviationsoccurmid‐day

•  Informationcanbegatheredonbothdynamicelectricityusageandpriceswiththeaimofreducingenergyconsumptionofbothindividualhouseholdsandbusinesses

37 M. Pedram, USC

M. Martonosi Princeton

Summary•  Theseareexcitingtimeswithmanynewopportunitiesandchallengesdue

toplannedupgradestothePowerGrid,introductionofrenewablesourcesofenergy,smartmeteringanddynamicenergypricing,people’sawarenessofenvironmentalissues,adventandpopularityofsocialnetworking

•  MustbeabletocharacterizethelifecycleenvironmentalimpactsoftheICToncarbonemission,waterfootprint,andairpollution

•  Aholistic,cross‐layerapproachtoICTenergyefficiencyandgreennessisneeded,whichspans–  Applicationefficiencyandenergymanagement,micro‐architectureandsystem

design,storageandnetworks,resourcemanagementandscheduling

•  MustcontributetounderstandingthebasicissuesandoptionsavailabletopolicymakersandguidethemonhowtomakecrucialdecisionsaboutproposedchangesoftheICIandthe“inter‐cloud”byincentivizingcommercialandindividualusers,throughmarketdesign,orbyregulation

38 M. Pedram, USC