Federal Data Center Programs

7 X 24 Exchange October, 2007

William Tschudi

wftschudi@lbl.gov

Acknowledgement of LBNL Acknowledgement of LBNL sponsorssponsors

California Energy Commission – PIER California Energy Commission – PIER programprogram

Pacific Gas and Electric CompanyPacific Gas and Electric Company New York State Energy and New York State Energy and

Development Agency (NYSERDA)Development Agency (NYSERDA) US - Environmental Protection AgencyUS - Environmental Protection Agency US – Department of EnergyUS – Department of Energy

LBNL data center activitiesLBNL data center activities

Research RoadmapResearch Roadmap Benchmarking and 23 data center case Benchmarking and 23 data center case

studies studies Self-benchmarking protocol Self-benchmarking protocol Power supply &UPS efficiency studiesPower supply &UPS efficiency studies Standby generation lossesStandby generation losses Performance metrics – Computation/wattPerformance metrics – Computation/watt Demonstration projectsDemonstration projects EPA report to CongressEPA report to Congress DOE Save Energy Now programDOE Save Energy Now program

Data center energy roadmapData center energy roadmap

July, 2003 – Research July, 2003 – Research roadmap developed roadmap developed for the California for the California Energy CommissionEnergy Commission

The problem receded The problem receded for several years as for several years as excess data center excess data center space was absorbedspace was absorbed

Now the concerns are Now the concerns are back back

Current LBNL Current LBNL CACA focus focus

Developing LEEDDeveloping LEEDTMTM type criteria for data type criteria for data centerscenters

Evaluating modular and scalable cooling Evaluating modular and scalable cooling solutionssolutions

Promoting use of air economizersPromoting use of air economizers– study filtration optionsstudy filtration options– research failure modesresearch failure modes– collaborate with ASHRAEcollaborate with ASHRAE

Demonstrating spray cool technologyDemonstrating spray cool technology Continuing DC power initiativeContinuing DC power initiative

Current LBNL Current LBNL internalinternal focus focus

Designing new supercomputing facilityDesigning new supercomputing facility Goal - a model of energy efficiencyGoal - a model of energy efficiency

– Convened workshop of all supercomputer Convened workshop of all supercomputer vendorsvendors

– Better projections of future power needsBetter projections of future power needs– Pushing the envelop – environmental Pushing the envelop – environmental

conditions, power distribution conditions, power distribution Leading scientific computing facilityLeading scientific computing facility First hand exposure to barriersFirst hand exposure to barriers

Current LBNL Current LBNL FederalFederal focus focus

EPA activitiesEPA activities– Report to Congress follow-onReport to Congress follow-on– Energy Star server specEnergy Star server spec– Energy Star buildings – data center metricEnergy Star buildings – data center metric

DOE activitiesDOE activities– EERE - Save Energy Now EERE - Save Energy Now

Pilot assessmentsPilot assessments Profiler toolProfiler tool Assessment toolsAssessment tools

– FEMP – Technical assistanceFEMP – Technical assistance

Why Data Centers?Why Data Centers?

Highly energy-intensive and rapidly growing Highly energy-intensive and rapidly growing Consume 10 to 100 times more energy per Consume 10 to 100 times more energy per

square foot than a typical office buildingsquare foot than a typical office building Large potential impact on electricity supplyLarge potential impact on electricity supply

and distributionand distribution Used about 45 billionUsed about 45 billion

kWh in 2005kWh in 2005 At current rate, powerAt current rate, power

requirements couldrequirements coulddouble in 5 years.double in 5 years.

Where do data centers fit in?Where do data centers fit in?

Data centers are energy intensive facilitiesData centers are energy intensive facilities

– Server racks now designed to carry 25 kW or moreServer racks now designed to carry 25 kW or more

– Typical facility ~ 1MW, but can be > 20 MWTypical facility ~ 1MW, but can be > 20 MW Information factoriesInformation factories

– Nationally Nationally 1.5% of US Electricity consumption 1.5% of US Electricity consumption in in 20062006

Could double in next 5 yearsCould double in next 5 years

Critical national and global infrastructureCritical national and global infrastructure

– Few technology barriers to increased efficiency Few technology barriers to increased efficiency

– Good candidates for efficiency investments by Good candidates for efficiency investments by utilities to reduce peak loadsutilities to reduce peak loads

IT equipment load density – IT equipment load density – LBNL benchmark resultsLBNL benchmark results

IT Equipment Load Intensity

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Data Center Number

Wat

ts/s

q.

ft.

2003 BenchmarksAve. ~ 25

2005 BenchmarksAve. ~ 52

Rising cost of ownershipRising cost of ownership

From 2000 From 2000 –– 2006, computing performance 2006, computing performance increased 25x but energy efficiency only 8xincreased 25x but energy efficiency only 8x– Amount of power consumed per $1,000 of Amount of power consumed per $1,000 of

server spending has increased 4xserver spending has increased 4x

Cost of electricity and supporting infrastructure Cost of electricity and supporting infrastructure now surpasses capital cost of IT equipmentnow surpasses capital cost of IT equipment

Perverse incentives --- IT and facilities costs Perverse incentives --- IT and facilities costs separateseparate

Source: The Uptime Institute, 2007

IT Industry is taking actionIT Industry is taking action

www.climatesaverscomputing.org.

www.thegreengrid.com

What is US Government’s What is US Government’s role?role?

EPA & DOE want to be the catalystEPA & DOE want to be the catalyst – Stimulate competition on energy efficiencyStimulate competition on energy efficiency – Foster discussions between key stakeholdersFoster discussions between key stakeholders– Provide key recommendations (EPA Report to Provide key recommendations (EPA Report to

Congress)Congress)– Develop standardized protocols and metrics to Develop standardized protocols and metrics to

measure energy consumption (e.g., ENERGY STAR)measure energy consumption (e.g., ENERGY STAR)

Take the lead on assessing and implementing Take the lead on assessing and implementing best practices in federal datacentersbest practices in federal datacenters

Promote initiatives globally Promote initiatives globally

Findings from EPA report to Findings from EPA report to CongressCongress

Trends in data center energy use Trends in data center energy use Sector consumed about Sector consumed about 61 billion KWh61 billion KWh in in

2006 2006 – Equates to Equates to ~1.5%~1.5% total U.S. electricity total U.S. electricity

consumption and consumption and ~$4.5 billion~$4.5 billion– Federal sector: ~6 billion kWh and ~$450 Federal sector: ~6 billion kWh and ~$450

millionmillion

Projected to increase to Projected to increase to 100 billion kWh100 billion kWh in in 20112011– Equates toEquates to ~2.5% ~2.5% of total U.S. electricity of total U.S. electricity

consumption and consumption and ~$7.4 billion~$7.4 billion

Projected electricity useProjected electricity useVarious scenarios 2007 - 2011Various scenarios 2007 - 2011

Percentage of electricity Percentage of electricity delivered delivered to IT equipmentto IT equipment

IT Power to Total Data Center Power

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Data Center Number

Ra

tio

Average .57

Higher is better

Source: LBNL Benchmarking

Data Center total electrical Data Center total electrical demand/ IT equipment demanddemand/ IT equipment demand

Total Data Center Power/IT Power

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Center Number

Rat

io

Average 1.83

Lower is better

Source: LBNL Benchmarking

LBNL design targetLBNL design target

Total Data Center Power/IT Power

1.00

1.50

2.00

2.50

3.00

3.50

LBNL’s new supercomputer facility is being designed to achieve a ratio of 1.2

EPA report EPA report recommendationsrecommendations

Standardized performance measurements for IT Standardized performance measurements for IT equipment and data centersequipment and data centers

– Development of benchmark/metric for data centersDevelopment of benchmark/metric for data centers

– ENERGY STAR label for serversENERGY STAR label for servers

Leadership by Federal GovernmentLeadership by Federal Government

– Publicly report energy performance of datacenters Publicly report energy performance of datacenters

– Conduct energy efficiency assessments in all Conduct energy efficiency assessments in all datacenters in 2-3 yearsdatacenters in 2-3 years

– Architect of the Capital, implement server-related Architect of the Capital, implement server-related recommendations in Greening of the Capital reportrecommendations in Greening of the Capital report

Recommendations cont.Recommendations cont.

Private sector challengePrivate sector challenge– CEOs conduct CEOs conduct DOE Save Energy NowDOE Save Energy Now

energy efficiency assessments, implement energy efficiency assessments, implement measures, and report performancemeasures, and report performance

Information on best practicesInformation on best practices– Raise awareness and reduce perceived risk of Raise awareness and reduce perceived risk of

energy efficiency improvements in datacenterenergy efficiency improvements in datacenter– Government partner with private industry: Government partner with private industry:

case studies, best practicescase studies, best practices Research and developmentResearch and development

– Develop technologies and practices for Develop technologies and practices for datacenter energy efficiency (e.g., hardware, datacenter energy efficiency (e.g., hardware, software, power conversion)software, power conversion)

DOE data center program DOE data center program objectivesobjectives

Build tools, expertise, Build tools, expertise, and strategy in and strategy in collaboration with collaboration with industryindustry

Raise awareness of the Raise awareness of the opportunityopportunity

Recognize industry Recognize industry leadersleaders

DOE program strategyDOE program strategy

Build on Build on Save Energy NowSave Energy Now model model

– DOE deployed software tools,DOE deployed software tools,training curriculum, and qualifiedtraining curriculum, and qualifiedexperts to train and work withexperts to train and work withstaff at large U.S. industrial plants.staff at large U.S. industrial plants.

– 65% of recommended actions now65% of recommended actions nowcompleted, in progress, or planned.completed, in progress, or planned.

With industry input, develop appropriate tools, With industry input, develop appropriate tools, training, and qualified experts to improve data training, and qualified experts to improve data centers.centers.

Conduct pilots, peer reviewed by industry, promote Conduct pilots, peer reviewed by industry, promote and facilitate industry implementationand facilitate industry implementation..

Data center programData center program

Build on Build on Save Energy NowSave Energy Now modelmodel

With industry input, develop With industry input, develop appropriate tools, training, and appropriate tools, training, and qualified experts to improve qualified experts to improve data centers.data centers.

Conduct pilots, peer reviewed Conduct pilots, peer reviewed by industry, promote and by industry, promote and facilitate industry facilitate industry implementationimplementation..

DOE near term planDOE near term plan

Build strong liaisons and partnerships with industry

Develop robust new energy assessment program

Begin to develop tools and info on best practices– Assessment framework and energy profiling tool

– System level assessment protocol and analysis tools

Partner with industry – joint development/peer review Conduct pilot assessments Provide awareness training

Screen for demonstration opportunities

Update Federal procurement specifications

Server Load/ComputingOperations

Cooling Equipment

Power Conversion & Distribution

AlternativePower

Generation

• High voltage distribution

• Use of DC power

• Highly efficient UPS systems

• Efficient redundancy strategies

• Load management

• Server innovation

Energy efficiency opportunities Energy efficiency opportunities are everywhereare everywhere

• Better air management

• Move to liquid cooling

• Optimized chilled-water plants

• Use of free cooling

• On-site generation

• Waste heat for cooling

• Use of renewable energy/fuel cells

Potential benefits of improved Potential benefits of improved data center energy efficiencydata center energy efficiency

Save 20 billion kWh per year by 2015Save 20 billion kWh per year by 2015

– Worth $2 billion, Worth $2 billion, ≈ ≈ annual annual electricity use in 1.8 million electricity use in 1.8 million American homesAmerican homes

Potentially defer need to build 2,300 Potentially defer need to build 2,300 MW of new generating capacityMW of new generating capacity

– Avoid 3.4 million metric tons of Avoid 3.4 million metric tons of carbon emissions (like taking carbon emissions (like taking 675,000 cars off the road)675,000 cars off the road)

Extend life and capacity of existing Extend life and capacity of existing data center infrastructuresdata center infrastructures

Typical data center energy end Typical data center energy end

useuse

Server Load/ComputingOperations

Cooling Equipment

Power Conversions

& Distribution100

Units

33 UnitsDelivered

35 Units

Performance varies

Data Center Server Load

51%

Data Center CRAC Units

25%

Cooling Tower Plant4%

Electrical Room Cooling

4%

Office Space Conditioning

1%

Lighting2%

Other13%

Computer Loads67%

HVAC - Air Movement

7%

Lighting2%

HVAC - Chiller and

Pumps24%

The relative percentages of the energy actually doing computing varied considerably.

Source: LBNL case studies

U.S. data centerU.S. data centerenergy efficiency goal by 2011energy efficiency goal by 2011

Industry, DOE and partners perform 1,000 Save Energy Industry, DOE and partners perform 1,000 Save Energy Now assessments using a common protocol and tool suite Now assessments using a common protocol and tool suite to yield savings of 5 billion kWh per yearto yield savings of 5 billion kWh per year

Industry-DOE energy efficiency campaign saves an Industry-DOE energy efficiency campaign saves an additional 20 billion kWh per yearadditional 20 billion kWh per year

Total energy savings of 25 billion kWh per year by 2011; Total energy savings of 25 billion kWh per year by 2011; 25% below current trends, which yields:25% below current trends, which yields:

- $2 billion of electricity cost savings$2 billion of electricity cost savings- carbon emission reduction of 5 MTCEE; equal carbon emission reduction of 5 MTCEE; equal

to the emissions from 3.5 million carsto the emissions from 3.5 million cars- All the electricity consumed in the state of All the electricity consumed in the state of

Utah in one yearUtah in one year

Web-based resourcesWeb-based resourceshttp://hightech.lbl.gov/datacenters.htmlGood starting point for those seeking efficiency measures

Best Practices

Case Studies

Design Guidance

Self-benchmarking Guide

Benchmark data

Other Reports (demonstrations)

Additional resources Additional resources

Guides available through PG&E’s Energy Design Resources Website http://hightech.lbl.gov/dctraining/

TOP.html

How can industry participate? How can industry participate?

Participate in EPA and DOE workshopsParticipate in EPA and DOE workshops Participate in peer review of products, protocols and Participate in peer review of products, protocols and

best practicesbest practices Conduct self-benchmarking and report resultsConduct self-benchmarking and report results

– Use tools from LBNL site and download protocol at: Use tools from LBNL site and download protocol at: http://hightech.lbl.gov/datacenters.htmlhttp://hightech.lbl.gov/datacenters.html

Participate in DOE data center energy efficiency Participate in DOE data center energy efficiency program program

– Register on web site to get regular updates Register on web site to get regular updates www.eere.energy.gov/datacenters/

Paul Scheihing

Paul.Scheihing@ee.doe.gov

202-586-7234

Contacts

Andrew Fanara

Fanara.Andrew@epa.gov

Bill Tschudi

wftschudi@lbl.gov

510-495-2417

websites: websites:

http://hightech.lbl.gov/datacenters/http://hightech.lbl.gov/datacenters/www.energystar.gov/datacenters

www.eere.energy.gov/datacenters/www.eere.energy.gov/datacenters/