Energy Efficiency Policy at

CESGA

Javier García Tobío , Ignacio López Cabído, Carlos

Fernández Sánchez, Javier Cacheiro López

Centro de Supercomputación de Galicia

Zaragoza, January 2013

INDEX CESGA:

- Institution

- Infrastructures

- Evolution of uses

- Activities and Services

- Some Projects

INDEX CESGA:

- Energy Efficiency Policy:

- Reasons and Objective

- Before the Plan

- The Plan:

- Acivities

- Metrics

- Results

- Next Actions

CESGA MISSION STATEMENT

To provide high performance computing and advanced

communications resources and services to the

scientific community of Galicia and to the Spanish

National Research Council (CSIC), as well as, to

institutions and enterprises with RTD activity.

To promote high quality research in Computational

Science in close collaboration with the research

community from Galicia as well as from other

regions or countries all over the world; contributing in

this way to the advancement of science, to transfer

technology to industry and administrations, and

as consequence, to the wellbeing of society as a

whole.

CESGA BOARD OF TRUSTEES

• Non Profit Foundation

• Regional Government of Galicia

• National Reseach Council of Spain

PARTNERS

CESGA USER COMMUNITY

Universities of Galicia

Regional Research Centres

Spanish National Research Council Centres

Other National and Foreign Institutions:

Hospitals: RTD Departments

Private Companies

Technological Centres

Other Non Profit RTD Centres

Excellence in Management

ISO

9001:2008 Quality Management

UNE166002

R&D Management

CESGA IS:

• Service Provider

CESGA IS:

Research Centre:

Joint RTD&i Project Promoter

Projects (2001-2011)

-UE 30

-National 19

-Regional 70

-With Industry & Others 31

Total 150

CESGA IS:

Outstanding e-Infrastructures for RTDi

– Computing platforms

– Data storage servers

– Communications network

CESGA IS:

Government Advisor

-Participation in EC prospective activities

-Co-author of the Spanish e-Science White Paper

-Galician Representative in Spanish e-Science Initiative

-Galician Representative in IBERGRID

-Galician Representative in RedIRIS

-Galician Representative in National Grid Initiative (Spanish NGI)

-Promoter of the Galician Universities Library Consortium (BUGALICIA)

-Promoter of the Galician Open Software Initiative (Mancomun)

-Advisor in the drafting of Galician Research Technological Development &

Innovation Plans

-Advisor in the drafting of Galician Information Society Plans

- …

CESGA IS:

Promoter of Scientific Thematic Networks &

Technological Platforms

-Promoter of Galician Technology Centres Network (RETGALIA)

-Promoter of Galician ICT Technology Platform (Vindeira)

-Partner & Promoter of “Instituto Tecnológico de Matemática Industrial (ITMATI)”

-Partner & Promoter of various scientific thematic networks:

• Galician Colorectal Cancer Research Network

• Galician Bioinformatics Network

• Galician Network of High Perfomance Computing

• Mathematica Consulting: Transfering Math to Industry

• Galician e-learning Thematic Network

• …

CESGA IS:

European Project Best Practice

– eBSN: European e-Business Support Network (e-MINDER Project).

– Lifelong Learning Initiatives (e-HOSPITAL Project).

– Youth Employment Support, New Skills Network (YES Project).

– European Grid for e-Science Production Site 2008 (EGEE Project).

– European Grid Initiative (EGI Project)

CESGA IS:

Partner to Universities’ Excellence Campuses

- Partner of International Campus of Excellence in

Ocean Science for the EuroRegion Galicia – North of

Portugal

- Partner of International Campus of Excellence for Life

Science

CESGA IS:

Promoter of e-Infrastructure Knowledge

- Partner of Official Post Graduate Programmes

74 Ph.D. Theses defended using CESGA computing resources (2003 – 2011)

- Masters Programme in Mathematical Methods and Numerical Simulation in Engineering and Applied Sciences.

- Masters Programme in High Performance Computing

Technology evolution at CESGA

Evolution of CPU usage

Computing users

Users scientific production

RECETGA exchanged traffic

CESGA USER TECHNICAL HELPDESK SERVICE REQUEST IN 2011

HPC SERVICES USER SATISFACTION LEVELS

CESGA’S STAFF

CESGA: A selection of significant Projects

Galician Regional Weather Forecasting Service

Galician Regional Weather Ocean Service

More detailed Climate simulations to better inform the Intergovernmental Panel on Climate Change (IPCC)

UVIGO

Juan Antonio Añel Cabanelas

Puertos del Estado

Spanish Port Authority

Enrique Álvarez Fanjul

European IBI Area: Ocean Forecasting Service

Oceanographic simulation to better understand fisheries population dynamics

Instituto Español de Oceanografía

Manuel Ruiz Villarreal

Galician Public Health Administration

ULTRASEQUENCING SERVICE: complete genome sequencing

FGMX – USC – CHUS - CESGA

Ángel Carracedo

Fundación Galega de Medicina Xenómica

Molecular Docking for fast track malaria drug design

Scientists Against Malaria

Hugo Gutierrez de Terán

Fundación Galega de

Medicina Xenómica

Molecular Docking techniques aimed at design of Alzheimer drugs

UAB – USC - FGMX

Xavier Daura

Hugo Gutierrez de Terán

Simulation aimed at the optimization of radiotherapy treatment dosage

CHUVI-CHUS-USC-CESGA

Proxecto e-IMRT

New material Design for nuclear plant safety

Instituto de Fusión Nuclear UPM

Sergio Eduardo Courtin Vega

Computational fluid dynamics for new turbine design

CENER

Sugoi Gómez

CTAG : Galician Automotive Technology Centre

Numerical Simulation

for Engineering

Large scale electromagnetism simulation for improved radar systems

UVIGO – UNEX – CESGA

Fernando Obelleiro Basteiro

Exploring complexity to better describe Ecological networks

Estación Biológica de Doñana

CSIC

Jordi Bascompte

European Grid Initiative (EGI): Monitoring & Accounting Service

CESGA DATA CENTRE

FINIS TERRAE SUPERCOMPUTER

CESGA DATA CENTRE: FinisTerrae

CESGA ENERGY EFFICIENCY PLAN (CEEP)

CEEP: REASONS

THE MOST IMPORTANT COSTS AT CESGA:

- PERSONNEL

- ELECTRICITY

- MAINTENANCE

CEEP: REASONS

IMPACT OF COST OF ENERGY AT CESGA:

- INCREASING OF IT RESOURCES

- INCREASING OF KW COST

- BUDGET RESTRICITIONS

- CONTRIBUTION TO REDUCTION OF CO2 EMISION

CEEP: OBJECTIVE

OBJECTIVE OF THE ENERGY EFFCIENCY PLAN:

ACHIEVE SIMILAR INDICATORS TO THE MOST

EFFICIENT SUPERCOMPUTING CENTRES

CEEP: BEFORE THE PLAN

SOME CONDITIONS:

- PLAN DEVELOPED AN MANAGED BY SYSTEMS DPT.

- SPECIAL SUPPORT BY THE DIRECTION

- IMPLICATION OF ALL DPTS.

- ROI IN SHORT-MEDIUM TERM

- SPECIAL ATTENTION TO RAS

- TIME PERIOD: 2010-2015

CEEP: BEFORE THE PLAN

SOME PREVIOUS ACTIONS:

- TECHNOLOGY OBSERVATION.

- ANALYSIS THERMOGRAPHIC OF COMPUTERS

ROOM AND INFRASTRUCTURES.

- INITIAL STUDY MADE BY AN EXTERNAL AUDITOR

CEEP: BEFORE THE PLAN

- INITIAL STUDY MADE BY AN EXTERNAL AUDITOR

INITIAL STATUS ANALYSIS (2009)

51

CLIMATISATION

Chilled Water Plants: 2 x 580 KW8

CRAH: 8 x 150 Kw

GENERATOR & UPS

DATA INTAKE

54

Electrical line schemas

Building blueprints

Equipment technical data & manuals

Energy consumption data

Consumption and billing registry

55

ELECTRIC MEASUREMENTS

1. General Switch

2. PP SAI 1

3. PP SAI 2

4. UPS 60

5. UPS 120

6. CPD Climatisation

7. Centre Climatisation

8. Chiller Plant 2

9. UPS Climatisation unit 2

10. Lighting General Switch

11. Basement Lighting General

Switch (IGALS)

12. Exterior Lighting

13. CPD Lighting

14. Corridor Lighting

15. Main Floor Lighting

16. First Floor Lighting

56

Puntos existentes que favorecen el ahorro energético • Free Cooling- Enfriadoras

• Recuperación Calor-Enfriadora

• Bomba de calor

Sistema de Climatización

1 2 3 4 5 6 7 8

EQUIPOS L-15

ENFRIADORAS

RECUPERACIÓN

CALOR ENFRIADORA

SALA CPD

GF1

GF2

FAN-COILS

OFICINAS

SIST. PASILLOS

B.C

DATA CENTRE CHILL DEMAND

57

• Detectores de presencia sala CPD

• Lámparas fluorescentes con balasto electrónico zona centra planta primera

Sistema de Iluminación

58

LIGHTING, TEMPERATURE AND HUMIDITY

MEASUREMENTS (1/3)

59

LIGHTING, TEMPERATURE AND HUMIDITY

MEASUREMENTS (2/3)

60

LIGHTING, TEMPERATURE AND HUMIDITY

MEASUREMENTS (3/3)

61

THERMOGRAPHIC MEASUREMENTS

62

BUILDING CONSUMPTION DISTRIBUTION

63

POWER CONSUMPTION DISTRIBUTION

64

CLIMATISATION CONSUMPTION DISTRIBUTION

65

LIGHTING CONSUMPTION DISTRIBUTION

ADVISE FOR IMPROVEMENT

66

.

Improvements in control

Improvements in lighting

Improvements in climatisation

Improvements in computing servers

Improvements in the water

Substituting energy sources

CEEP: PLAN OF ACTIVITIES

ACTIVITY TYPE DATE COST IMPACT

- Monitor & Management integrated System Inf Q4, 10 Medium High

- Closing cold aisles Inf Q1, 10 Medium High

- Interface for temperature monitoring Inf Q3, 10 Medium

- Measurement of impact increasing of temperature in EDP Inf Q1, 11 Low

- Measurement of impact of increasing of temperature of water Inf Q1, 11 Low

- Increasing of temperature in EDP room Inf Q2, 11 Low High

- Increasing of temperature in water Inf Q2, 11 Low High

- Chillers watering cooling Inf Q2,11 Low Low

- Updating heating system Inf Q2, 12 High Medium

- Updating fan systems Inf Q4, 11 High High

- Free cooling optimisation and heat recovering Inf Q1, 12 Medium High

- Improvement of cold / hot aisles Inf Q1-Q4, 12 Low Medium

- Improvement of wires under floor deployment Inf Q2, 10 Low Low

- Deployment of comfort temperature. Management System Inf Q1, 11 Medium Medium

-Techno-económic study of diabatic system Inf 3Q,13

- Improvement of Humidity Management Inf Q3,12 Low Low

- Techno-economic study of hot water system Inf/IT Q1,13

- Effective acomplishment of rules for public buildings climatization Inf Q2, 11 Low Medium

- Climatization System

CEEP: PLAN OF ACTIVITIES

ACTIVITY TYPE DATE COST IMPACT

- Interface for energy monitoring Inf. Q4, 10 Low High

- Installation of capacitors Inf. Q2, 10 Medium Medium

- SAI's substitution Inf. Q4, 12 High Medium

- Photovoltaic deployement Inf. Q1, 14 High Low

- Reduction of standby energy comsumption in power generator Inf. Q1,14 Medium Low

- Techno-economic three generation energy study Inf. Q4,14

- Techno-economic study of DC vs AC Inf. Q3,15

- POWER SUPPLY

CEEP: PLAN OF ACTIVITIES

ACTIVITY TYPE DATE COST IMPACT

- Identification of low productive equipments IT 4Q, 10 Low

- Updating of critical low productive equipments IT 1Q11-4Q11 High High

- Updating of critical low productive storage systems IT 4Q, 11 High High

- Consolidating servers IT 2Q11-2Q12 High High

- Analysis of cloud for HPC IT 3Q, 10 Low

- Cloud deployment IT 1Q, 11 Low Low

- IT EQUIPMENTS

CEEP: PLAN OF ACTIVITIES

ACTIVITY TYPE DATE COST IMPACT LEADER

- Technology Observation IT/Inf Monthly Low

- Periodic External Auditories IT/Inf Yearly High Medium

- Following Auditories recommendations IT/Inf Yearly

- Best Practices: education and training IT/Inf Yearly Low

- Including energy efficiency like a value in public tenders IT/Inf Permanent Low High

- UNE-EN 16001:2010 Inf. Q4, 12 High Medium

- Integration of CESGA in GreenStart Network IT/Inf Q3,13

- Development of R&D in this field IT/Inf Permanent Low Low

- Staff awakening: seminars, information, etc. IT/Inf Permanent Low Low

- GENERAL ACTIVITIES

CEEP: MAIN METRICS

- PUE (Power Usage Effectiveness)

- KWh consumed

- Computational performance (GFLOPS) by kw

- Monthly budget for electricuty

- Temperature in computational nodes

- Temperature in water circuit

- Relative humidity in EDP room

CLOSURE OF COLD AISLES

CLOSURE OF COLD AISLES

CLOSURE OF COLD AISLES

CLOSURE OF COLD AISLES

CONCLUSIONS: savings obtained

-Mechanisms for server’s power saving:

33.900 €/year

- Consolidation of SVG nodes:

6.300 €/year

- Disk consumption vs Tape consumption:

575 €/year

- Free-AIR Cooling:

260.000 €/year

- Enclosing of cold aisles: 6 ºC temperature decrease

and increase of cold aisle temperature up to 27ºC

105.000 €/year

- CRAHs: variable speed fans

30.000 €/year

- Automatic suspension/turn off of inactive nodes:

22.000 €/year

- Partial renovation of FinisTerrae nodes

70.000 €/year

CONCLUSIONS: savings obtained

CURRENT AND FUTURE WORK

- Optimisation of the use of memories

internal hierarchy.

- Job planner that takes into account the

cost of KWh depending on the hour and

date.

- Renewable energy supply.

CESGA: a way to make EUROPE