ENERGY AND COST CASE STUDIES - European Commission · 2014. 8. 11. · Case Study 1 City of Bad...

ffi c

EEEEEEEEEEEEEEEENNNNNNNNNNNNNEEEEEEEEEERRRGY AAAAAAAANNNNND COST SSSSSSSSSAAAVINNNNNNGGGGGGGGSSSSSS BY ENERGYEFFIIIIIIIIICCCCCCCCIIENT SERVERS –CCCCCCCCCCCAAAAAAAAAAAAAAAAAASSSSSSSSE STUDIES

Case Study 1 City of Bad Soden am Taunus page 2 Energy effi cient IT system by consolidation and desktop virtualisation in a municipality

Case Study 2 STRATO AG page 8 96% energy savings through effi cient and high performance servers at Europe’s second largest web host

Case Study 3 Bundesumweltministerium page 14 Energy-effi cient IT Infrastructure at the Federal Ministry for the Environment, Nature Conservation and Nuclear Safety in Bonn, Germany

Case Study 4 „Padergreen“ page 20 Energy effi ciency by blade consolidation and virtualisation at Wincor Nixdorf in Paderborn

Case Study 5 Austrian Energy Agency page 23 Consolidation, virtualisation and power management deployed in an IT renewal project

Case Study 6 Encontrol AG page 30 Server virtualisation in a Swiss service company on SME level increased energy effi ciency

Case Study 7 Sozialwerk Nürnberg page 34 Energy-effi cient IT Infrastructure at “Sozialwerk Nürnberg”

1

Electricity consumption for hardware and infra-structure in data centres has been steadily growing in the EU over the past years and cur-rently amounts to approximately 40 TWh/a. This is about the order of magnitude of the annual power generation of 7-8 average n uclear pow-er plants. According to forecasts energy consumption will double during the next 4–5 years, unless effec-tive measures to improve energy effi ciency are going to be implemented.A very effective approach to reduce energy de-mand in data centres is the use of effi cient ser-ver technology. Such measures have a double effect, as each kilowatt hour saved on the IT-level, means less heat production and conse-quently less energy demand for cooling.The main technologies currently available to improve the energy effi ciency of servers are effi cient hardware, virtualisation and power management.Regarding the practical application of concepts

Effi cient server technology

Energy and cost savings in the data centre

in companies, there are still some obstacles to be met in order to tap the full energy saving poten-tials: mere lack of awareness of the existing saving potentials is just one of them. This is part-ly due to the fact that many companies do not have the appropriate tools nor technical know-how on hand to make saving potentials visible. A suffi cient monitoring of energy consumption is quite rare. Secondly, best practise for effective solutions is still lacking. Which measures for effective optimisation and cost savings have yet been shown in practise? What are the appro-priate technologies and what is the best ap-proach to achieve signifi cant savings? These are ques tions of major interest for the IT manager. However, attractive best practise cases going beyond typical marketing literature are still rare.

This report shall help to close this gap, presen-ting best practise cases of different organi-sations from the private and public sector, that show how certain effi ciency strategies for

servers have been implemented effectively. These cases cover small enterprises of 20 employees up to large data centres in the private and public area. The technologies and strategies applied in the cases include hard-ware consolidation, virtualisation and power management.

These case studies show that the use of appro-priate technologies allows to achieve energy savings between 25 and more than 90%. These potentials can be tapped most effectively in the course of overall renewal processes or, right from the beginning, during the planning of new data centres.

The case studies have been conducted as part of the EU-Project e-Server based on the programme Intelligent Energy Europe.

The e-Server-Consortium, February 2009

e-Server – an international project for enhancing energy- and cost effi ciency of servers and data centers within the framework of the Intelligent Energy Europe Programme.This project aims to demonstrate the enormous economical and ecological effi ciency potentials in the area of server-IT and server infrastructure and to disseminate energy and cost effi cient solutions to the server market .The best practice brochure at hand as well as the guidelines for the procure-ment and management of energy effi cient servers are central outcomes of e-Server.

Project team: Austrian Energy Agency (Project coordination), IBM, SUN, the University of Karlsruhe, the French Environment and Energy Management Agency and Robert Harrison Associates

Detailed information and downloads available onwww.effi cient-servers.eu

2

Background and project objectives

A MUNICIPAL IT RENEWAL PROJECTIn late 2007 the city of Bad Soden am Taunus started an IT renewal project to replace the old IT infrastructure of 16 distributed servers and old desktop tower servers. Options for the IT renewal included the 1:1 replacement of the old servers and desktops by new equipment and consolidation and virtualisation of the old inf-rastructure. The city of Bad Soden am Taunus decided to go for the• modernization of the network infrastructure

of the city,• consolidation and virtualisation of the server

infrastructure,• deployment of a Virtual Desktop Infrastruc-

ture for employee workplaces,• consolidation of the printer environment.Reasons for this decision were manifold, the most important was the expected ease of ad-ministration of a centralized IT infrastructure. Based on the increasing prices for energy there were also expectations to lower the previous energy usage or at least to keep it constant with the new IT architecture. Especially the deploy-ment of a Virtual Desktop Infrastructure was seen as a way to save energy.The consolidation and virtualisation of old

City of Bad Soden am TaunusEnergy effi cient IT system by consolidation and desktop

virtualisation in a municipality

Bernhard Przywara, Sun MicrosystemsAndreas Fiedler, City of Bad Soden am TaunusJens Wagener, Sun Microsystems

SUMMARYThe city of Bad Soden am Taunus is a smaller town nearby Frankfurt/Main in Germany with about 21.500 inhabitants. The city inhabits a full town adminis-tration structured with departments like city hall, fi re department, water works, building yard and building authority. The city services of Bad Soden am Taunus offered to its inhabitants heavily depend on IT for the execution of administra-tive tasks. The administration of the city operates a central server infrastructure for all the application necessary and ca. 100 desktops for the employees.

With the deployment of Sun Servers based on the dual-core AMD CPU together with a massive consolidation and virtualisation of services using VMware the city of Bad Soden am Taunus achieves huge savings in energy used for powering the IT equipment. Beside the consolidation of servers the IT department of the city introduced concepts of high availability, i.e. a redundant server infrastruc-ture – strongly recommended for highly consolidated environments – and re-placed 80 desktop computers by a Virtual Desktop Infrastructure with Sun Ult-ra Thin Clients SunRay. The consolidation project at the city of Bad Soden am Taunus achieved energy saving of more than 60% for the consolidated municipal server environment. The additional power consumption introduced by the implementation of high availability (the second server) was more than compensated with the deploy-ment of Virtual Desktop Infrastructure and Ultra Thin Clients, leading to overall savings in energy consumption of more than 61%.

Fig.1.1 Schematic diagram of Virtual Desktop Infrastructure

1

Ritch Client

Thin Client Connection Broker Virtual Desktops in the Vi3 Environment

3

Windows servers and the deployment of Ultra Thin clients was found to fi t excellently to pro-vide a best practice case for energy effi ciency approaches for the E-Server project supported by the European Union.

ENERGY EFFICIENCY PROJECTThe best practice case was conducted within the IT renewal project for the consolidation of services to a smaller number of server systems with a virtualisation of Windows operating sys-tem instances onto the VMware.In addition the replacement of workplace com-puters with Ultra Thin Clients was included into the best practice case to illustrate the energy savings possible through a deployment of the Virtual Desktop Infrastructure.

The Virtual Desktop Infrastructure was imple-mented by using Windows Terminal Services and Sun Secure Global Desktop Software, both running as virtual instances on VMware.

Description of systemsThe measurement was performed during the migration of the old to the new infrastructure, from mid of April to end of June 2008. The main goal was a comparison between the aged ser-vers that were about to be taken out of produc-

tion with more recent server products just being deployed at the customer. All available systems were under observation:• 16 old x86 based systems purchased in 2003

(mainly “white space” rack servers based on Asus boards and Adaptec controllers)

• 2 new systems already employing modern multi-core, multi-process CPU technology from AMD (dual-core AMD 8218, 2,6 GHz)

• 2 Storage systems consolidating all the ser-ver data disks in a central SAN architecture

All systems are providing application services for city administration to end users or do serve as alarm systems for critical facilities like fi re department or water works.Furthermore we did inspect the power con-sumption of the workplace computers planned for replacement by SunRay Ultra Thin Clients and of the SunRays itself:• 80 old PCs purchased in 2003 (for 5 PCs mea-

surements have been explicitly conducted),• 80 SunRay Ultra Thin Clients (for 2 Sun-

Rays measurements have been explicitly conducted).

RUNTIME ENVIRONMENTAll old server systems at Bad Soden run Win-dows Operating environments, either Windows 2003R2 or Windows NT4. Since the applica-

tions from the old NT servers have been migra-ted to the W2003R2 servers in the fi rst step, only W2003R2 remains as the actual operating system. Since this migration happened before the actual power measurements the power con-sumption of old Windows NT servers was mea-sured in idle mode without any application load. The old desktops were equipped with Win-dows 2000.For the new server environment VMware ESX 3.0.2 is used as software layer for virtualisation. VMware itself is set up as a cluster to offer high availability for the new infrastructure. All the old W2003R2 instances have been migrated 1:1 with VMware tools to become VMware guests. At the moment 32 different Windows instances are running as guests in the VMware environment. Power management features – as far as they did exist in hardware and software - were not activated on the old nor are activated on the new server environment. The reason for the new servers not to activate power management is the high density of consolidation (32 guest ope-rating systems on each server) and the strong requirements regarding the alarm applications running as guests on VMware.

MUNICIPAL SERVICESThe city administration uses 54 different IT ap-plications. This very nicely refl ects a typical situ-ation in smaller administration entities or smal-ler or medium enterprises: lot of specialized or home grown applications and low usage of standard applications.

Case Study 1City of Bad Soden

Fig.1.2 Old infrastruc-ture Bad Soden am Taunus

Fig.1.3 New infrastructure in Bad Soden, two servers and centralized storage in two separated data centres

4

With the standard procedure of a 1:1 migration of the old environment into a virtualised infra-structure based on VMware we fi nd absolutely similar environments (old and new) for a com-parison of energy consumption before and after the consolidation.We conducted measurements of power con-sumption in parallel with measurements of the CPU load. As a result we found that the impact of CPU load on power consumption is not signifi cant in either environments, the old and the new one. Here one of the reasons for the missing impact may be, that Power Manage-ment is not used in both environments.

Method for the assessment of energy con-sumption and workloads

MEASURED WORKLOADThe workloads of the old systems were logged with the Windows performance monitor, where statistical data with a sampling rate of one se-cond was collected. Diagrams of the measurements for two old servers can be found in Fig.1.4 and Fig.1.5. The time period for the diagrams is 8:30 until 16:30, the main time of activities for a city administration.

With the exception of the Exchange server (see Fig.1.4) we found a situation where the average load for each server and application is very low while a lot of very short peaks with high performance demand occur. The DC Easy server (load profi le in Fig.1.5) may serve as an example.

This environment is best fi tted for a dense con-solidation of applications onto just few servers. Virtualisation and consolidation rises the average load and gives enough headroom for application peaks.The result of virtualisation and consolidation on the Sun X4600 servers regarding the system load is shown in Fig.1.6. The graph for the ESX-01 server shows the load for June since the migration to the new systems onto ESX-01 was fi nalized on June, 13th.

MEASUREMENT OF POWER CONSUMPTIONThe power measurement was done on the fused outlet line with a total amount of twenty power meters, where always one of the meters was connected to each power supply separately, cf. Fig.1.7. The power meters used were the Conrad Energy Logger 3500 devices with the following specifi cations:Power consumption measurement: 1 Wh - 9999 kWhDimensions (W x H x D): 135 x 82 x70 mmOperating voltage: 230 V/ACEffective power range: 1.5 - 3000 WResolution: 0.1 W; Frequency: 50 HzAccuracy Class: ± 1 % + 1 WConsumption: 1.5 W

Sampling rate: 1 minuteRecording period: max 6 monthsSD card slot for data export

The measurement lasted from April 17th to June 26th, where old systems and the new target systems were measured during diffe-rent periods. The fi nal measurement of the new virtualised servers was done in the se-cond half of June when the migration of all applications was fi nished. The samples were taken every minute and saved on a SD card inside the individual power meters, which was later copied to an external computer for further processing.

1

Fig.1.4Friday 18/04/2008;

Average load: 39,99%

Fig1.5Monday 21/04/2008; Average load: 1,58%

Fig.1.6ESX-01 CPU Load

June, Average load: 10,69%

CPU Load Exchange Server processor load (%)

CPU Load DCEasy Server processor load (%)

CPU Load June processor load (%)

Time

Fig.1.7 Setup for power measurements

Results and evaluation of energy savings

ENERGY USAGE IN IDLE STATEThe table below gives examples of the measu-red power usage for some of the systems in idle state, i.e. after booting with Windows or ESX and no application load. Results for all servcers are available in the fi nal report on the E-Server web site www.effi cient-servers.eu.

ENERGY CONSUPTION DURING THE PERIOD OF MEASUREMENTDuring more than 2 months the old server and PC infrastructure was measured to collect their power consumption data. During the last two weeks in June the fi nal data for new inf-rastructure and the SunRay Thin Client was gathered.

The table below summarizes the energy con-sumption for all the old system.

For few of the servers the data was extrapola-ted, especially due to the number of desktop computers we only measured power for fi ve different workplaces. Here we have chosen PCs which are switched off after the day work and on weekends (which is a typical behaviour of the city employees) as well as systems which run during the night for special reasons. The measured power consumption of these desk-tops was then extrapolated to all 80 PCs. For that the weighting of the different desktop computers (developed together with the IT de-partment of the city) is as follows:• 2x PC #1: the PC for IT administration is run-

ning all the time for fast access in emergency cases,

• 19x PC #3: PC with usage time longer than 8 hours, which represents the the offices with public traffic, typically open to the public for a long period of the day and even Satur-days,

5


• 9x PC #2: PC running more than 8 hours per day, which represents a smaller number of employees,

• 10x PC #4: this PC was not switched on eve-ry day, which represents a smaller number of employees,

• 40x PC #5: PC switched on on 5 days (swit-ched off overnight), which represents the typical user.

For the calculation of the yearly power con-sumption of the desktops the vacation period of 6 weeks in Germany was taken into ac-count. Since the employees of the city typi-cally switch off their PC in the evening or before a weekend the measurements for over one week lead to a realistic value for power consumption. One result we found was that the switched-off PC does have a consumption of ~6 Watt just from the power supply being connected to the power line, more than the consumption of the SunRay Ultra Thin Client of 4 Watt.For the new infrastructure the measured data is consolidated in the table below.

Server Server-Type Power SupplyRating

MeasuredPower Con-sumption(Idle-Mode)

DC-Easy Intel Pentium 4,2,4 Ghz, 2 GB

1x 300W ca. 119W

DC-Neuen-hain

Intel Pentium 4, 2,4 Ghz, 1 GB

1x 300W ca. 392W

Ingrada-Web

HP DL380 G3, Intel Xeon 3,06 Ghz, 4 GB

1x 499W ca. 392W

Old NT #1 - #7, WNT4

Intel Pentium III,Fam 6 Mod 7Step 3, max 600 MHz, 384 MB, 2x2GB

unbek. ca. 61W

PC #1 LG, AMD Athlon XP2400+, 256 MB, 40 GB, W2000

Unbek. ca. 94W

ESX-01 Sun X4600, 4x AMD DC, 2,6 GHz, 32 GB, 2x72GB, VMware ESX 3.0.2

4x 850W ca. 701W

Storage #1 Sun StorEdge 6140, 2x RAID-Controller, 5x 300 GB FC-AL

2x 460W ca. 242W

SunRay Sun SunRay 2, RMI Alchemy Au1550 security network CPU

30WexternesAC/DCNetzteil

ca. 3,8W

Old Infrastructure: 16 Windows servers, 80 workplace PC‘s

Server/PC Measured Power Con-sumption for 1 Week (Wh)

Calculated Power Con-sumption for 1 Year (kWh)

DC-Easy 20,404.83 1,063.97

DC-Finanz +DC-Exchange

97,385.16 5,077.94

DC-Neuhain + Ingradaweb

93,030.13 4,850.86

DC-Paulinen 69,128.98 3,604.58

Wasserwerk1 10,871.74 566.88

Wasserwerk2 26,103.92 1,361.13

Old NT #1 –Old NT #7

71,425.48 3,724.33

DC-Bauhof 20,404.83 1,063.97

Summe Server 408,755.08 21,313.66

PC #1 – IT 18,967.51 875.22

PC #2 – fi re dep. 9,537.41 440.08

PC #3 – fi re dep. 12,396.55 572.01

PC #4 – city hall 4,900.55 226.13

PC #5 – city hall 5,787.43 267.05

Sum 80 PCs (weighted sum)

639,808.86 29,522.61

Sum 1,048,563.94 50,836.27

New Infrastructure: 2 Sun X4600 Server (4 DC AMD CPU), 80 SunRay Ultar Thin Client

Server/Sunray Measured Power Con-sumption for 1 Week (Wh)

Calculated Power Con-sumption for 1 Year (kWh

X4600#1 PS1 29.676,51 1.547,42

X4600#1 PS2 30.117,20 1.570,40

X4600#1 PS3 31.567,78 1.646,03

X4600#1 PS4 28.850,82 1.504,36

X4600#2 PS1 28.346,70 1.478,08

X4600#2 PS2 32.384,90 1.688,64

X4600#2 PS3 32.384,90 1.688,64

X4600#2 PS4 29.387,56 1.532,35

2x SE6140 FC RAID PS1

35.163,19 1.833,51

2x SE6140 FC RAID PS2

46.879,77 2.444,45

Sum Server + Storage

324.759,33 16.933,88

2x Sun Ray 668,91 34,88

80 Sun Rays 53.512,67 2.790,30

Sum 378272,00 19.724,18

6

For few of the values given above an extrapolati-on of the data was necessary, especially the mea-surements of the SunRay Ultra Thin Clients were done just for two of that devices. Here the mea-surements have been conducted for 6 days and the extrapolation to a yearly consumption took all 365 days of the year into account (no reduction because of vacation or public holidays).

POWER DRAW AND SYSTEM LOADThe power draw and the system load for the old server environment do show a correlation but the difference between low and maximal values is in the range of just 15%. To illustrate this the graphs of CPU load and power draw for the server DC-Paulinen are shown in Fig.1.8. The same observation we do fi nd for the new infrastructure of X4600 servers. The server

load and the power consumption of one of the redundant power supplies of the server ESX-02 is depicted in Fig.1.9. One easily recognizes a correlation but with very low difference bet-ween minimal and maximal values of power consumption.

COMPARISON OF OLD AND NEW ENVIRONMENTDuring the consolidation project at the city of Bad Soden the old non-redundant infrastructure was replaced by a highly redundant infrastruc-ture: two small datacenters, high availability by the means of VMware cluster, servers with red-undant power supplies etc.In a fi rst step the analysis of power savings for this best practice case will consider just a 1:1 replacement of the old infrastructure, omitting

the second server and storage components used for redundancy.Secondly, an overall calculation of energy sa-vings will be given, taking the redundant server and storage and the replacement of the desk-top computers into account.

ENERGY SAVINGS FOR A DIRECT 1:1 REPLACEMENT OF THE OLD INFRASTRUCTURELooking at a 1:1 replacement of the old infra-structure by new servers and storage we cal-culate with the power consumption data for just one server and one storage box. This ap-proach will give us a direct result for the ener-gy savings that can be achieved by consolida-ting a smaller IT environment like that of the City of Bad Soden.

In a 1:1 comparison a hardware renewal com-bined with consolidation and virtualisation of all applications results in energy savings of 12.906 kWh per year, or an energy consumpti-on of just 39,45% compared to the power con-sumption before. This calculates to a cost saving of 4.456,60 EUR in a period of 3 years at an average price per kWh of 11,51 Eurocent .

ENERGY SAVINGS FOR THE CONSOLIDATION AND VIRTUAL DESKTOP INFRASTRUCTUREWith the consolidation to one single server the IT based municipal work may be exposed to a com-plete stop when this server (or power, disk, ...)

1

Fig.1.8DC-Paulinen: CPU load in % (top) and correlated power consumption of one power supply (bottom)

Server/PC Measured Power Con-sumption for 1 Week (Wh)

Calculated Power Con-sumption for 1 Year (kWh

Old Infrastructure: 16 Windows servers

Sum 408.755,08 21.313,63

New Infrastructure: Sun X6600 server (4DC, AMD CPUs), SE 6140 storage

X4600#1 PS1 29.676,51 1.547,42

X4600#1 PS2 30.117,20 1.570,40

X4600#1 PS3 31.567,78 1.646,03

X4600#1 PS4 28.850,82 1.504,36

SE6140 FC RAID PS1 17.581,80 916,76

SE6140 FC RAID PS2 23.439,89 1.222,23

Sum 161.233,79 8.407,16

Fig.1.9ESX-02: CPU load in % (top) and correlated po-wer consumption of one power supply (bottom)

DCPaulinen CPU Load processor load (%)

Power Draw DCPaulinen Actual Consumption/W

experiences a failure. When consolidating to few or just one server precautions must be taken for continuous IT services for the municipal emplo-yees in case of a failure.The new IT architecture at the city of Bad Soden am Taunus introduced the concept of redun-dancy and high availability to the IT. The server and storage used for production is duplicated in a second separated server room in an other building, providing the necessary quality to the IT services being deployed in the city.The old – and in terms of administration highly complex – PC infrastructure at the desk was replaced by an Virtual Desktop Infrastructure with Ultra Thin Clients.In an overall view – taking the complete red-undant server environment and the replace-ment of desktop computers by a Virtual desk-top Infrastructure into account – the city of Bad Soden achieved the following savings in energy consumption:

One of the interesting results of the measure-ment of the old infrastructure is the fact that the power consumption of the workplace clients is 40% higher compared to that of the installed servers: 29.523 kWh for the desktops versus 21.314 kWh for the servers per year. Despite of introduction of modern high availa-bility concepts (i.e. twice the IT power neces sary to fulfi ll the normal requirements) the city of Bad Soden was able to reduce the energy con-sumption of their IT environment by 31.112 kWh per year or 61,2%, achieving a reduction of nearly 11.000 EUR in energy costs in a three year period (at an average cost of 11,51 Euro-cent per kWh in Germany).In addition to the direct fi nancial savings the

new architecture leads to much more advanta-ges for the municipal IT:• lower complexity of the workplace IT,• lower administration efforts through centra-

lized administration,• faster deployment of new office workplaces,• faster deployment of new applications,• flexibility and new mobility on the desk,• high availability.

The IT renewal project at the city of Bad Soden am TaunusBeside the consolidation of servers and the in-troduction of the Virtual Desktop Infrastructure the IT renewal project at the city of Bad Soden am Taunus consisted of the modernization of the network and the consolidation of printers. While these topics are not a part of the E-Serv-er project a short description of achievements within this sub-project will be given.

MODERNIZATION OF THE NETWORKThe old network of the city was build with lot of 1 Mbit/s connections between the different IT locations of the city. During the IT renewal

project the old network was replaced by a new infrastructure based on point-to-point radio links with at least 100 Mbit/s.The new network infrastructure was the neces-sary basis to allow the consolidation and cen-tralization of the old servers and the deploy-ment of Ultra Thin clients. And in addition in terms of costs it has a Return-of-Investment period of 24 months (information by Stadt Bad Soden am Taunus).

CONSOLIDATION OF PRINTERSThe fi nal project realized by the IT of Bad Soden am Taunus was a consolidation of printers. From a printer on every desk (more than 100 old prin-ters) the new printer infrastructure consists of just 60 modern network enabled printers. Based on advantages in printer technology regarding stand-by features and very low stand-by power usage (typically < 1W) here additional savings in power consumption and energy costs have been realized.

7

CONCLUSIONS AND LESSONS LEARNEDThe consolidation and virtualisation project at the city of Bad Soden am Tau-nus did show the predicted and expected energy savings. The project illustra-tes nicely two different but complementary ways to achieve energy effi ciency in an IT environment:• consolidation and virtualisation of widely spread server infrastructures,• virtualisation of desktop infrastructures.

First the project proves the proposition that consolidation and virtualisation are main topics to be considered for huge energy savings. Secondly the replacement of deskside servers by Ultra Thin Clients reveals the huge potentials for energy savings on the desktops of enterprises and public authorities.


Energy Consumption1 year (kWh)

% Energy Consumption

vs. 0ld

Server old 21.314

End user PC´s 29.523

Sum old Infrastructure

50.837

Servers+Storage new 16.934

SunRay‘s 2.790

Sum new Infrastructure

19.724 38,809%

8


THE IT OVERHAUL PROJECT AT STRATO AGWith more than one million customers in six European countries STRATO is Europe’s second largest Web host. STRATO represents over 10 years of experience and expertise in the Web hosting market. In 1998 STRATO launched its campaign www.wunschname.de (www.desi-redname.de) and turned the .de-domain into the most successful top level domain ever. It is one of the company’s declared goals to enable its customers to run their own website or online shop themselves. The company runs over 4 mil-lion domains on more than 35,000 servers in two state-of-the-art data centres. A smart inf-rastructure allows for a high energy effi ciency and optimal performance of the hardware and software. STRATO offers customers different services: home pages, online-shop as well as the lease of a server.

As many other companies the STRATO AG in the last years encountered a strange situation in their data centres: while there was still room for new hardware in the data centre the power and cooling infrastructure was not able to provide the necessary resources for further growth of hardware and services for customers. The data centre facilities built some years ago were plan-ned for average energy densities of server systems at that time and are only partially able to fully support the high energy densities you get today with modern computer hardware.Instead of building a new data centre or trying to rebuild the old one while the computer systems are in production the STRATO AG decided for a general overhaul of their computer platforms. The aim was to reduce the energy usage of the com-puting platform and the facilities infrastructure by different means to save the huge costs for a new data centre (and of course for energy itself) and to be able to grow services within the existing data centre buildings:• install new and energy efficient server

hardware,• optimize the storage infrastructure,

STRATO AG96% energy savings through effi cient and high

performance servers at Europe’s second largest web host

Bernhard Przywara, Sun MicrosystemsMartin Müller, Sun MicrosystemsRene Wienholtz, STRATO AGOliver Fuckner, STRATO AG

SUMMARYWith more than one million customers in six European countries STRATO is Europe’s second largest Web host. The company runs over 4 million domains on more than 35,000 servers in two state-of-the-art data centres. A smart in-frastructure allows for a high energy effi ciency and optimal performance of the hardware and software. STRATO offers customers different services: home pages, online-shop as well as the lease of a server.

This article documents the measurements of energy usage conducted at the STRATO data centres within the Shared Services environment. With the conso-lidation of web services to a new and energy effi cient T2000 server the web host STRATO can save more than 90% of the energy necessary to power the Shared Services for http traffi c.While the original project of consolidation of more than 100 servers in the Shared Services environment was conducted with the T2000 server, STRATO now continuously deploys the newest Chip Multithreading servers (T5220) from Sun based on the UltraSPARC T2 processor. This new server allows to lower the already reduced energy usage in the Shared Services environment of STRATO by additional more than 50%. Overall the report shows the way for more than 96% energy savings for the Shared Services infrastructure at STRATO.

2

Fig.2.1: Shared Services Architecture at STRATO AG

INTERNET

Routing/Switching Level Anti-DoS Filters

LoadBalancingClusters

LoadBalancingClusters

SMTPinFarms

Web Farm #1

SMTPoutFarms

POP3IMAPOther

Farms

Web Farm #X

Tape-Archive

NAS Head Systems NAS Head Systems

Modular DiscSubsystem

Disc-basedBackup-System

9

• optimize applications to better use modern multi-core and multi-thread CPU archi-tectures,

• optimize the air flow and cooling in the data centre.

In a fi nal step STRATO AG further decided to go for regenerative energy sources to run their data centres.The complete IT overhaul project led to overall energy savings of around 30% up to date as communicated by STRATO. The savings of energy usage and the optimization of the data centre cooling made it obsolete to build a new data centre or to rebuild the old one.

STRATO AG AND THE E-SERVER PROJECTWhile the approach to energy effi ciency at STRATO included all energy using products and facilities in the data centre the main point to reduce energy usage was surely the renewal of server hardware which is the source of up-coming problems with energy and cooling.This process of hardware renewal was an excel-lent opportunity for both, the E-Server project and STRATO AG, to set-up energy measure-ments in the old and new environments to showcase energy savings in a real production environment. The goal of the project was to• document real energy savings with modern

energy efficient servers,• gain experience in energy efficiency beyond

marketing and product sheets,• provide a best practice case (and motivation

and example) for the use of energy efficient hardware.

To keep the efforts affordable we chose the Shared Services environment at STRATO as a target for the best practice case of energy effi -ciency of servers.With Shared Services STRATO do offer a whole bunch of web services to customers (home pa-ges, domains, mail, databases, shops,...) while using a shared server environment to host the services. Sharing the server infrastructure allows for optimal usage of the resources, e.g. deploy-ment of load-balancer enabled horizontal ser-ver architectures which lead to a high load on

the servers using the available resources in an optimal way.

PLATFORM ARCHITECTURE SHARED SERVICES AT STRATO AGThe server renewal project of the Shared Servi-ces environment at STRATO replaced more than 100 old systems by 28 Sun T2000 and 14 Sun X4200 systems, serving approximately 3.5 mil-lion customers with web services. The Sun T2000 server used for the renewal is part of this study on energy effi ciency as well as the Sun T5220 server introduced later into the Shared Services environment.

Description of systemsThe measurement was performed on the production environment of the web farm of STRATO. The main purpose was a comparison between aged servers that were about to be taken out of production with more recent server products just being deployed at the service provider‘s web farm. Three generations of systems were under observation:• Four old systems using old technology, based

on single core CPUs (UltraSPARC IIIi and Ul-traSPARC III), without any hardware support for multi-threading (SunFire V240, V440, F280R, V490)

• One newer system already employing mo-dern multi-core, multi-threading CPU tech-nology from Sun Microsystems (UltraSPARC T1 based SPARC Enterprise T2000),

• One system using the latest generation multi-core, multi-process CPU technology from Sun Microsystems (UltraSPARC T2 based SPARC Enterprise T5220).

All systems are providing web content to end users. Incoming user requests are fi rst processed by a load-balancer that has a predefi ned set of “weights” to decide on where to send a particular request. These weights basically de-scribe the expected relative performance of one system compared to another.Fig.2.2 schematically describes the weights chosen for the systems under test:The weights were based on the wide experience of many benchmarks done with these systems. Usually the weights in this web farm are care-fully tuned to achieve an average load on each individual system of 60% to provide additional headroom for unexpected peaks in user traffi c. We did not do this tuning to guarantee a con-stant load distribution during the measurement period and thus had to choose a more conser-vative distribution.

Case Study 2STRATO AG

Fig.2.2: Schematic overview over the server weights

T2000 T5220

Load balancer

USERS ON THE INTERNET

Aged systems

3 4 2 4 12 24

Tab.2.1

10

Method for the assessment of energy consumption and workloads

MEASUREMENT OF POWER CONSUMPTIONThe power measurement was done on the fused outlet line with a total of twelve power meters, a pair of for each system under test. The power meters used were the Conrad Energy Logger 3500 devices with the following specifi -cations:Power consumption measurement: 1 Wh - 9999 kWhDimensions (W x H x D): 135 x 82 x70 mmOperating voltage: 230 V/ACEffective power range: 1.5 - 3000 WResolution: 0.1 W; Frequency: 50 HzAccuracy Class: ± 1 % + 1 WConsumption: 1.5 WSampling rate: 1 minuteRecording period: max. 6 monthsSD card slot for data export

The measurement lasted from March 18th to April 1st 2008, a total of nearly two weeks while the systems were under load. The samples were taken every minute and saved on a SD card inside the individual power meters, which

was later copied to an external computer for further processing.

RUNTIME ENVIRONMENTAll systems were installed with an identical soft-ware stack based on the same Solaris 10 opera-ting system release and a highly customized web server software based on Apache httpd. All ser-vers had access to the same http content which is served via NFS from a central NAS fi ler instal-lation to all systems. The load balancer weight-ing scheme was kept constant during the whole test although we observed that especially one system could have handled a much higher load without being overloaded or even exceeding the 60% average load limit.Since power management features do not exist in hardware up to now for the servers under observation, no power management was activa-ted or used during the measurements.

SHARED SERVICES WORKLOADAll systems served http requests to end user pa-ges hosted at STRATO. The customers of STRATO pay for hosting their content on shared equip-ment at the ISP‘s site, the load is generated by HTTP GET requests to these web pages. These requests can be static or dynamic. Static requests are pure HTML requests with possibly embedded static fi les like pictures. Dy-

namic requests summarise all the requests which generates web content to be delivered through the invoking of scripts (CGI) which itself can involve different software like PHP, Perl, Python, SQL etc. Here the customers within the Shared Services environment at STRATO defi ne by themselves which kind of scripts and software they use for dynamic content. The databases for SQL statements are running on different servers in the Shared Services environment.No customer has a dedicated web server within this environment, the requests are fi rst pro-cessed by a load balancer which decides which server has to answer the request based on the already mentioned table of weights. (A system with a weight of 12 on the average gets 3 times more requests compared to a system with a weight of 4.)

WORKLOAD PROFILE OVER TIMEThe workload was measured by the averaged number of HTTP requests per second, the averaging being done over one minute. These numbers were provided by the ISP derived from the access logs of the web server software. The maximum number of requests per second is bound by the CPU performance of the underlying system. The set-up of this server farm allows for managing the maximum load a single system

2

System Power Measured Power Supply Consumption Rating (Idle-Mode)

V240 2x 546 W ~260 W

V440 2x 650 W ~415 W

F280R 2x 810 W ~392 W

V480 2x 1440 W ~803 W

T2000 2x 450 W ~222 W

T5220 2x 750 W ~245 W

Fig.2.3Blue: CPU Load; Yellow and Red: Number ofdynamic and static Apache hits

Fig.2.4: Blue: Power Usage; Yellow and Red: Number of dynamic and static Apache hits

V240: CPU and Web Server Load

CPU

load

Actu

al p

ower

con

sum

ptio

n (W

)

Apac

he h

itsAp

ache

hits

has to handle by means of the load balancing parametrization.

Three parameters were recorded for every single system during the test period:• CPU utilization average over a two minute

interval, done using a NAGIOS framework already in place at the web host. NAGIOS is an open source computer system and net-work monitoring software, able to monitor processor load and lot of other resources (see http://www.nagios.org).

• Number of requests per second averaged over one minute, split up in requests for static and dynamic pages. These numbers were derived by the ISP from the access logs of the web server software.

• Power meters recorded actual and apparent power consumption per minute and power line (each server system has a redundant connection to two power grids).


ENERGY USAGE IN IDLE STATETab.2.1 gives the measured power usage for all systems in idle state, i.e. after booting with the operating system running and no application load.

MEASURED WORKLOAD AND POWERThe analysis of the data shows a clear and expected correlation between the hits served by the Apache web server and the CPU load, depicted in Fig.2.3 for the old server V240 as an example.The measured workload has a strong correla-tion to the time of day but a low correlation to the day of the week. This is a well known behaviour of the internet users, generating a high load during the day and a comparably low utilization during the night time. Except for special circumstances (e.g. sports events that have their web sites hosted in the ISP‘s infrastructure) the utilization is quite similar from day to day. When looking on the power consumption (see Fig.2.4) one fi nds that the power draw and the system load show a surprisingly low correlati-on. Although the number of static hits per second within 24 hours changes from e.g. 1.000 to 8.000 the power draw only shows a little variation. Here again the V240 serves as an example. The same applies to the other aged servers V440, F280 and V480.

The observations for the new servers T2000 and T5220 are similar but here a slight correlation

11


between power consumption and system load is easily seen. Taking the T5220 as an example (see Fig.2.5) we again fi nd the expected corre-lation between Apache hits and CPU load.

But now when looking for dependencies bet-ween Apache hits (strongly correlated to CPU load) and power consumption (see Fig.2.6) one easily fi nds the day/night wave known from the Apache load mirrored in the power draw from the servers. Nevertheless the qualitative diffe-rence in power consumption is just about 15% or 30W for the T5220 when comparing times of low and high application load.

The same applies for the older T2000 server. Fig.2.7 combines for the T2000 the Apache load (static and dynamic, visible as a wave distinguishing the day with high load from the night with low load) and the measure-ments of the power consumption of both of the redundant power supplies (recognisable as the red and green slightly waving lines).

Load usr+sys (%)Static hits/2 minDynamic hits/2 min

Load usr+sys (%)Static hits/2 minDynamic hits/2 min

Fig.2.5: Blue: CPU Load; Yellow and Red: Number of dynamic and static Apache hits

Fig.2.6: Blue: Power Usage; Yellow and Red: Number of dynamic and static Apache hits

Fig.2.7: T2000 server: Dependency of power usage (red/green) on Apache hits (blue/magenta)

Line ALine BStaticDynamic

T5220: CPU and Web Server Load

SPARC Enterprise T2000

CPU

load

Actu

al p

ower

con

sum

ptio

n (W

)

Appa

rent

pow

er (W

)

Apac

he h

itsAp

ache

hits

Time, date

page

hits

/sec

12

ENERGY CONSUMPTION DURING THE PERIOD OF MEASUREMENTTab.2.2 below summarizes the energy con-sumption and the web traffi c throughput for all the systems being measured. The systems have been operated in parallel in the produc-tion environment of Shared Services at STRATO during the time period of March, 18th until April, 1st 2008. The incoming user requests (http) were processed by a load-balancer with a predefi ned set of “weights” to be distributed to the particular systems. So the power con-sumption was metered for all these servers in parallel.The results show that the new systems T2000 and T5220 consume much less energy while handling much higher application loads. The T2000 uses just 12,5% of the energy the four old servers consume while providing twice the application throughput performance (although at higher average system load).

The T5220 uses just 13,6% of the energy the four old servers consume while providing more than three times the application throughput at a comparable average system load.

NORMALIZATION AND EVALUATIONBecause of the differences in application throughput performance of the old and new

servers a slight normalization of the acquired data is appropriate. The normalization is to ad-just the throughput performance to get compa-rable throughput and energy consumptions of the old and new environment. Such an ap-proach shows the achievable savings in a con-solidation project with T2000 and T5220 as target systems.

Since the measurements shows that the power draw is nearly independent of the number of served requests per seconds it is a valid approach to normalize the CPU load towards an average CPU load of 50%. Assuming a linear depen-dency of http traffi c to CPU load (as show by the measurements of Apache hits and CPU load) this leads to the following power consumption and e.g. static throughput values for the measure-ment period of 2 weeks.

Energy Savings with Sun SPARC Enterprise T2000A comparison of the throughput • 2.755.815 MB static for the old environment

and• 3.459.889 MB static for the T2000,implies

that one would need in average 5 of the older systems to obtain the throughput and perfor-mance of the T2000 server:

• On average an old server is capable to handle

2.755.815 MB / 4 = 688.954 MB of through-put at a CPU load of 50%.

• To achieve a throughput of 3.459.889 MB one would need 3.459.889 / 688.954 = 5,02 old servers on average.

A last extrapolation of the energy consumption to a period of a whole year leads to the follo-wing result: With the consolidation of web ser-vices to a new and energy effi cient T2000 server the web host STRATO can save more than 90% of the energy necessary to power the Shared Services for http traffi c.These measurements do refl ect the real conso-lidation project the STRATO AG undertook in 2006 and 2007: here over 100 older systems have been replaced by 28 Sun T2000 and 14 Sun X4200 (x64) systems in the Shared Services environment.

ENERGY SAVINGS WITH SUN SPARC ENTERPRISE T5220Again a comparison of the throughput implies that one would need 14 of the old (average) systems to obtain the throughput and perfor-mance of the T5220 server. The calculation is done completely analogous to that above. A last extrapolation of the energy consumption to a period of a whole year leads to the follo-wing result:

2

System CPU-load (avg)

Energy A (kWh)

Energy B (kWh)

Throughput Static (MB)

V240 50,00 42,83 49,93 564.168,43

V440 50,00 61,27 72,54 959.696,26

E280R 50,00 65,23 68,97 423.063,96

V480 50,00 122,54 153,14 808.887,08

Total older systems

291,85 344,58 2.755.815,73

T2000 50,00 38,15 41,39 3.459.889,26

T5220 50,00 42,97 43,53 9.721.055,38

Lime period of measurement: 03/18/08, 12am – 04/01/08, 12am

System CPU-load (avg)

Energy A (kWh)

Energy B (kWh)

Static hits (1000/s)

Throughput Static (MB)

Dynamic hits (1000/s)

Throughput dynamic

(MB)V240 24,90 42,83 49,93 10.103,69 280.955,88 2.162,55 30.488,81

V440 29,34 61,27 72,54 20.189,61 563.149,77 4.328,31 60.116,17

E280R 32,96 65,23 68,97 10.067,57 278.883,76 2.143,92 29.402,42

V480 35,01 122,54 153,14 20.083,71 566.382,73 4.295,52 58.749,13

Total older systems

291,85 344,58 60.444,58 1.689.372,14 12.930,30 178.756,93

T2000 49,75 38,15 41,39 122.466,04 3.442.589,82 25.399,50 416.660,64

T5220 29,29 42,97 43,53 201.800,61 5.694.594,24 42.798,67 586.356,78

Tab.2.3: Normalized Apache throughput and power usage

Tab.2.2: A comparison of the throughput

With the consolidation of web services to a new and energy effi cient T5220 server the web host STRATO can save more than 96% of the energy necessary to power the Shared Services for http traffi c.While the original project of consolidation of more than 100 servers in the Shared Services environment was conducted with the T2000 ser-ver, STRATO now continuously deploys the ne-west Chip Multithreading servers (T5xxx) from Sun based on the UltraSPARC T2 processor. When comparing the savings achievable with the new T5220 with the results from the last section (T2000) the deployment of the new server allows to lower the already reduced energy usage in the Shared Services environment of STRATO by addi-tional more than 50%.

FINANCIAL ASPECTS OF ENERGY SAVINGS WITH T5220Assuming an average price per kWh in Germa-ny of 11,51 Eurocent the power savings resulting from replacement of 14 old servers by just one SPARC Enterprise T5220 (i.e 56.103 kWh) are as high as 6.457,45 EUR per year, or 19.372,35 EUR in three years. This nearly equals the list price of the T5220 server of 19.500,00 EUR.

CONCLUSIONS AND LESSONS LEARNED

The measurements of energy at STRATO documented the savings achieved by the consolidation and hardware renewal project in the Shared Services environment. The huge amount of savings is due to two reasons:Generally the refresh of old hard-ware by latest server products with higher CPU and overall performance gives opportunities for consolidati-on of services to less servers. This in turn leads to less energy usage. At the Shared Services more than 100 old server have been replaced with 28 Sun T2000 and 14 Sun X4200 sys-tems in a fi rst step. With the new Sun T5220 being deployed now STRATO is continuing the consolida-tion.For the web services STRATO is de-ploying a server hardware (Sun T2000 and T5220) optimally desig-ned for high-bandwidth multi-threa-ded application loads. Despite of the fact that normally modern servers do use more energy when compared to older products these servers show high energy effi ciency. Both of them use less power than any of the older servers.So, the combination of consolidation with energy effi cient hardware dri-ves huge energy savings in the Shared Services environment of STRATO and provides an excellent best practice case for the use of energy effi cient servers.

13


Tab.2.4: Energy savings with the Sun T2000 Server

Tab.2.5: Energy savings with the Sun T5220 Server

System Energy Rack % Energy consumption Units consumption (kWh/year)

5 old Systems 20.765,94 14 RU 100%

T2000 2.060,60 2 RU 9,92%

System Energy Rack % Energy consumption Units consumption (kWh/year)

14 old Systems 58.344,88 40 RU 100%

T5220 2.241,91,60 2 RU 3,84%

14

Initial situation and targets

In early 2007 BMU has tendered a full service contract for the data centers in Bonn and Berlin and the corresponding client infrastructure for nearly 1100 users. Thus, the BMU pursued the following goals:• Installation and operation of central IT Infra-

structure• High availability over two separated data

centers• Flexible provisioning of compute power• Flexible provisioning of storage• Energy savings of at least 40%

Within this project 3-5 years old single servers have been replaced with IBM System x3850

Bundesumweltministerium Energy-effi cient IT Infrastructure at the Federal Ministry for the

Environment, Nature Conservation and Nuclear Safety in Bonn, Germany

Rudolf Herlitze, Bundesministerium für Umweltschutz, Naturschutz und ReaktorsicherheitSilvio Weeren, IBM Deutschland MBS GmbHOliver Gambero, IBM Deutschland BS GmbHMichael Schepanske, Bechtle AG

SUMMARYThis best practice case demonstrates the energy savings achieved through IT server and storage consolidation in the data center of the “German Ministry for Environment” (BMU) in Bonn.The BMU operates two data centers in Bonn and Berlin with IT services for ne-arly 1100 users at both sites. Within the normal replacement cycle, the central IT (66 servers) has been renewed with focus on high availability and energy effi ciencyThe IT solution architecture was designed to introduce high availability with redundant servers in the data centers and fail over between both sites. The server virtualisation and consolidation was implemented with VMWare ESX 3.5 on IBM System x3850 (Intel Xeon dual core) server and IBM System Storage N5300.

The new infrastructure with extendable server performance and increased storage capacity achieved energy savings of 68% through virtualisation of all application servers and consolidation onto one x3850 host server. In the red-undant operation with a second x3850 server energy savings of still 60% are reached. New server applications with about 50% load have added only 2% in energy con-sumption. The established dynamic infrastructure will al-low the signifi cant growth of the application landscape with little increase in energy con-sumption, hence increasing the effi ciency of new solution. A redesign of the data center layout and opti-misation of cooling promise further signifi cant energy savings.

3

Fig.3.1: High level server architecture for server virtualisation on hosts VMWare ESX 1 and ESX2 and Storage N5300.

15

4-way Intel Xeon servers and IBM System Stora-ge N5300 to provide a fl exible basis for virtua-lisation and consolidation.Main requirements of the BMU have been a high availability of the server applications, ex-tension of the available storage and to increase the fl exibility and capacity to deploy new appli-cations. At the same time, the deployment of the new system should be realised without in-terruption of the business operation. To meet these requirements, the IBM/Bechtle team has implemented a redundant server and storage solution with fail-over across the data centers. The high level server architecture for one loca-tion is shown in Fig.3.1. For the E-server project, the server consolidati-on has been evaluated for the data center in Bonn but the results are representative for the

Berlin data center as well. The fail over with Berlin using a third ESX-host will be implemen-ted later related to the move of the BMU to a new location in Berlin.

System description

The old environment was based on 3-5 year old servers implementing “one server – one applica-tion” for the Microsoft Windows based applica-tions. The utilisation and energy consumption of all application servers in Bonn – see Tab.3.1 - have been measured before the virtualisation.Fig.3.2 shows two of four racks with the old environment and Fig.3.3 shows the new IT inf-rastructure with two half full racks.In addition, four racks with fans in the top cover

Case Study 3Bundesumweltministerium

Server Model CPU & Clock Rate Memory MB Number HDDs

Total Netto GB

Conference Proxy ProLiant DL140 Xeon 2400MHz 1024 1 80

Exchange FE ProLiant DL380 G2 P III 1266MHz 256 3 26

Exchange ProLiant ML370 G4 2 Proz.: Xeon 3400MHz 4096 8 932

Exchange ProLiant ML530 G2 2 Proz.: Xeon 3000MHz 2048 12 689

Offi ce DC ProLiant DL380 G2 P III 1266MHz 512 3 24

Offi ce DC ProLiant DL380 G2 P III 1266MHz 512 3 29

Root DC ProLiant DL380 G2 P III 1266MHz 512 3 29

SMS ProLiant DL380 G2 P III 1266MHz 512 6 116

SPS ProLiant DL380 G2 2 Proz.: P III 1266MHz 2560 6 92

MS SQL ProLiant DL380 G2 2 Proz.: P III 1266MHz 1536 2 28

File Server ProLiant ML370 G4 2 Proz.: Xeon 3400MHz 4096 8 910

SAN-Enclosure 14 216

VMWare1 ProLiant ML530 G2 2 Proz.: Xeon 3000MHz 5120 9 126

VMWare2 ProLiant ML530 G2 2 Proz.: Xeon 3000MHz 5120 12 180

Help Desk ProLiant DL380 P III 1000MHz 512 6 91

Intranet ProLiant DL380 G2 P III 1266MHz 512 6 72

Access Control ProLiant DL360 P III 1266MHz 256 2 28

Summe 30GB 3739

ESX1 Server IBM 3850 2 Xeon 3,16 GHz 32GB

N Series Controler IBM N5300

N Series Storage IBM N4000 FC HDD 300GB 10k RPM 14 3200

N Series Storage IBM N4000 FC HDD 300GB 10k RPM 14 3200

FC Switch IBM SAN32B

Tab.3.1

Fig.3.2: Old-world system with four racks in total.

Fig.3.3: New-world IBM systems with two half full racks

Tab.3.3: CPU utilisation of old-world applica-tion servers measured with Tivoli monitoring.

CPU

%

Tim

e

16

x3550 to run dedicated infrastructure services like fi rewalls and proxies equivalent to the old environment. All OS and application data are stored on IBM System Storage N5300.The existing physical servers have been virtualized using the VMWare converter keeping the OS level and server applications basically unchanged and the confi gurations are adapted to the new storage environment. All virtualized servers are consoli-dated using VMWare ESX 3.5.

Runtime environment

The servers run various fl avours of Microsoft Win-dows and have been converted to VMWare without relevant changes to the operating system. On some servers the RAM allocation in the virtual machines has been changed to achieve higher performance without altering the overall available RAM.

Workload measurements

In Bonn, all 26 application servers are operated basically unchanged. The two exchange servers have been setup with a new image to allow a stepwise migration of the email fi les.To measure the workload and performance of the existing applications (e.g. CPU utilisation and disk

I/O throughput), IBM has installed a Tivoli server for monitoring and data storage on a x3550 server and corresponding software agents in the opera-ting systems to gather the application data and send them to the Tivoli server (time resolution fi ve minutes). The measurements were done over one week from 11. – 18.2.2008 for the old and from 8. – 14.10.2008 for the new environment.Out of the over forty workload parameters which have been measured, only the CPU utili-sation shows a good correlation to energy con-sumption.The tab.3.3 shows the CPU utilisation of some old-world application servers.The load on most servers is rather low on ave-rage. The share point server (SPS) is the most loaded server with utilisation up to 99%. In the new environment the system utilisation has been monitored using the VMWare console to verify, that the environments are comparable and hence the power consumptions can be eva-luated without corrections for changed wor-kloads.

Fig.3.4 shows a 3D view of the CPU utilisa tion of all 21 application servers running on VM-Ware ESX1 for one day with highest loads on the exchange servers and the intranet server at the right.

and KVM switches with CRT 15” monitors have been considered and measured. The other racks with test servers have not been taken into ac-count as not all test servers have been operated all the time.Between October 2007 and February 2008 ten smaller application servers - see Tab.3.2 - have already been converted to VMWare and deplo-yed into production within continuous optimi-sation activities. Therefore the original servers have no longer been available for performance and energy measurements.Table 3.2: shows the application servers which have been virtualised before the measurementsThe new server environment (in Bonn and Ber-lin) consists of 2 servers IBM System x3850 to host all virtual applications in a VMWare ESX environment and eight servers IBM System

3

Tab.3.2: Application servers which have been virtualised before the measurements

Fig.3.4: 3D view of CPU utilisation of all application servers on ESX1 over one day

VMWare1 VMWare2

IT Controlling Certifi cate Server

TeleWorking Logging Server

Terminal Server Software Packaging

System Monitoring Inventory Server

Novatime

Help Line

Server CPU Utilisation %

AverageStandard Deviation

Mini-mum

Maxi-mum

Exchange FE 3,6 4,4 1 67

Exchange1 2,5 2,6 0 29

Exchange2 2,4 3,0 0 27

Offi ce DC1 1,6 2,9 0 52

SMS 5,5 6,1 1 67

SPS 14,6 21,4 0 99

MS SQL 9,8 13,5 0 72

File Server 5,0 6,7 0 53

Intranet 4,3 3,6 0 40

17


Fig.3.5: Front side intelligent PDU (1U height) with six independent ports for power measurements

Tab.3.5: Measured average power with standard deviation (STD), minimum and maximum values.

power supply was switched. After a waiting time of about 30-60 seconds this power supply was powered up again and working normal check green LED if available. Finally the second power was switched. This worked for all servers although generally there is a small risk for a power supply to fail when it has to take the full load.The IBM iPDU is an easy to install, fl exible & scaleable 1U Power Distribution Unit that mo-nitors the power usage at the breaker level. The device has the main outlets, breakers, in-dicator lights and cord connection all on same side of the unit for easy access. Integrated in the network (own IP address) the data can be read out via web interface or IBM Director and Active Energy Manager. The relative failure is +- 5%.

The power measurements of the iPDUs have been taken in intervals of one minute over a whole week and been consolidated in a spread-sheet. They are presented here as average power over the whole week of measurements. The standard deviation of the power measure-ments is typically only a few percent and there-fore average power is more intuitive to use than daily or weekly energy consumption.The table 3.5 shows the weekly averaged power for the old and new environment.

Day CPU % Max CPU % Average

Memory % Max

Memory % Average

STDEVCPU %

STDEVMem %

08.10. 71,1 46,9 73,5 63,8 4,8 1,6

09.10. 83,7 48,7 73,5 63,6 3,1 3,3

10.10. 80,3 53,6 66,3 62,2 2,4 2,8

11.10. 53,4 44,7 64,2 63,7 12,3 0,7

12.10. 53,0 52,8 64,3 64,3 0,3 0,2

13.10. 54,2 54,1 65,6 65,0 0,1 0,9

14.10. 56,7 49,1 65,5 64,6 0,7 1,3

average 83,7 50,0 73,5 63,9

Tab.3.4: Good maximum and average resource utilisation on ESX1 server with standard deviation (STD)

The time dependence is roughly the same indi-cating the equivalence of the workload. As CPU utilisation is a relative measure an exact quan-titative comparison is not possible.The table 3.4 shows the average CPU and RAM utilisations of the ESX1 server for different days.

The results in table 3.4 proof that the current confi guration and sizing fi ts well delivering good resource utilisation of CPU and RAM wi-thout limiting the performance and still enables application server growth.In summary the workloads of the old and new environment are assessed as equal resulting in an ESX1 server daily average of utilisation of about 50% for the CPU and 64% for the RAM.

Energy measurements

In January 2008 the IBM/Bechtle team has ins-talled 12 intelligent Power Distribution Units (iPDUs)[1] with integrated six port power meter and IP communication and has measured the energy consumption of all old and new hard-ware for one week (with time resolution one minute) (Fig.3.5).The servers with redundant power supplies have been switched from the existing passive PDU to the iPDUs during normal operation. First

Server NamePower

(W)Power STD (W)

Power Min (W)

Power Max (W)

Conference Proxy

106 0 105 139

Exchange FE 178 1 170 188Exchange 284 10 268 368

Exchange 362 9 350 431

Offi ce DC 194 1 189 202

Offi ce DC 194 1 189 202

Root DC 179 1 171 188

SMS 232 2 220 251

SPS 219 2 212 243

MS SQL 225 2 218 250

File Server 270 6 149 347

SAN-Enclosure 211 6 202 245

VMWare1 364 3 356 429

IT Controlling in VMWare 1

TeleWorking in VMWare 1

Terminal Server

in VMWare 1

System Monitoring

in VMWare 1

Novatime in VMWare 1

Help Line in VMWare 1

VMWare2 391 16 361 454

Certifi cate Server

in VMWare 2

Logging Server in VMWare 2

Software Packaging

in VMWare 2

Inventory Server

in VMWare 2

Help Desk 150 1 147 179

Intranet 199 2 190 226

Access Control 161 0 160 162

4 Racks Fan & KVM

616 0 612 620

4 Racks 1 Monitor

257 1 252 260

Sum old 4791 66 4520 5384

ESX1 Server 547 15 517 595

N Series Controler

262 6 244 287

N Series Storage

331 4 318 391

N Series Storage

335 8 308 424

FC Switch 48 0 33 66

Sum new 1524 19 903 1168

18

Figure 3.7: shows a screenshot of IBM director 5.2. system management with performance and energy dataFigure 3.6: shows a good correlation of the CPU utilisation with the power consumption over about 6 hours. Please note the adapted scales (power 500-600W). The RAM utilisation is rat-her constant at a level around 64%.Finally, for normal operation systems manage-ment has been implemented applying IBM Systems Director with the Capacity Manager to assess the resource utilisations of the individu-al servers (VMWare images) and the Active Energy Manager to measure the energy con-sumption. This is illustrated in the screenshot fi gure 3.7.

ResultsAll server applications of the old environment have been virtualised and have been running on one VMWare server ESX1 with unchanged work loads to enable an apple to apple compa-rison of the old and new environment.Further, the overall RAM is equivalent (both about 32GB) but the RAM allocation has been optimized in the virtual servers. The ability of the virtual infrastructure to quickly change the ini-tial RAM allocation and to dynamically reallo-cate RAM resources as needed (VMWare baloo-

ning) contributed in higher performance of the applications while only effectively using 64% of the installed RAM.The net storage capacity has been increased by 71% while increasing the availability (double parity RAID5) and the performance signifi cantly. The storage virtualisation enables a dynamic use of the storage capacity avoiding the storage li-mitations experienced in the old environment.The ESX2 server has been installed as redundant server and is running new applications which represent about 50% of the CPU and RAM re-sources of all old applications. ESX1 and ESX2 are confi gured to enable fail over in case of a server hardware failure.ESX3 is not used and is reserved for software development and testing and to take over the Berlin applications in fail over mode after the move to the new location.

ENERGY SAVINGSFig.3.8 shows the stacked average power draw of all application servers of the old and new en-vironment and the resulting total energy saving of 68%. Including the additional power for the redundant server ESX2 in the evaluation still results in 60% energy saving compared to the old infrastructure without redundancy.

Fig.3.9 shows the average power draw (Watt)

stacked for the old servers and compared to the power draw of the new ESX1 server. As the in-itial servers had the system and application data stored locally, we have subtracted the po-wer of the corresponding local disks accounting 10W each to assess the computing part of the consolidation. This is shown in the middle column. Thus, the energy saving of the compu-ting part alone is 85%.The storage contribution to the energy savings is about 6% neglecting the higher redundancy, performance and capacity of the storage solu-tion. If we had scaled the capacity, 46% energy savings could have been reported for the storage part.The assumption of 10W per hard disk is con-firmed by the analysis indicating savings of 67% compared to the real savings of 68% Fig.3.8). Based on the implemented dynamic server and storage infrastructure application and storage growth can be achieved adding only little energy consumption and thus increasing further the overall energy effi ciency of the BMU IT infrastructure.The new applications on server ESX2 can be operated additionally on ESX1 just adding 2% energy consumption for additional 50% work-load.Future application loads (e.g. from desktop virtualisation) and storage requirements can be achieved with easy and quick upgrades of CPUs (2 sockets free), RAM and hard disks with only little additional energy consumption compared to the extended capacities.

3

Fig.3.6: Correlation of CPU utilisation with power

ESX1: CPU & RAM Utilisation vs. Power

Proc

ent

TimeCPU Usage [%]

Memory Usage [%]

Power [W]

19

CONCLUSIONS AND LESSONS LEARNEDThe server virtualisation project in the BMU has implemented a dyna-mic IT infrastructure with extended capacities and additional high avai-labitlity while reducing the energy consumption by 60%. The project results also indicate how further im-provements could be achieved in similar projects:1. A structured measurement of the

energy consumption and utilisa-tion of the IT infrastructure be-fore tendering would enable providers to architect and design more effi cient solutions best tailored to the real needs of the customer and optimised for low resource and energy consump-tion.

2. The implementation of virtual LAN in the BMU would release hardware requirements on the ESX servers and thus enable infrastructure server consolida-tion. If implemented on latest IBM BladeCenter S technology the total energy savings could re-sult in about 85%.


Fig.3.9: Analysis of energy savings from computing with 85% and storage 6%. energy savings could result in about 85%a

Fig.3.8: Figure 8: Stacked average power comparison of old and new application server.

Pow

er [W

att]

68% Power Saving by Consolidation to IBM SystemServer x3850 & Storage N5300

Pow

er [W

att]

67% Power Saving by Consolidation to IBM x3850 w/o all Hdd 10W each

Old

Old

Old-HDD

New

68%

67%85%

New

Fig.3.7: Screenshot of IBM director 5.2. system management with performance and energy data

20


In early 2008 an IT project was launched at Wincor Nixdorf for the replacement of 150 ser-vers. At the same time a consolidation and vir-tualization were implemented in order to ex-ploit further saving potential. This resulted in an energy-effi cient, forward-looking concept for the existing data centre.

The concept includes • the replacement of the existing 150 Blade

servers• consolidation • virtualization

The reasons for this approach were many. In terms of the eco-friendly/sustainable orientati-on of the IT equipment, the potential for savings with this energy-effi cient hardware will lead to greater awareness. Future projects are already in the planning/implementation phase.

This project offers an excellent opportunity to present the Best-Practice-Case for energy efficiency.

„Padergreen“Energy effi ciency by blade consolidation and

virtualisation at Wincor Nixdorf in Paderborn

Hansfried Block, Sabrina Eßer, Juergen Heidegger, Fujitsu Siemens Computers

SUMMARYThe following report describes a Best-Practice-Case, which shows the poten-tial savings that can be achieved through the use of effi cient technology in the server area. The optimisation of energy effi ciency was achieved by the use of effi cient blade server hardware in combination with virtualization.

The best practice case demonstrates the achieved energy savings under real operating conditions in an application environment at a customer site of Wincor Nixdorf in Paderborn. Wincor Nixdorf is one of the world’s leading providers of IT solutions and services for retail banks and retailers and has its headquarters in Paderborn.

The main aim was to reduce costs and complexity and to improve services for the end-user. The use of more effi cient servers in the data centre serves as a basis for saving energy and helping to reduce costs.

By using PRIMERGY servers from Fujitsu Siemens Computers together with consolidation and virtualization through the use of VMWare, Wincor Nixdorf achieves huge savings in energy consumption of up to 75%. This report summarizes the results of the project.

Fujitsu Siemens Computers would like to thank Wincor Nixdorf in Paderborn for its cooperation within the E-Servers (Effi cient-Servers) project.

4

Fig.4.1 The concept for the measurement of

electricity consumption is shown

Power AnalyzerControl System

RS-232

230V

/50H

z

21

Description of systems

HARDWAREThe Best-Practice-Case was conducted within a project of replacement of hardware combined with a consolidation to a smaller number of systems and virtualization of hardware. The main purpose was a comparison between aged blade servers with new innovative blade servers.The detailed confi guration of the old system was a PRIMERGY BX600 S1 Server with BX620 S1, BX 620 S2 and BX 630 Blades. The new virtualized server system contained a PRIMERGY BX 600 S3 Chassis with BX 620 S4 Blades.

WORKLOADThe power consumption of the two confi gura-tions was measured under normal data centre conditions under the real required load of users and service programs. No special loads were started on the participating servers.

WORKLOAD OF THE OLD SYSTEMDifferent characteristic values were recorded at intervals of one minute using the Windows Per-formance Monitor in order to describe the work load on the individual servers.With the exception of one server which runs the

management software for the whole data cen-tre, the utilization was very low, i.e. for the most part much less than 10% with occasional peaks of up to 50%.

WORKLOAD OF THE NEW SYSTEMAs for the old system utilization was recorded with the Windows Performance Monitor in each virtual server after the servers had been mig rated to the virtual environment and this information was stored in log fi les.After virtualization the utilization curves corre-spond to the pattern of the previous measure-ment. There are only minor differences.

Method for the assessment of energy consumption and work loads

The measurements were performed during the migration of the old to the new infrastructure, from mid of June to the end of September 2008.Each measurement (of the original and new system) was carried out over a period of one week or 168 hours. The power and performance values were logged in 1 minute intervals.In order to determine the electricity consumed

by the two confi gurations to be compared, the necessary AC data was measured at the input to the power supply units. In both confi gura-tions the Blade server housings were equipped with four power supply units each, two of which were redundant components. The power supply lines for the power supply units were also de-signed for redundancy, so that full power could be supplied to the Blade servers by the remai-ning components in the event of the failure of individual power supply units or lines.

To ensure that data center operations could continue to be guaranteed with full redundancy during the measurements, the individual feed lines had to be measured separately. A pre-cision power analyzer (LMG450 from ZES Zimmer Electronic Systems) was used, enabling four separate channels to be measured simul-taneously. The individual feed lines were con-nected to the measuring device by means of adapters and then forwarded to the consumer.

Case Study 4„Padergreen“

BX600 Cabinet

Energy (kWh)Costs 0.18 /kWhCosts 0.16 /kWhCosts 0.14 /kWhCosts 0.12 /kWh

BX600 S31 Blade

BX600 S16 Blades

1600

1400

1200

1000

800

600

400

200

0

Blade Server Comparison – Energy Consumption per Year

Cost

s(€)

kWh

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

BX600 S310 Blade

BX600 S160 Blades

0

2000

4000

6000

8000

10000

12000

14000

Blade Server Comparison – Energy Consumption per Year

Cost

s(€)

kWh

0

1000

2000

3000

4000

5000

6000

7000

8000

22


In the following section fi rst of all the energy savings documented by the measurements will be described. Second the potential savings over a fully utilized Blade chassis are discussed (con-solidation of 60 to 10 blades). The project describes the replacement of old hardware with equivalent new hardware. The amount of physical hardware was reduced by using virtualization. The maximum capacity of the new blade chassis is 10 blades. Based on the security policy of the customer, a maximum of 6 virtual machines is build on a single physi-cal hardware, to achieve a suffi cient amount of computing power as spare in case of peak de-mands. The energy and cost savings achieved with the new system amount to 46%.

EXTRAPOLATIONBased on the guidelines of the customer and the maximum number of blades per chassis (10 physical blades for each BX600 S3 possible) the highest possible savings are computed in the second example. For this reason, the scenario is calculated with a total of 60 virtual machines running in only one chassis.

The following diagram compares the power consumption of the two variants per year. Again the energy consumption and the energy costs for the old and the new system are shown and different scenarios for energy prices are used. Fig.4.3

A 75% potential saving on energy costs can be achieved if the capacity of the new blade enclo-sure is used to its full extent consolidating 60 old physical servers on virtual systems running on 10 new server blades.

4

CONCLUSION AND LESSONS LEARNEDThe “Padergreen” project shows high potential for energy savings achieved by the application of effi cient blade technology in combination with virtua-lisation. For the specifi c case energy consumption and energy costs can be reduced by 75%.The savings were achieved by the use of less and more effi cient hardware.

The overall performance of the new hardware (PRIMERGY BX600 S3) enables consolidation on fewer servers. This in turn leads to less energy consumption. The requirements of the customer regarding availability, safety and per-formance were met.The combination of consolidation and virtualization with energy-effi cient blade server hardware offers huge potential for savings and provides an excellent best practise case.

Fig. 4.2Energy and cost savings

achieved by consolidation from 6 blades to 1 blade.

The diagram compares the power consumption of the two variants per year. The

energy consumption in kWh is shown on the Y axis on

the left and the costs on the Y axis on the right.

The energy prices (industry rate) are based on different price scenarios in the range

of € 0.12 – € 0.18 per kWh..

The energy and cost savings achieved with the new

system amount to 46%.

Fig.4.3Energy and cost savings in

the consolidation of 60 physical servers on one

Blade chassis with 10 Blades

Case Study 4„Padergreen“

23

Initial situation and goals

The Austrian Energy Agency is Austrias national research and service organisation for sustaina-ble energy supply and energy use. Central tasks of the agency are the development and imple-mentation of national strategies and measures to support the security of energy supply and an effi cient energy use. Customers are the public service sector as well as private companies. A special fi eld of activity is the collaboration in the development of instruments supporting energy effi ciency at EU-level. The Austrian Energy Agency employs about 100 permanent staff members at its Vienna location.The case in hand refl ects the situation of many Austrian small and medium enterprises and shows the high potential for energy savings in this domain in combination with an optimizati-on of quality of IT services and a signifi cant reduction of administrative costs. As the enterprises mentioned above most often do not make use of externally hosted services – except for web services –, this case will provide relevant information for planning and implemen-tation of future IT replacement/upgrade projects.

The project focuses on replacement of the old ser-vers by an energy effi cient new system, providing additional capacity for future implementation of a new ERP system and many new web/intranet ser-vices during the upcoming years, as well as incre-ased redundancy and easier administration.With the deployment of 2 FSC Primergy RX300S4 servers and an EMC Clarion Storage System together with a massive consolidation and virtualization of services by using VMWa-re ESX Server 3.5i and VMWare Virtual Infra-structure Services, the Austrian Energy Agency achieves huge savings in energy used for po-wering the IT equipment. In addition to the consolidation of servers the IT department introduced concepts of high availability, i.e. a redundant server infrastructure – strongly re-commended for highly consolidated environ-ments. As the new system design also includes a HA Cluster concept, the concentration of ser-vices on one server during periods of low sys-tem loads in combination with powering off the other one during weekends could be thought of and tested.A virtualization of the desktop environment using VMWare Virtual Desktop was also thought of, but was found to be not very effi cient due to the fact, that about 70% of the desktop sys-

tems in use are notebooks often used outside the offi ce environment.

System description

HARDWARE DESCRIPTION BEFORE AND AFTER CONSOLIDATION The existing IT infrastructure of the Austrian Energy Agency refl ects a typical Austrian SME’s environment consisting of a Microsoft Windows Server2003/Active Directory based server sys-tem for internal IT services and a Linux based routing/web services system. The approximately 100 desktop systems include about 70 percent Windows XP Prof. SP2/SP3 based notebooks, mostly used by scientifi c em-ployees and about 30 percent Windows 2000/SP4 based workstations mainly used by other staff members.The client applications in use include Microsoft Offi ce2003, Adobe Indesign, Adobe Acrobat and some specialized applications like ESRI/ArcGIS, S.Draw, E!Sankey, LiteRat, GAMS/MAR-KAL modelling SW, Adobe Creative Suite CS2, Adobe Acrobat Prof.The printing environment consists of 5 Hewlett-Packard BW/Color workgroup printers and 2

Austrian Energy AgencyConsolidation, virtualisation and power

management deployed in an IT renewal project

Hellmut Teschner, Thomas Bogner, Bernd Schäppi, Austrian Energy Agency

SUMMARYThis report provides a description of a best practice case for consolidation and virtualization of server equipment in a medium sized Austrian enterprise. The existing IT equipment was consolidated to a cluster of two servers and one storage unit using VMWare ESX 3.5i. Additionally the migration of services during periods of low system loads in combination with powering down of one server during weekends was evaluated. One of the project’s aims was to demonstrate the saving potential of energy effi cient hard- and software for SMEs. Implementing consolidation techniques led to an amount of 40% of energy savings in combination with easier administration of the systems, resulting in lower administra tion costs and a signifi cantly better quality of IT services. Powering down of one server during periods of low loads on weekends allows for additional 7% of energy savings.

5

Case Study 5Austrian Energy Agency

Analog Line

ISDN Line

CISCO CallManagerVOiP Telephone System

ADSL Router

Vmotion VLAN

Data VLAN

EMC CLARiiON AX4i2 storage processors1GB Cache mirroredRW cache 3,1TB

redundantStorage Switch

Cisco 3750/24x1GB

6th floor SwitchCisco 2940/48

5th floor SwitchCisco 3750/48

1GBi

t Fib

erop

tic L

ink

4th floorSwitchCisco 2940/48

IPProviderSilverserver

4Mbit SDSL Link

Cisco 800SeriesRouter

Juniper SecureServices Gateway

extern IP range AEAredundantCore SwitchCisco 3750/24x1GB

3rd floorSwitchCisco 2940/48

Admin VLAN

Admin VLAN

ISCSI VLAN

VMSRV2VMWareServer 2

FSC PrimergyRX300S42xXEON

E5405 CPU16BG, ESXi3.5

AEADC1Backupserver

VMSRV1VMWareServer 1

FSC PrimergyRX300S42xXEON

E5405 CPU16GB, ESXi3.5

Telephone VLAN

MediaVLAN

intern IP Range DMZ EA

Web/ApplicationServer,PiV/1GBIP-intern:IP-extern:

Router intern/DMZ/extern/VPNPIV/1GB

Cisco 800 SeriesRouter

4Mbit SDSL Link

intern-IP Range DMZ EA

IP ProviderSilverserver

extern IP range AEA

DATABASE W2003SP2PIV/2GB/160GB RAID1SQLServer, exact Globe,e-synergy

Core SwitchCisco 3750/48

TERMINALDualXeon4GB/160GBW2003SP2

Data VLAN

Analog Line

ISDN Line

CISCOCallManager

VOiP TelephoneSystem

ADSL Modem

MediaVLAN

Admin VLAN

Telephone VLAN

MAIL W2003SP2Xeon/2GB/160GBRAID 2003,Sybari Client

AEADC1 AD Domain-controller SUS ServerDHCP/DNSCertificateserver,LicenseServer,

ARCVIEWApplication-ServerPS2GB, 160GB

EVA2KW2000SP4File/Print/Applications-server

AEADATENApplicationFileServerDualXeon2GB, 1000GB

24

XEROX Copy/Print/Scan stations, connected to the system via 100MBit Ethernet offering net-work wide services.A CISCO Call Manager based VOIP system pro-vides telephony services.The Austrian Energy Agency runs a Windows Server2003/ActiveDirectory based server sys-tem to provide the normal offi ce environment, as well as a rich palette of other applications, such as GIS systems and modelling tools both home grown and commercial. Several web services are served by 4 LAMP(Linux/Apache/MySQL/PHP) based systems both in house and externally hosted.During the relocation of the Austrian Energy Agency to the new location at Mariahilfer-strasse 136 in 2007 a new VOIP based telepho-ne system was deployed as well as the comple-

te IP network infrastructure based on CISCO Catalyst 3750 and 2940 switches.In the course of deployment of a new ERP sys-tem a revision of the existing server hardware took place. The former 8 Pentium IV/XEON based server systems were found to have reached the end of their lifetime concerning both performance and serviceability issues.The Clarion AX4i storage system providing 3.1 TB capacity is connected with 2 FSC Primergy RX300S4 Servers via ISCSI VLAN. An additional CISCO 3750 switch serves the ISCSI VLAN and the admin VLAN and will be combined with the existing server switch in a second step, providing additional redundancy also to the existing media-, server-, and tele-phone VLAN‘s.

One additional physical server is used for back-up purposes.The following charts show the environment before and after the consolidation project: Fig.5.1 & 5.2Tab.5.1 Hardware before Consolidation

Server Hardware OS; Software

DatabasePIV; 2GB RAM; 160GB RAID-1;

W2K3/SP2; MSSQLServer2000

TerminalDualXEON; 4GBRAM; 160GB

W2K3/SP2,MSTerminalservices

MailXEON; 2GB RAM,160GB RAID-1

W2K3/SP2,Exchange2003MSForefrontSecurity

Eva2K PIV; 512MB RAM W2KS; misc SW

AeadatenDualXEON,2GB RAM, 500GB

W2K3SP2,misc SW

Wien PIV, 512MB, 160GB W2K3

ArcView PIV,2GB, 160GB WXP

Aeadc1 XEON, 2GB, 160GB W2K3, AD

Fig.5.1: Austrian Energy Agency, IT-Infrastructure Server Overview before consolidation

Fig.5.2: Austrian Energy Agency, IT-Infrastructure Server Overview after consolidation

INTERNET

INTERNET

5

25


The system is confi gured as a DRS Cluster offe-ring DRS and HA functionality(5.2 shows state immediately after implemen-tation with HA features still disabled) Tab.5.2: Hardware after Consolidation

RUNTIME ENVIRONMENTAll replaced server systems were either Win-dows 2003R2 or Windows 2000SP2. Since some applications, running on the W2K en vironment could not easily be ported to a W2K3 environment, one old W2KS system has been migrated via the VMWare warm migration tool. These applications will stay on W2KSP2 environment until the end of their lifetime.

Since power consumption of the old servers differs only about 10% between idle mode and average load there is very little difference when measuring idle mode or average load.

For the new server environment VMware ESX 3.5i is used as software layer for virtualization and VMWare Virtual Infrastructure is used for controlling the whole environment. VMware itself is set up as a cluster to offer high availa-bility for the new infrastructure. The VMWare VMotion functionality is used also for migration of virtual machines during times of reduced loads, e.g. on weekends.

Optionally shutting down one of the two ser-vers during weekends using the VMWare Dis-tributed Power Management feature was tes-ted. Since this feature is still classifi ed as “experimental” under ESX 3.5i and the server system was already running productive at the time of the measurements and furthermore additional features on the core switches have to be enabled for automatically restarting the

systems, in a fi rst step we identifi ed all services not necessary during non offi ce hours, like for instance development systems etc. After deter-mining the resource availability on the target server the remaining virtual machines were migrated using VMotion and the server was brought to standby mode. As soon as the VM-Ware DPM feature is released for production environments, which is announced for VMWa-re ESX 4.0, the concerned server can be shut down completely during times of low system loads and be brought up again automatically, when loads rise.

Administration of the W2K3 and Linux guests is highly enhanced using the VMWare snapshot functionality, for instance after application of patches it is possible to revert to the old state in case of problems almost without any service interruption.

Linux web servers as well as W2K3 or WXP in-stances for testing purposes can be rolled out in minutes using previously prepared templates. The virtual switching functionality allows for building of internal demilitarized zones and other networking segmentation.

All the old W2003R2 instances have been mig-rated 1:1 with VMware tools to become a VM-ware guest. At the moment 7 different Windows instances and 3 Linux instances are running as guests in the VMware environment. Power management features – as far as they did exist in hardware and software – were not activated on the old nor are activated on the new server environment.

Method for capturing energy consumption and workloads

CAPTURING THE WORKLOADS For measuring the utilization of the old systems we used the “perfmon“ tool, capturing CPU utilization, disc IO, storage utilization and net-work traffi c. As this tool would not be able to collect reasonable data on virtualized systems,

we used the performance chart functionality provided by the VMWare Virtual Infrastructure Client.

MEASUREMENT OF POWER CONSUMPTION

For a measurement of the total power con-sumption of the existing server system the fused outlet line powering the UPS was selected as best choice providing an opportunity for a non intrusive measurement without any shut down, as the majority of the servers were equipped with one single power supply only. The data logging was done technically by using a 4 channel power analyser and clamp on current transformer in the switchboard in the time period 15th to 21st Sept. 2008.Beyond this two servers out of 8 populated with redundant power supplies were measured individually with plug-in power meters (Energy-Logger 3500)

In this setting the consumption of the servers and switches as well as the power loss of the UPS was monitored. As mentioned above a separate measurement of every single server was not feasible due to the constraint of avoiding any power down.

Since the new servers and the external storage unit are equipped with redundant power sup-plies a measurement of the power draw was done directly. A power analyser PNA 560 was used to capture the power demand of the servers (2 data chan-nels each) and a power meter was chosen to measure the energy consumption of the storage unit.Power data was recorded from 5th to 19th December 2008.Consequently the UPS’s power loss and the power consumption of network devices were not covered in this concept of data acquisition.

For a proper comparison of the power data of the initial and the new system it had to be assured to exclude the share of UPS losses and switches from the total power draw.

Server Hardware OS; Software

VMSRV1VMSRV2

FSC Primergy RX300S4, DualXEON E5405, 16GB RAM

ESX3.5i

SAN: EMC 3.1TB Raid-5,

Clarion IV4 2 Storage processors

The energy effi ciency of the UPS in part load was estimated at 85 % based on product documentation. The power consumption of the switches was extrapolated from two measured values of switches.Following this approach a robust assessment of the achieved energy savings from using new hardware can be derived.

SHORT DESCRIPTION OF MEASUREMENT EQUIPMENT • Power Network Analyser Dewetron PNA

560The PNA 560, a power analyser designed for professional purposes and equipped with four measurement channels each for voltage and current was used as a power logger. Currents can be metered directly or via current transformers. Recordings can last up to several months and data is stored on a hard disk. (Fig.5.3)

• Power meter Voltcraft Energy Logger 3500The power meter has to be plugged between the device being measured and the power outlet. Power data can be recorded for several weeks, whereas one value is stored per minute. The power data can be exported via a SD memory card to common fi le types (csv or xls).

The measurement uncertainty of the Power meter is specifi ed for ±1 % + 1 Count in the metering range of 5–3500 W, for lower power range (2–5 W resp. < 2 W) a accuracy of ± 5% + 1 Count resp. ±15% + 1 Count has to be excepted.

Measurement results and quantifi cation of energy savings

COMPARISON OF WORKLOADS BETWEEN OLD AND CONSOLIDATED SYSTEMSSystem loads generally show big differences depending on the server type.

The Exchange server shows relatively high loads during offi ce hours, but partially also during nights and weekends due to webmail usage and intermittent heavy email traffi c.

The loads on the fi le servers are generally low on weekends and at nights except during backup runs and vary with some peaks during offi ce hours.Application servers certainly show low loads during non offi ce hours and partially high loads

26

5

otherwise, especially during runs of energy modelling software etc.

The average CPU load of the old servers was evaluated being rather low. In concrete fi gures the average CPU load of the email resp. termi-nal server amounted to 8% resp. 5 %. Never-theless peak load values were monitored in the range of 90% and 40%.Those dependencies infl uence the confi guration of VMWare distributed resource management on the consolidated/virtualized system in order to maximize the possibility to shut down one of the two servers on weekends. Backup runs as well as jobs for database consolidation etc. should be scheduled to be run during weekday nights.

The following charts fi g.5.5 and fi g.5.6 show the load profi les on the two ESX servers on the consolidated system. CPU and memory loads on the second EXS server are generally higher due to distributed resource management con-siderations. The loads do not affect the perfor-mance of the hosted servers as they are within the specifi ed performance spectrum of the 2 socket quad core systems.

Fig.5.3: Power Analyser PNA 560 (Source: Dewetron) Fig.5.4.: Energy Logger 3500 (Source: Voltcraft)

MHz

MHz

27


Fig.5.5: Load curve of ESX server 1

Fig.5.6:

Load curve of ESX server 2

The 2000MHz level equals

approximately 25% load.



5

28

Fig.5.8: power consumption new system

Fig.5.9: power consumption of the new system workload dependent server operation – power down of 1 ESX server on weekends

Fig.5.10: power consumption of the new system workload dependent server operation – power down of 1 ESX server on weekends and during the night

Fig.5.7: power consumption initial system

P_S_exch: power consumption exchange server (measured)P_S_fi le power consumption File-Server (measured)P_S_rest … power consumption of other servers (derived, cumulative)P_Switch power consumption of switches (measured, extrapolated)P_USV_verl power loss of UPS (estimated)

W_server [kWh]

P_storage [W]

P_S2 [W]

P_S1[W]

W_serv+stor [kWh/d]

P_storage [W]

P_S2 [W]

P_S1[W]

W_serv+stor [kWh/d]

P_storage [W]

P_S2 [W]

P_S1[W]

W_serv+stor [kWh/d]

Pow

er [W

]Po

wer

[W]

Pow

er [W

]Po

wer

[W]

Ener

gy [k

Wh/

d]En

ergy

[kW

h/d]

Ener

gy [k

Wh/

d]En

ergy

[kW

h/d]

time

time

time

time

RESULTS AND EVALUATION OF ENERGY SAVINGS:The measurement of the power consumption of the old server system showed a total power draw of 1.295 W (avg.) This corresponds to an annual energy consumption of 11,345 kWh. For comparison of energy saving effects three options are distinguished:

OPTION 1: All existing servers are migrated to one physical ESX server. This options would offer a slightly improved availability as the new server is equip-ped with two redundant power supplies. In the old server system only the minority of servers was powered by redundant power supplies. Bey-ond this no more hard disc is integrated in the server, as the entire storage volume is contained in an external storage unit. The performance of the server is appropriate in terms of expected workload.This 1:1 replacement would result in energy savings of 59,5 % (see Tab.5.3)

OPTION 2: In option 2 a second ESX-server is implemen-ted. All applications are run on both servers in a shared way in a virtualised environment. The availability of this system is higher compared to fi rst option, although some decrease in energy effi ciency has to be accepted. This option fi nally has been implemented.The power consumption of the new ESX servers and the storage unit amounts to approx. 525 W. In terms of annual energy consumption this re-sults in 4,600 kWh. Considering the complete consolidated system including fi ve additional vir-tual servers, added redundancy and higher avai-lability in combination with easier administration energy savings of about 40 % could be realized.OPTION 3 (currently tested only): This alternative is based on the hardware system described in option 2, but deploys power management on server level.One server is shut down on weekends (between Friday 10 p.m. and Monday 5 a.m.) after all applications have been migrated via VMotion

on the remaining ESX-server (Fig.5.9). Beyond this the energy consumption can be reduced further by switching off one server during the week between 10 p.m. and 5 a.m. as well (Fig.5.10). When utulizing this additional saving potential using the VMware Vmotion function, energy savings of about 47% for weekend shut down and 50% during overnight shut down can be achieved (Tab. 5.3).

Tab.5.3: Options and energy savings

29

We found, that the high availability feature offered by VMWARE Virtual Infrastructure provides enough re dundancy to nearly any application used at Austrian Energy Agency, without implementing any software cluster functionality with the need of at least two servers per application, also avoiding the additional power consumption of the hardware. The scenario using the VMWare Distributed Power Management feature is currently effi cient at this time for smaller enterprises, as VMWare Virtual Infrastructure Client in combination with a specialized switch confi guration is needed, which normally is only available as part of the bigger licences.

As energy demand for cooling has not been considered the real energy savings including infrastructure are twice as high compared to the values reported here.Hardware requirements for im ple-mentation of a new ERP system as well as for new intranet and internet services could be satisfi ed with the consolidated system.Administration of the system is highly enhanced and additional re-dun dancy is added, enabling us to im prove quality of IT services to our staff members as well as to web service users. The snapshot functio-na lity of VMWare Virtual In fra -structure makes implementation and testing of new services, as well as patch ma nagement much easier and

re duces the risk of service inter-ruptions.Desktop virtualization was not taken in consideration, as it would not meet the requirements of most of our users.It should be stated, that structured planning with the help of external experts as well as intense training of the involved staff members have proved to be absolutely crucial to the success of such a consolidation/virtualisation project.

This project has shown that con -so lidation/virtualization can be suc cessfully implemented in small and medium enterprises to improve the quality of IT services in conjunc-tion with saving energy.


System options System old

System new

Opt. 1 Opt. 2 Opt. 3

P _ges gem [W] 1,295 525 773 690

W [kWh/w] 217 88 130 116

W [kWh/a] 11,345 4,600 6,770 6,056

Annual energy savings [kWh/a]

-6,745 -4,575 -5,289

Savings [%] 59.5 40 47

CONCLUSION AND LESSONS LEARNED

30

Objective

In a software development and engineering company with 12 employees, the existing cen-tralised servers are to be regrouped into a single colocation centre. The aim here is to reduce hardware costs and cut electricity consumption by virtualising the servers.

Description of system

The servers run under the “Windows 2003 Server” operating system, and operate the following main applications: • Small business server• Exchange server• SQL server• IIS server• MS Office• Citrix Metaframe server• File server

Four servers are installed in a local, small server centre, each of which is equipped with its own power supply unit:The servers were powered via a local UPS de-vice, and were switched off via this device at night between 10 p.m. and 5.45. a.m.

Encontrol AGServer virtualisation in a Swiss service company

on SME level increased energy effi ciency

Alois Huser, Encontrol AG

SUMMARYThe study shows that, in a typical small company, it is possible to reduce elec-tricity consumption by around twenty-fi ve percent through the virtualisation of servers. It was not possible to reduce the electrical power by the same factor as the number of physical servers (reduction from 4 to 2 servers). The reasons for this are that the IT capacity of new servers is higher and their power re-quirement has generally increased in the past few years. CPU load is still low even after virtualisation, and this means the virtualisation potential has by no means been fully exploited. With large-scale virtualisation projects involving a greater number of servers, it is possible to reduce the number of physical servers by a much higher factor, and this means that the potential for reducing electricity consumption is also much greater. The measurements yielded valuable fi ndings for IT managers, and the results regarding CPU load indicate that further studies should be carried out con-cerning the status of server applications.

6

31

With the new system, two servers have been installed in an external colocation centre, and each server is equipped with two power supply units:

Measurement procedure

Both servers were measured at the input to the IT appliances (i.e. after the UPS device). The following device was used for measuring the electrical power:Manufacturer and type: Voltcraft Plus Ener-gy Logger 3500Active output range: 1.5 to 3,500 WLevel response: 0.1 WAccuracy: +/- 1%Own consumption: 1.8 WMaximum recording duration: 6 monthsThe active output was measured at intervals of 1 minute.At the same time the CPU load was also mea-sured with the aid of the performance monitor provided by the Windows operating system.

Results of electrical power measurements

The electrical power of the IT appliances was measured in normal operating mode.

The electrical power and CPU load were mea-sured for a period of 1 week.

EXPERIENCES WITH THE MEASURING DEVICEThe device used in this study was recently brought onto the market in Germany. It permits the simple storage of a variety of measurement data on a standard card so that the data can be conveniently transferred to a PC for evaluation purposes. Measurements carried out for com-

Case Study 6Encontrol AG

1 Windows ist eine eingetragene Marke der Firma Microsoft

new system Tab. 6.2

Server2 x HP DL380G5 (1 x Intel Quad-Core Xeon / 2, 3 x 146 GB Hard disk, Power supply 2 x 700W)

Router/Firewall ZyWall 35/Firewall DMZ VPN

Switch AT-FS705LE

old system Tab. 6.1

Communications server (2007)

IBM x3400 (2 x Intel Xeon 2 Core 2.0 GHz, 3 GB RAM, 3 x 73 + 1 x 300 GB Hard disk, 1 Power supply 835 W)

Database server and domain cont-roller (2005)

IBM eServer xSeries 226 (1 x Intel, Xeon 2 Core 2.0 GHz, 3 GB RAM, 3 x 73 GB, Hard disk, 1 Power supply)

Fileserver (2003)IBM eServer xSeries 225 (2 x Intel Xeon 2 Core 2.67 GHz, 3 GB RAM, 3 x 36 GB Hard disk, 1 Power supply)

Web-Interface HP Evo D510 (2003)

Router Cisco Router 1721 (2003)

FirewallCisco PIX 506E Firewall 3 DES Bundle (2007)

Switch Cisco AT-FS705L (2006)

Results of electrical

power measurement Tab. 6.3old system new system

Communications server

230 W2 x

Servers 2 x 256 W

Database server and domain controller

150 WRouter/Firewall

12 W

File server 140 W Switch 3 W

Web interface 40 W

Router: Cisco Router 1721

9 W

Firewall: Cisco PIX 506E Firewall 3 DES Bundle

23 W

Switch: Cisco AT-FS705L

20 W

Total power 612 W Totalpower

527 W

Old systemFig.6.1: Electrical power and CPU load

of servers in old system during 1 day

Fig.6.2: Electrical power CPU load of servers in old system during 1 week

New systemFig.6.3: Electrical power and CPU load of servers in new system during 1day

Fig.6.4: Electrical power and CPU load of servers in new system during 1 week

el. P

ower

[W]

el. P

ower

[W]

el. P

ower

[W]

el. P

ower

[W]

CPU-

Load

[%]

CPU-

Load

[%]

CPU-

Load

[%]

CPU-

Load

[%]

el. Power

CPU‘s Comm-Server

CPU‘s File-Server

CPU‘s DB/Domain-Server

el. Power

CPU‘s Comm-Server

CPU‘s File-Server

CPU‘s DB/Domain-Server

el. Power

CPU‘s Comm-Server

CPU‘s M-Virt-Server

el. Power

CPU‘s Comm-Server

CPU‘s M-Virt-Server

Time

Time

Time

Time

el. Power and CPU-Load Tuesday 13/05/08

el. Power and CPU-Load from Tu 13/05/08 to Mo 19/05/08

el. Power and CPU-Load Mo 02/06/08 to So 08/06/08

el. Power and CPU-Load Mo 02/06/08

32

parison purposes using an EMU device that has been available for several years demonstrated that the accuracy of the new device is accepta-ble. Like SUN in Germany, we found that the quality of the new device is not constant, and this means that each device needs to be tested after purchase. SUN reported that around 20 percent of the purchased devices did not func-tion correctly.

We found that the devices we used would occasionally record an incorrect electrical pow-er level (voltage and current were correct) and power factor. However, these faulty readings could easily be fi ltered out with the aid of soft-ware (displayed power factor, 2.55). Other users of this device have also reported this effect. It should also be noted that the internal clock of the measuring device is not precise, as a result of which it deviates from the actual time if the device is used for lengthy periods.

INFLUENCE OF A SECOND REDUNDANT POWER SUPPLY UNITEach of the two servers in the new confi gurati-on is equipped with a second redundant power supply unit. The measurements of the electrical power of these units were as follows: • Mean output with 2 power supply units run-

ning in tandem: 527 W• Mean output with one power supply unit in

operation: 467 W

The recorded level with two power supply units is thus 13 percent higher than with one unit.

SAVINGS IN TERMS OF ELECTRICITY CONSUMPTIONWith the new system (new servers with one power supply unit) the electrical power was re duced by 24 percent, and thanks to virtualisa-tion the number of physical servers was halved. This means it was not possible to reduce the electrical power by the same factor. The IT capacity of the new servers is higher and their power requirement has generally increased in the past few years. For example, the typical elec-trical power of a low-end server a few years ago was around 150 W, compared with an average of 200 to 250 W today.

In practice, the electrical power in the new sys-tem is in fact higher than in the old system since the system requirements have changed: • To prevent interruptions, the new servers use

two redundant power supply units. • Since ASP services for clients are also provi-

ded via the servers, the latter are no longer s witched off at night, whereas with the old system by doing so it was possible to reduce elect ricity consumption by around 30 percent.

With virtualisation projects involving a large number of servers, it is possible to reduce the number of physical servers by a much higher factor, and this means that the potential for

33

CONCLUSION AND LESSONS LEARNED Despite a reduction of the physical servers by a factor of 2 the energy savings only amount to approx. 24%. This case study proves that the deployment of concepts for consolidation and virtualisation in very small systems in small enterprises only leads to very limited effects.

Benefi ts from energy effi ciency are overcompensated by higher performance and different system requirements by trend. Nevertheless this case study offers very valuable insights.

reducing electricity consumption is also much greater.

POWER FACTORThe power factor is around 0.9 on average. A better result could have been achieved if the power supply units had been equipped with a more effi cient power factor correction.

ELECTRICAL POWER AND CPU LOAD The electrical power of the overall system only depends on the CPU load to a limited extent, since other devices (disks, RAID controller, net-work card, motherboard, etc.) account for a very high proportion of the overall power requi-rement. The variation in electrical power due to the fl uctuating CPU load is around 13 percent with the old system and 2 percent with the new system. The average CPU load is low: Old system: Communications server: 12% | File server: 3% | Database / domain server: 14%New system: Communications server: 8% | Multi-virtual server: 14%

The measurements also showed that high CPU loads occur periodically and frequently (e.g. hourly peak loads for periods of 3 minutes). These peak loads indicate activities of applica-tions that are not normal and therefore need to be investigated by IT managers.

Case Study 6Encontrol AG

34

Initial Situation and objectives

The cooperation between Sozialwerk Nürnberg and VALEO IT (a leading IT system provider and IBM BladeCenter & Storage Solution Center in Mittelfranken/Oberpfalz) started in 2005. VALEO IT now provides full service, administration and management of the IT infrastructure for approximately 80 users. The strategic investment in the IT environment helped to overcome the following limitations:

ECONOMIC ASPECTS• The stepwise growth of the old infrastructure

generated an increased power consumption and waste heat. In addition the existing di-versity made it more difficult and more time-consuming to manage the infrastructure.

SERVER ROOM• The location for the IT equipment is restricted

primarily in space (one rack) and in the ca-pacity of the air conditioning. The maximum temperature in the server room was about 30°C in summer.

Sozialwerk NürnbergEnergy-effi cient IT Infrastructure at “Sozialwerk Nürnberg”

Manfred Kaiser, VALEO IT GmbHRagnar Tanase , VALEO IT GmbHAnne Klose , IBM DeutschlandSilvio Weeren, IBM DeutschlandFrank Hummert, Sozialwerk Nürnberg

SUMMARYThe following best practice case demonstrates the energy savings achieved through the consolidation of the IT Infrastructure at the Salvation Army in Nuremberg. The Sozialwerk Nürnberg gGmbH provides aid for approximately 220 homeless men and women. The institute employs over 90 workers of multiple professions varying from social workers and care specialists to administration and craftsmen.

The mission statement of the Sozialwerk Nürnberg is “the individual is central”. This means that all help and support offered is focused on the needs of the individual.

Within this project the server hardware of ten single servers was replaced by an IBM Bladecenter E and external IBM Storage DS3400. Thereby the energy consumption of the whole IT hardware could be reduced by 782W (43%). The decrease in thermal output (and therefore improved effi ciency) and the fact that the IBM Bladecenter is able to operate at temperatures up to 35°C, allowed to switch off the air conditioner leading to further savings of 576W (32%). Additionally the infrastructure became more consistent and easier to manage.

7

Fig.7.1:

Sozialwerk Nürnberg in Nuremberg

35

SERVER HARDWARE • The hardware consisted of ten different ser-

vers. Some of these were about to run out of warranty/ maintenance shortly.

The server room is not equipped with a special data center air conditioner but with an usual split air conditioner. So on warm summer days the temperature can raise critically and the air conditioner can not provide the cooling to the targeted 24°C.The Sozialwerk Nürnberg stores the private data for their inhabitants. These data are very sensitive and confi dential. As many other public utility institutions, they have a rather limited budget for the IT. So one focus for the IT is the relation between costs and quality.To achieve the requirements of an adequate availability, reliability and cost effectiveness, Sozialwerk has decided to implement a BladeCenter infrastructure with external storage. They have taken the choice for the IBM BladeCenter E and Storage DS3400. The BladeCenter E infrastructure is highly energy effi cient, fully redundant with best density.

System Description

The old systems consisted of different x86 servers from IBM. These were predominantly “one server – one application” implementations based on Microsoft Windows and Linux. All servers in Nuremberg have been covered in the evaluation.

The new infrastructure consists of six HS21 (Model 8853-GLG) BladeServer. In this server-model Intel’s Xeon L5420 quad core low voltage processors (50W) are used.

Currently, the Bladecenter is equipped with six servers – up to fourteen servers are possible. The total RAM has been extended from 18,5 GB to 30 GB.The IBM System Storage DS3400 is confi gured with seven 136GB SAS 15k RPM Raid 10 and fi ve 700GB SATA 7,2k RPM Raid 5 hard disks. With each HotSpare HDD the total net storage capacity has been increased from 1,3 TB to 2,5 TB. So the planned growth in the future is easy to implement.

Case Study 7Sozialwerk Nürnberg

ServerName

Model CPU & Clock Rate

Memory MB

Size HDDs

SQL - Server IBM x345 Intel Xeon 3.06 GHz

4096 73

Datev WTS IBM x336 Intel Xeon 3 GHz

1024 36

SozDatev Server

IBM x345 Intel Xeon 3.06 GHz

1024 73

COM Server IBM x306m

Intel Pentium4

1024 160

NAS Server IBM x3655

Opteron Dual Core

4096 500

Newton (VMware ESX Server)

IBM x346 Xeon EM64T 3 GHz

2048 73

Backup Server

IBM x306 Intel Pentium 4 3GHz

512 160

Oracle Server

IBM x345 Intel Xeon 3.06 GHz

2048 73

Horizont IBM x345 Intel Xeon 3.06 GHz

2048 36

Tab.7.1: Serverlist „old infrastructure“

Fig.7.2:

Old IT infrastructure

ServerName

Model CPU & Clock Rate

Memory MB

No. of CPUs

Exchange HS21 2.5 GHz Xeon

8192 2

TERMINAL HS21 2.5 GHz Xeon

4096 1

File Server HS21 2.5 GHz Xeon

4096 1

Backup Domain Controller

HS21 2.5 GHz Xeon

4096 1

DB Server HS21 2.5 GHz Xeon

4096 1

Domain Controller

HS21 2.5 GHz Xeon

6144 1

Tab.7.2: Serverlist „new infrastructure“ blade server

36

Workload Measurements

The servers run various fl avours of Microsoft Windows and Linux. In the long run the main operating system will be Microsoft Windows. The measurement of the workload and performance of the infrastructure were done with the Windows performance monitor on the old and new environment. So the monitoring of e.g. CPU utilisation and RAM utilisation for each system is possible.

Period 1: Measurement of the old infrastructure Start: 5th December 2008 Finish: 14th December 2008

Period 2: Measurement of the new infrastructure Start: 6th February 2009 Finish: 13th February 2009

OLD INFRASTRUCTUREDue to the “one application – one server” implementation the x86-servers operated at low utilisations. Table 3 lists the CPU utilisation as this is the only performance value signifi cantly correlating with energy consumption. Figure 7.4

shows the corresponding variation of utilisation over one working day.The server utilisation may be further opti-mised by virtualisation technology. Thereby the number of physical servers could be reduced as well as the energy consumption. The virtualisation as implemented on the previous VMware ESX server (see fi gure 7.5) is no longer used because a virtualization solution in this project is uneconomical. This is because the major appli-cations e.g. DATEV tax and social insurance applications are not supported on VMware by the SME software manufacturers.

NEW INFRASTRUCTUREThe servers and applications were not simply ported 1:1 on the new systems. By consolidation of some applications, the new infrastructure has been optimized and the number of servers was reduced from ten to six. Infrastructure was reviewed by IT architects to check which appli-ca tions can run on the same server, are no longer needed in the future or can take over other services. Figure 7.5 shows in which way all old applications and functionalities are mapped to the new infrastructure. The graph shows that a fair comparison of the energy consumption of the old and new system is possible.

Tab.7.3: CPU utilisation per server from the old infrastructure

Server

ServerName

Ave-rage

Standard Deviation

Mini-mum

Maxi-mum

Domain-controller

4.79 0.62 3.93 10.44

Horizont 0.41 0.39 0.19 4.69

Com-Server 0.22 0.46 0.00 4.83

NAS-Server 0.98 0.18 0.73 4.06

SozDatev Server

0.61 1.57 0.05 19.34

SQL-Server 3.28 0.58 2.57 6.77

Datev WTS 1.66 3.84 0.22 41.47

Backup-Server

2.76 0.35 1.64 6.69

CPU Utilisation %

Fig.7.3: New IT infrastructure

with IBM Bladecenter E & Storage DS3400

37

Energy Measurements

MEASUREMENT CONCEPT AND METERIBM has installed two intelligent Power Distribution Units with integrated power meter and IP communication and has measured the energy consumption of all old and new hardware for one week (with time resolution of one minute).The servers with redundant power supplies have been switched from the existing passive PDU to the iPDUs during normal operation. First one power supply was switched. After a period of about 30-60 seconds this power supply was powered up again. Finally the second power was switched. This worked for all servers although there is a small risk for a power supply to fail when it has to take the full load.The IBM iPDU is an easy to install, fl exible & scaleable 1U power distribution unit that monitors the power usage at the breaker level. The device provides the main outlets, breakers, indicator lights and cord connection all on the same side of the unit for easy access. Integrated in the network (own IP address) the data can be read out via web interface or IBM Director with Active Energy Manager. The relative failure is +- 5%.

Fig.7.4:

CPU utilisation per

server of the old

infrastructure over

one day

Fig.7.5: Plan of server consolidation

45

40

35

30

25

25

Utilisationin Percent

Time

Domain

contro

ller

Backup-

Server

SQL-S

erver

Datev W

TS

SozDate

v Serv

er

NAS-Serv

er

Kom-Se

rver

Horizon

t

15

10

5

0

18:4517:55

17:0516:15

15:2514:35

13:4512:55

12:0511:15

10:259:35

Fig.7.6: Front side iPDU (1U high) with six independent ports for energy measurements


38

As the IBM BladeCenter and storage can be operated at ambient temperatures of up to 35°C, air conditioning is not needed except at extremely hot summer days. Sozialwerk plans to optimise the air fl ow with direct feeds of fresh air to the Bladecenter and storage and passive exhaust through bottom and top holes replacing the current window. Based on this measure more than 32% of energy consumption can be saved during summer.The existing A/C will remain installed for manual operation on extremely hot summer days.

Results

The new dynamic IT infrastructure with IBM BladeCenter E and Storage DS3400 provides the same services and applications as the old infrastructure. Through the server consolidation from ten single servers to six low power blade servers the energy effi ciency, performance capabilities, capacity and availability has been strongly improved.

MEASUREMENT RESULTSTables 7.4 and 7.5 show the weekly average power (consumption) for the old and new environment.

In the new infrastructure the system manage-ment solution IBM Director 6.1 is used to monitor, control and manage the whole system and the Active Energy Manager plug-in provides data on power of all individual components including the single BladeServer as shown in fi gure 7.7.

SWITCH-OFF OF AIR CONDITIONFigure 8 shows the split A/C and the window next to the rack. The A/C draws an average of 576 W measured on rather cold winter days. This is 1/3 of the total power of the old infrastructure. At peak times the air conditioner consumes 1244 watts – about 2/3 of the complete old IT infrastructure. During summer a higher energy consumption up to 1300W can be assumed.

Server Name Power

SQL-Server 267 W

Datev WTS 128 W

Datev Server 186 W

COM-Server 123 W

NAS Server 158 W

Newton (VMware ESX) 226 W

Backup 151 W

DC 184 W

Oracle Server 166 W

Horizont 229 W

Total 1818 W

Server Name Power

BladeCenter 758 W

DS3400 278 W

Summe 1036 W

Tab.7.4: Average power of old infrastructrure

Tab.7.5: Average power of new infrastructure

Fig.7.7: Active Energy Manager interface with

the energy consumption of a single blade server - the NAS Server

39

Despite of the consolidation a second domain controller was deployed to additionally increase the availability and to host the IBM Systems Director management.The confi guration of the servers has been upgraded. Beside other improvements the server performance has been increased through new quad core processors, a larger memory (+62%) and nearly double storage capacity.

Figure 7.9 shows a comparison of the stacked average power of the old and new hardware and infrastructure and the resulting overall energy saving of 43% for the IT and the additional 32% saving from switching off the air conditioning.

LESSONS LEARNED

With the IBM BladeCenter and storage used in this case study, even small and medium enterprises can implement a dynamic IT solution with signifi cant energy savings. The even more effi cient BladeCenter S, which integrates a maximum of six blade server and storage with twelve disks, was not applied here since additional servers shall be implemented in the near future. Up to eight new servers can be added with the potential to triple the performance with very low additional energy consumption of about 60-80 Watt per server. With server virtualisation further energy savings can be achieved if the virtualisation of main applications is supported by the software manufacturers in the future.The blade centre and storage provide a fl exible “data center in a box” solution for a dynamic IT infrastructure for SMEs with minimum requirements on air conditioning and thus extreme energy effi ciency.

Fig.7.8: Split air conditioner with average power of 576 W (in winter time)

43% Power Savings by Consolidation toIBM BladeCenter E & Storage DS3400

Fig.7.9: Energy savings due to new dynamic IBM IT infrastructure (43%) and due to switching off the air conditioning (32%)

Air conditioner

Horizont

Oracle Server

DC

Backup

Newton (VMware ESX)

NAS Server

COM-Server

Datev Server

Datev WTS

SQL-Server

DS3400

BladeCenter

43%

32%

2500

2000

1500

1000

500

0

POW

ER [w

att]

OLD NEW


40

Project team: Austrian Energy Agency (Project coordination)IBMSUNUniversity of Karlsruhe French Environment and Energy Management Agency (ADEMERobert Harrison Associates LTD

Detailed information and downloads available onwww.effi cient-servers.eu

design: fl oorfour.at

The sole responsibility for the content of this brochure info folder lies with the authors. It does not necessarily refl ect the opinion of the European Communities. The European Commission is not responsible for any use that may be made of the information contained therein.

Case Study 1 City of Bad Soden am TaunusBernhard Przywara, Sun Microsystems Andreas Fiedler, IT City of Bad Soden am TaunusJens Wagener, Sun MicrosystemsContact: [email protected]

Case Study 2 STRATO AGBernhard Przywara, Sun Microsystems Martin Müller, Sun MicrosystemsRene Wienholtz, STRATO AGOliver Fuckner, STRATO AGContact: [email protected]

Case Study 3 BundesumweltministeriumRudolf Herlitze, Bundesministerium für Umwelt, Naturschutz und ReaktorsicherheitSilvio Weeren, IBM Deutschland GmbH Oliver Gambero, IBM Deutschland GmbHMichael Schepanske, Bechtle AGContact: [email protected]

Case Study 4 „Padergreen“Hansfried Block, Fujitsu Siemens ComputersSabrina Eßer, Fujitsu Siemens ComputersJuergen Heidegger, Fujitsu Siemens ComputersContact: [email protected]

Case Study 5 Austrian Energy AgencyHellmut Teschner, Austrian Energy AgencyThomas Bogner, Austrian Energy AgencyBernd Schäppi, Austrian Energy AgencyContact: [email protected]

Case Study 6 Encontrol AG Alois Huser, Encontrol AG Contact: [email protected]

Case Study 7 Sozialwerk NürnbergManfred Kaiser, VALEO IT GmbHRagnar Tanase , VALEO IT GmbHAnne Klose , IBM DeutschlandSilvio Weeren, IBM DeutschlandFrank Hummert, Sozialwerk NürnbergContact: [email protected]

ENERGY AND COST CASE STUDIES - European Commission · 2014. 8. 11. · Case Study 1 City of Bad...

Documents

Transcript of ENERGY AND COST CASE STUDIES - European Commission · 2014. 8. 11. · Case Study 1 City of Bad...