The Science Behind Airflow

Management Best Practices

Lars Strong, P.E., Upsite Technologies

Mark Seymour, Future Facilities

Airflow Management Awareness Month

June 1, 2016

Identify Your Opportunity

© Upsite Technologies, Inc.6/2/2016 2

There are two types of computer rooms

1. Those without IT equipment intake air temperature problems

• Improving AFM enables them to be more efficient

2. Those with IT equipment intake air temperature problems

• Improving AFM solves intake air temperature problems (hot

spots and cold spots) and enables them to be more efficient

• More efficient results in: lower OPEX, deferred CAPEX,

increased cooling capacity

Why Airflow Management

© Upsite Technologies, Inc.6/2/2016 3

Total Load

IT LoadPUE =

Mechanical plant + cooling fans: 35% of total load, 73% of non IT Load

BYPASS AIRFLOW CLARIFIED

© Upsite Technologies, Inc.6/2/2016 4

Definition of Bypass Airflow

Any cooling air that does not pass through IT equipment before returning to a cooling unit

Bypass flow rate (CFM) = total cooling supply flow rate (CFM) – total IT equipment flow rate (CFM)

© Upsite Technologies, Inc.

ΔT Through IT Equipment

UPS load (kW) x avg CFM/kW = Total IT CFM

IT Equipment Required Flow Rate

IT

Equipment

Delta T (deg F)

15° 20° 25° 30° 35° 40°

Required

flow rate (CFM/kW)

211 158 126 105 90 79

© Upsite Technologies, Inc.

Bypass Airflow Clarified

Typical existing conditions

© Upsite Technologies, Inc.

Bypass Airflow Clarified

After AFM improvements

© Upsite Technologies, Inc.

Bypass Airflow Clarified

Optimized

© Upsite Technologies, Inc.

Upsite’s 4 R’s of Airflow Management™

A holistic approach

An iterative process

‘check in’ at the room level

after making any

adjustments

6/2/2016 10

Watch video at:upsite.com/resources/airflow-management/

© Upsite Technologies, Inc.

Airflow Management Best Practices

Gone Wrong, What We Can Learn

Mark Seymour

Future Facilities 11

This presentation illustrates some of the classic actions in a

data center that can create airflow management issues

The intent is to reveal:

• Why considering airflow is important

• That good practice is just that – it’s not a guarantee

• Engineering simulation is used to illustrate airflow that is

otherwise unseen and enabling proactive operational

management

• Airflow management applies to both the rack/cabinet level

and the room level

Introduction

© Upsite Technologies, Inc.6/2/2016 12

futurefacilities.co

m

Energy efficiency of the

cooling system can be

undermined by

• Negative flow

o Often through perforated tiles

near the cooling unit

• By-pass

o Can overshoot IT or pass

through unmanaged leakage.

E.g. cable penetrations

• Recirculation

o Occurs in the rack as well as

the room is important

Common issues

© Upsite Technologies, Inc.6/2/2016 13

CABLE PENETRATIONS & LEAKAGE

© Upsite Technologies, Inc.6/2/2016 14

futurefacilities.co

m

Unmanaged cable

penetrations

Typically installed in

raised floor in rear if the

cabinet

Installing cable

management is an easy

win but care should be

taken to ensure the

cooling is not in use

Leakage Paths

Watch for leakage paths

• Under the cabinet

• Over the cabinet (through containment)

• Between cabinets

• End of rows (through containment)

• Through Cooling Units on standby

• Similarly in the rack

o Under

o Over

o Between

o Around equipment

o Through IT equipment that is off

© Upsite Technologies, Inc.6/2/2016 15

futurefacilities.co

m

Cable Penetration Seals

Cable Penetration Management

Why practice it?

• Stop air leaking out of the

cable penetrations

• Increase the pressure in

the raised floor plenum

• Stop bypass cooling down

the hot return air

• Force more air out of the

perforated tiles and grates

in front of the IT

equipment

© Upsite Technologies, Inc.6/2/2016 16

futurefacilities.co

m

Hot Aisle / Cold

Aisle

Mix of Equipment

Size: 1,066 sq. ft.

Cooling: 55.6 tons

(196 kW)

Load: 152 kW

Example Room

© Upsite Technologies, Inc.6/2/2016 17

futurefacilities.co

m

ASHRAE Temperature Compliance

One rack has inlet temperatures above the ASHRAE

Temperature Compliance recommended range

© Upsite Technologies, Inc.6/2/2016 18

Using simulation we can investigate the impact of adding

brush seals to the cable penetrations:

Modeling of Penetration Seals

© Upsite Technologies, Inc.6/2/2016 19

futurefacilities.co

m

Static Pressure

© Upsite Technologies, Inc.6/2/2016 20

futurefacilities.co

m

Perforated Tile Flow

© Upsite Technologies, Inc.6/2/2016 21

futurefacilities.co

m

ASHRAE Temperature Results

Comparison

© Upsite Technologies, Inc.6/2/2016 22

Improving raised floor open area

management reveals need to improve

rack airflow management

futurefacilities.co

m

Why Did the Room Get Worse?

With Brush

SealsNo Seals

Improving rack airflow management with blanking

panels, rail seals, and under rack panels will bring IT

intake air temperatures back into compliance

© Upsite Technologies, Inc.6/2/2016 23

futurefacilities.co

m

Sealing holes in the floor are still the best practice

It is critical to apply best practices to sealing the

cabinet at the same time.

Final engineering simulation image would reveal

optimized cooling with no exhaust air recirculation

6/2/2016 © Upsite Technologies, Inc. 24

futurefacilities.co

m

Equipment Configuration

Consider a cabinet housing

a blade system and 3 1U

servers

• Does it matter how the 2

equipment types are

installed in this cabinet?

• Option 1 – Three 1U

servers placed on top of a

BladeCenter H

3kW Blade

Enclosure

3 x 1U 275W

Servers

15oC / 59°F Supply Air

© Upsite Technologies, Inc.6/2/2016 25

futurefacilities.co

m

Equipment Configuration

Base Configuration – Option 1

Recirculation

under cabinet

Maximum inlet temperature

oC (°F)

Blade 27.1 (80.8)

Server1 19.6 (67.3)

Server2 19.7 (67.5)

Server3 18.9 (66.0)

© Upsite Technologies, Inc.6/2/2016 26

futurefacilities.co

m

Equipment Configuration

Option: 1U servers under blade

Maximum inlet temperature

oC (°F)

Blade 18.1 (64.6)

Server1 19.1 (66.4)

Server2 20.9 (69.6)

Server3 21.8 (71.2)

Recirculation from

1u servers is cooler

Recirculation of this nature can occur

whether or not the data center is

contained. The solution is blocking the

under rack airflow.

© Upsite Technologies, Inc.6/2/2016 27

AISLE CONTAINMENT

© Upsite Technologies, Inc.6/2/2016 28

futurefacilities.co

m

Consider two rows of equipment with full cold aisle

containment:

Cold Aisle Containment

© Upsite Technologies, Inc.6/2/2016 29

futurefacilities.co

m

Containment Measured

Warmer Air

Supply

No change

Colder Air

Supply

© Upsite Technologies, Inc.6/2/2016 30

futurefacilities.co

m

Simulation shows what is happening:

• There is a lack of available airflow

• Uncontained, the additional air required recirculates via the room over the top of the cabinets. Consequently it mixes and so is colder than the air in the hot aisle

• Contained, the additional air recirculates directly from the back of the cabinets and, without any mixing, it is hotter.

Simulation Explains All

Uncontained

Contained

© Upsite Technologies, Inc.6/2/2016 31

COOLING UNITS

© Upsite Technologies, Inc.6/2/2016 32

futurefacilities.co

m

More efficient (up to 30%)

Easily controlled (0-10V

signal)

Longer life and reduced

maintenance

Improved airflow through

the CRAC/CRAH

Reduced noise

Benefits of EC fans?

© Upsite Technologies, Inc.6/2/2016 33

futurefacilities.co

m

Air Distribution

EC Plug Fans vs Traditional Blowers

Blower

© Upsite Technologies, Inc.6/2/2016 34

Traditional BlowersLowered EC Plug Fans

The different types of fans distribute air in different ways

The Effect on Airflow

© Upsite Technologies, Inc.6/2/2016 35

futurefacilities.co

m

6,727ft² of floor space

200+ Cabinets

800kW Cooling Capacity at

N+2

Test Facility

© Upsite Technologies, Inc.6/2/2016 36

Airflow distribution patterns look very different.

How has this affected the available cooling?

Grille Flow

© Upsite Technologies, Inc.6/2/2016 37

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13

Pote

ntia

l Coo

ling

(kW

)

Cold Aisle

Cooling Available per Row

© Upsite Technologies, Inc.6/2/2016 38

Reduced cooling capacity puts IT at risk of thermal

shutdown

Immediate Issues

Aisle 4 cooling

reduced 21kW

64 Blades now at

risk

Aisle 5 cooling

reduced 15kW

30 Proliants now at

risk

© Upsite Technologies, Inc.6/2/2016 39

Immediate Issues

© Upsite Technologies, Inc.6/2/2016 40

Upsite's 4 R's of Airflow Optimization provide a guide for

implement changes and achieving the greatest benefits in

your data center.

“Efficiency can’t be purchased,

it has to be managed”

Julian Kudritzki

- The Uptime Institute

Airflow management is essential for optimizing capacity

and efficiency of the entire data center

Good practices are often beneficial but should be

undertaken with due diligence

Airflow is invisible, therefore the hardest thing to manage

in a data center

Actions for one scenario may be beneficial while in

another counter productive – e.g. total containment in an

over-provisioned scenario compared with an under-

provisioned scenario.

Engineering simulation can visualize and help guide

airflow management choices

Apply the 4 R’s Methodology to Airflow Management

Summary & Conclusions

© Upsite Technologies, Inc.6/2/2016 41

June 8: 5 Myths of Data Center ContainmentPresented by Lars Strong, P.E.

June 15: How IT Decisions Impact Facilities: The Importance of Mutual UnderstandingPresented by Ian Seaton and Lars Strong, P.E.

June 22: The 4 Delta T’s of Airflow ManagementPresented by Lars Strong, P.E.

Register at upsite.com/airflow-management-awareness-month

Upcoming Schedule

© Upsite Technologies, Inc.6/2/2016 42

Thank you!

@UpsiteTech

blog.upsite.com

On LinkedIn

Lars Strong, P.E.

Senior Engineer, Upsite

Technologieslstrong@upsite.com

Follow Upsite for the latest

news and information on

data center AFM.

Mark Seymour

Senior Engineer, Future

FacilitiesMark.Seymour@futurefacilities.com