Green Data Centers Ken Hall, RCDD / NTS · 2008-10-03 · Green Data Centers Ken Hall, RCDD / NTS...
Transcript of Green Data Centers Ken Hall, RCDD / NTS · 2008-10-03 · Green Data Centers Ken Hall, RCDD / NTS...
August, 2008page 2 /Tyco Electronics Confidential and Proprietary
Green Initiatives: Materials
�Lead-Free, RoHS product set
�Reusable products �Modular Copper and Optical Fiber
Preterminated Communications cabling
August, 2008page 4 /Tyco Electronics Confidential and Proprietary
Next Generation Ethernet Networks
Power Consumption
Cooling Requirements
Transceiver Size
~2W ~8-15W
Fiber Copper
Data Center Area $ $$$$
August, 2008page 5 /Tyco Electronics Confidential and Proprietary
Green Initiatives: Power
� High-density Cabling � Enables 7-27% power reduction on per port basis� Smaller cable than six 4-pair cables
� Better airflow = better cooling = less power
� Shielded Cat6A Cabling� Up to 40% smaller cable than Cat 6a U/UTP� Potential power savings on network electronics
� XG Optical Fiber Cabling� Optical ports draw less power than copper ports
August, 2008page 6 /Tyco Electronics Confidential and Proprietary
Data Center Standards
�TIA 942 Telecommunications Infrastructure Standard, April 2005
� Substantial, available, North America
�EN 50173-5 Information technology � Data Centers
� Smaller scope, Europe
�BICSI 942, �complementary� to TIA 942
� ISO/IEC Draft 24764 Information technology
� Global Generic Cabling for Data Centers
August, 2008page 7 /Tyco Electronics Confidential and Proprietary
Hierarchical Structure of Data Centers
August, 2008page 8 /Tyco Electronics Confidential and Proprietary
Different Tiers of Data Centers (TIA942)
August, 2008page 9 /Tyco Electronics Confidential and Proprietary
Annual Downtime
24 min
1 hour
36 min
22 hours
1 day
4 hours
48 min
Downtime/
Month
Downtime/
Year
2 min8 min
1 hour
50 min
2 hours
24 min
0.4 hour1.6 hours22.0 hours28.8 hours
Tier 4Tier 3Tier 2Tier 1
Not much time for Re-cabling!
August, 2008page 10 /Tyco Electronics Confidential and Proprietary
Tiering For Cabling Systems (TIA942)
Security/Availability
August, 2008page 13 /Tyco Electronics Confidential and Proprietary
Power�.
� Demand for electricity is projected to grow 19% in next ten years� Generation capacity is projected to grow only 6% (Source: NA Electric Reliability Council)
� Virtualization & multi-core processors are driving dramatic increases in cabinet density� 2006: US servers consumed $14 B in power & cooling� 2009 Projection: $50 B (Source: IDC)
� 2011 Projection: $100 B � In 2008, half of the world�s data centers will not be able to utilize the
latest high-density equipment (Source: Gartner)
� Their power infrastructure simply won�t support it� 1W at the server takes ~ 2 watts at the meter
� Inefficiencies in the power chain components
August, 2008page 14 /Tyco Electronics Confidential and Proprietary
Availability, Space and Density
45%
52%
83%
0% 20% 40% 60% 80% 100%
Space
Availability
Power consumption
A survey of Data Centers owners and the biggest
concerns
Solution: flexible cabling solution, easy to migrate, just Place and Plug, high density
August, 2008page 15 /Tyco Electronics Confidential and Proprietary
Passive Cabling Solutions for Data Centers
� Data Center cabling requirements are different from the ones in Office Networks
� Dynamic vs. Static Environment� Easy deployment (weekend work/testing)
� High performance
� Structured scalability
� Fast & easy MAC�s
� Functional Security (less human errors)
� High density
� Effective cable management
� Availability (less planning, less downtime)
� Cooling support
August, 2008page 16 /Tyco Electronics Confidential and Proprietary
Passive Solutions for Data Centers
� Market situation� The market changed in 3 years to pre-terminated solutions� Copper
� High density copper 1 Gb/s systems� Integrated cable management
� Fiber� Multimode OM3 systems� MPO systems
August, 2008page 18 /Tyco Electronics Confidential and Proprietary
What applications are running in Data Centers ?� 10/100Base-T
� GigaBit Ethernet (1000Base-T, 1000Base-SX/LX)
� 10GigaBit Ethernet (10GBase-T, 10GBase-SR)
� Fibre Channel
� InfiniBand
� Coming: 40 & 100 GigaBit Ethernet
August, 2008page 19 /Tyco Electronics Confidential and Proprietary
0
5
10
15
20
25
3020
00
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
100M 1G 10G 40G 100G
mill
ions
ser
ver u
nits
x86 Servers by Ethernet Connection Speed (100 Mbps through 100 Gbps)
Source: Intel & Broadcom (April 2007)
10 year transition for 1G Ethernet 6 years for 10GBE 5 years for 40GBE
August, 2008page 20 /Tyco Electronics Confidential and Proprietary
GBE / 10GBE
� GigaBit Ethernet
� OM1/OM2/OM3 for fiber (1000Base-SX/LX)� Cat.5e for copper (1000Base-T)
� 10GigaBit Ethernet
� OM3 recommended for fiber (10GBase-SR)� Cat.6A/Cat.7/Cat.7A for copper (10GBase-T)
August, 2008page 21 /Tyco Electronics Confidential and Proprietary
� Summary of power measurements
0
5
10
15
Link speed (Mb/sec)
Pow
er u
se (W
)
10 100 1000 10000
10G Base-T is a power concern
g00.xls
10G Base-T Application Cabling Power
20082008
August, 2008page 22 /Tyco Electronics Confidential and Proprietary
10G Copper Cabling
� 10G Copper Expectations� 36% expect to install 10G copper in the next 1-2 years� 61% expect to install 10G copper in the next 3 years
� Out of those expectations, 75% of those plan to choose shielded copper over unshielded
0%
5%
10%
15%
20%
25%
Cat 6 F/UTP Cat 6A U/UTP Cat 6A F/UTP Cat 7 S/FTP Cat 7A S/FTP UndecidedSource: BSRIA: U.S. Data Center Structured Cabling Study, January 2008
August, 2008page 23 /Tyco Electronics Confidential and Proprietary
10G Base-T Application Cabling Power
� Reason� Signal to noise (S/N) ratio of UTP cabling systems� DSP�s are required to improve the signal quality
� Shielded systems have a much better EMC performance� Better S/N ratio� Less calculation for DSP�s� Less power consumption
� Conclusion� Shielded cabling enables reduction of network power consumption
August, 2008page 24 /Tyco Electronics Confidential and Proprietary
� 10GB-T PHY optimized for 100m UTP Cabling� 15-20W (in 90nm) (1000B-T: 500mW)
� Problem for High Density Line Cards, PCI Cards, MBs, Modules
� 2.5usec latency (1000B-T: 250ns)
� Too high for some High Performance Applications, FC, IB
� Power Hungry AFE
� 50% of Power is in AFE
� Not likely to improve much over time� Large percentage of the power is in the AFE
� Power reductions in smaller process geometries are questionable
10G Base-T Application Cabling Power
August, 2008page 25 /Tyco Electronics Confidential and Proprietary
� Power Dissipation� 3W Typical, 4W Worst Case
� Latency� Less than 500ns� Goal of 250ns
� Low Cost� Significant SNR Margin
(1000B-T: 10dB)� Demonstrable Design of 1M
Gates (2x 1000B-T)
� Short Reach� 30m on STP/FTP Cable (field
configurable)� 2 Connector, 3 Links Segments� Goal of 45m
� Auto Negotiation Compatible� RJ-45 Connector� Backwards Compatible with 100B-
T / 1000B-T over UTP
10G Base-T Application Shielded Cabling Power
August, 2008page 26 /Tyco Electronics Confidential and Proprietary
Characteristics of Data Center Cabling
� 90% of Links are in Server Rooms Less Than 5,000 sq. ft. (100% ≤ 30m)
� 5% of Links are in S. Rooms Between 5,000 ft² and 20,000 sq. ft.(80% ≤ 30m)
� 5% of Links are in Server Rooms Greater Than 20,000 sq. ft. (55% ≤ 30m)
� ≥ 97% Coverage with 30m
� ≥ 99% Coverage with 45m
August, 2008page 28 /Tyco Electronics Confidential and Proprietary
InfiniBand Bandwidth
2004 2005 2006 2007
Spee
d
40G
20G
10G8G4G2G
Infiniband10G EthernetFibre Channel1G Ethernet
120G QDR
DDR
SDR
Parallel lanes means parallel optics
August, 2008page 29 /Tyco Electronics Confidential and Proprietary
InfiniBand: High Speed Cabling Types
QSFP
August, 2008page 30 /Tyco Electronics Confidential and Proprietary
40/100 GBE Multimode Fiber (MMF) Solutions
!850nm 12x10G CWDM devices may support 100-300m
!Utilisation of MPO connectors is an important factor for
economic feasibility of 100G MMF
!Pre-terminated OM3 ribbon fiber will be cost-effective
!90% data centre links are MMF today and within 150m!High Performance Computing requires links to 100m!Majority of corporate building backbones under 300m!IBM survey found 70 - 80% of above links under 100m!Corning survey of data centre links found 96% <200m
August, 2008page 31 /Tyco Electronics Confidential and Proprietary
40/100 Gbps
� Official IEEE project � IEEE802.3ba� Current situation indicates an MPO solution� Reason: Parallel transmission needed
12-fiber MPO 72-fiber MPO
Examples of Tyco Electronics Parallel Optic connectors
August, 2008page 32 /Tyco Electronics Confidential and Proprietary
ISO/IEC 24764 data center standard
ZDMD LDP EQENI
ISO/IEC 24764 model for Data CentersENI
Network access cabling
subsystem
Main distributioncabling
subsystem
Zone distributioncabling subsystem
� Standard MPO systems allows mostly 2 cassettes for 10 Gb/s� Next Generation TE solution allows up to 6 cassettes
� Complete cabling from MD to EQ� Reduce number of actives� Allows more flexibility in data center design
August, 2008page 34 /Tyco Electronics Confidential and Proprietary
Infiniband
� The situation:� SAN�s a essential part of Data
centers� More and more data will be
stored� Infiniband and Fiber channel
most common applications� Mostly Copper for Infiniband and
fiber for Fiber channel
August, 2008page 35 /Tyco Electronics Confidential and Proprietary
Why Active Optical Cable Assemblies?
!Up to 200m!
!Bend radius 25mm!
!20 GB Bi- directional (Up to 40Gb)
!Multi Mode fiber optic cable
Copper compared to Fiber
!Limited lengths, max 20m
!Large Bend radius
!28-26 AWG Copper cable
!Max. 10 GB Bi- directional
Longer lengthsare
not possible due to
Noise and Crosstalk
!28-26 AWG Copper cable
August, 2008page 36 /Tyco Electronics Confidential and Proprietary
Product Overview
� Products� DDR4 - 4 Channels of 5 Gb/s
� 20 Gb/s Up to 200 Meter Reach
� QDR4 - 4 Channels of 10 Gb/s� 40 Gb/s up to 120 Meter Reach
� Next steps to develop� Product DDR12 -12 Ch 5 Gb/s (60 Gb/s) Up to 120 Meter Reach� Product QDR12 � 12 Ch 10 Gb/s (120 Gb/s) Up to 120 Meter Reach
Data Centers Solutions
How to combine DC cabling infrastructure requirements with today�s and tomorrow�s
applications ?
August, 2008page 38 /Tyco Electronics Confidential and Proprietary
Recall: Passive Cabling Requirements for Data Centers
� Easy deployment (weekend work/testing)� High performance� Structured scalability� Fast & easy MAC�s� Functional Security (less human errors)� High density� Effective cable management� Availability (less planning, less downtime)� Cooling support
MRJ21 MPO 1�
August, 2008page 39 /Tyco Electronics Confidential and Proprietary
Why High Density Technology?
� Eco-Friendly� Assists in efficient power use at the switch� Less material, thinner cable diameters (better airflow)� Assists in efficient power use in HVAC� Reusable
� Reduces need for expensive abandoned cable removal
� Modular� Speeds design & layout� Speeds installation / availability� Speeds training� Speeds MACs� Speeds recovery
August, 2008page 40 /Tyco Electronics Confidential and Proprietary
High Density Pre-terminated Solutions
� Pre-terminated MRJ21 Copper System
� High Density with 6 GBE Channels per
cable
� Pre-terminated MPO Fiber System
� High Density with 12 or 72 fibers per
cable
MRJ21 cable is 30% smaller, 20% lighter than 6 4-pair cables
August, 2008page 41 /Tyco Electronics Confidential and Proprietary
Current GbE application benefits
� Density, modularity & performance on equipment and in infrastructure
� MRJ21� provides Hi-D Green GbE � Connector: 1/3 -1/4 RJ45 cassette
� Cable: 30% smaller, 20% lighter
� 1 cable vs. wrapped 6-cable bundle
� Eco-Friendly
� All components permanent link tested,
documented and serialized
� 7-27% less power than RJ45 @ 2x ports
� 10G Shielded Version in process
1.14�x.74�6 GbE ports
MRJ21�MPO
Euro
August, 2008page 42 /Tyco Electronics Confidential and Proprietary
High-density Pre-terminated Copper
� The MRJ21� 24-pair Connector� 1�x 5/8� Footprint� Being adopted directly on electronics
12~399 in2RJ-45
8~266 in2RJ-45 (High Density)
6~200 in2MRJ21
Rack UnitsArea288 4-pair Ports
1 MRJ21 6 RJ45s
August, 2008page 43 /Tyco Electronics Confidential and Proprietary
MRJ21 Energy Efficiency
4.88 Watts234 Watts481 GbE48 port MRJ21
5.25 Watts126 Watts241 GbE24 port RJ 45
3.56 Watts320 Watts901 GbE90 port MRJ21
4.75 Watts190 Watts401 GbE40 port RJ 45
Total power
per port
Max power
per card
# of
ports
Appl.Card
configuration
Combines high density and ease of maintenance with energy savings
August, 2008page 44 /Tyco Electronics Confidential and Proprietary
Network Vendors with MRJ21 Equipment
� Alcatel:� OmniSwitch� Alcatel TiMetra� Lucent / Riverstone
� Barco Xenia� EIT� EntriSphere � Foundry Networks� Force10 Networks � Fujitsu � IBM � Motorola � NMS Communications � Proworks
� Shanghai Electronics� Stratex Networks� Tellabs / Vivace� UT Starcom
Force10 E120090 GbE ports/card
Foundry BigIron RX-4,8, 16, & 3248 GbE ports/card
Note: The vendors listed and shown have publicly launched MRJ21 equipment into the marketplace. Those not listed should be contacted directly for program status updates.
IBM eServerBladeCenter Copper Pass-
thru module 15 GbE ports
Alcatel OmniSwitchNI48 card
August, 2008page 45 /Tyco Electronics Confidential and Proprietary
Content of One 7-foot Rack (40 RUs)
~$0.70
~$1.56
~$1.40
~$3.12
~$2.80
~$6.25
Op Cost per
Port (year)
1920960480Cu Ports per
Rack
0 RUs20 RUs20 RUsCable Mgt.
per Rack
484824# Copper
Ports per RU
AngledHigh-densityTraditional
low HIGH
9 sq. ft. Traditional Room � Avg $1,350/yr ops. cost20 sq. ft. Hot/Cold Aisle � Avg $3,000/yr ops. cost
August, 2008page 46 /Tyco Electronics Confidential and Proprietary
Content of One 7-foot Rack (40 RUs)
~$0.70
~$1.56
~$1.40
~$3.12
~$2.80
~$6.25
Op Cost per
Port (year)
1920960480Cu Ports per
Rack
0 RUs20 RUs20 RUsCable Mgt.
per Rack
484824# Copper
Ports per RU
AngledHigh-densityTraditional
low HIGH
9 sq. ft. Traditional Room � Avg $1,350/yr ops. cost20 sq. ft. Hot/Cold Aisle � Avg $3,000/yr ops. cost
August, 2008page 47 /Tyco Electronics Confidential and Proprietary
Content of One 7-foot Rack (40 RUs)
~$0.70
~$1.56
~$1.40
~$3.12
~$2.80
~$6.25
Op Cost per
Port (year)
1920960480Cu Ports per
Rack
0 RUs20 RUs20 RUsCable Mgt.
per Rack
484824# Copper
Ports per RU
AngledHigh-densityTraditional
low HIGH
9 sq. ft. Traditional Room � Avg $1,350/yr ops. cost20 sq. ft. Hot/Cold Aisle � Avg $3,000/yr ops. cost
August, 2008page 48 /Tyco Electronics Confidential and Proprietary
Installation Time of One 7-foot Rack (40 RUs)
256 Hours128 Hours64 hoursInstall Time per
Rack � RJ45
2 minsAvg. Install
Time � MRJ21
10 hours,
40 mins
5 hours,
20 mins
2 hours,
40 mins
Install Time per
Rack � MRJ21
8 minsAvg. Install
Time - RJ45
AngledHigh-densityTraditional
Cable prep, termination, dress, insert & label
August, 2008page 49 /Tyco Electronics Confidential and Proprietary
Installation Time of One 7-foot Rack (40 RUs)
256 Hours128 Hours64 hoursInstall Time per
Rack � RJ45
2 minsAvg. Install
Time � MRJ21
10 hours,
40 mins
5 hours,
20 mins
2 hours,
40 mins
Install Time per
Rack � MRJ21
8 minsAvg. Install
Time - RJ45
AngledHigh-densityTraditional
Cable prep, termination, dress, insert & label
August, 2008page 50 /Tyco Electronics Confidential and Proprietary
Large Job (960 Ports) Cost Model Summary
~10 days less (E/P)~2 days less (NE/NP)
88% lessTotal
Time
~10% less (E/P)+15% more (NE/NP)
27% lessTotal
Cost
Fiber
12F MPO to Small Form
Factor
Copper
MRJ21 vs. RJ45
Product, placing cables, cable preparation, termination
The material referenced on this slide is for comparison purposes only and not for budgetary purposes. Actual costs may differ and may depend on project size, location or purchasing practice as well as other variables.
August, 2008page 51 /Tyco Electronics Confidential and Proprietary
High-density Pre-terminated Fiber
� The MPO� FOCIS 5, published in 1997
� 12 fibers in one connector� Being adopted directly on electronics
2~66 in212-fiber MPO Fiber
24~798 in2Duplex Fiber
12~399 in2Duplex Fiber (SFF)
<0.25~2 in272-fiber Para-Optix� MPO
Rack UnitsArea288 Duplex Ports
1 MPO
6 SC Duplex
1 Para-Optix
36 LC Duplex
August, 2008page 52 /Tyco Electronics Confidential and Proprietary
What is the value of time?
0 2 4 6 8 10 12 14 16
On-Site Person Hours
3) Para-Optix 36 DuplexPorts 1U
2) MPO Cassettes 1u (6x12Cable)
1) Field-terminated NE/NPDuplex LC
Installation Time 144 Fibers (72 OM3 fibers / end)
Time (Hrs)
� Installation Time� Uptime� Recovery Time
August, 2008page 54 /Tyco Electronics Confidential and Proprietary
Size Matters�
� How does cable size really affect your pathway costs?� Conduit fill � based on 40% fill ratio� 40% more cables in a conduit
� Patch cord comparison � F/UTP compared to U/UTP
Conduit F/UTP 6A UTPSize 7.6mm (.300") 9.0 mm (0.354")
2 18 133 40 294 71 51
August, 2008page 55 /Tyco Electronics Confidential and Proprietary
10 Gigabit per Second � Space (Air Handling � Cooling � Power)
ShieldSpaceMitigation
TechniquesANEXT
Reduction
57
.25 inch
48
Cat 6 U/UTP
42
(15 < Cat 6)
28
(29 < Cat 6)3� Conduit Cable Fill
0.290 inch
(Up to 16% > Cat 6)
≤ 0.354 inch
(Up to 41% > Cat 6)Cable OD
4816 � 24Port Density
(1 RU)
Cat 6aF/UTP
Cat 6aU/UTP
August, 2008page 56 /Tyco Electronics Confidential and Proprietary
Cable Management
� Does this look familiar?
� Unmanaged and unstructured connections
� Poor documentation
� 30-40% abandoned patch cords after 1 year of operation
� Dead ports in switches and cable runs 30-40% unused capacity
� MAC time > 1h per port
� Adds up to 150% of operational cost
� Adds up to 200% of floorspace needed
� Residual cable mass blocks airflow in rack and under floor
August, 2008page 57 /Tyco Electronics Confidential and Proprietary
Recall: Passive Cabling Requirements
� Easy deployment (weekend work/testing)
� Structured scalability
� Fast & easy MAC�s
� Functional Security
(less human errors)
� High density
� Effective cable management
� Availability
(less planning, less downtime)
� Cooling support
August, 2008page 58 /Tyco Electronics Confidential and Proprietary
AMP Hi-D technology
� Patented Corner Brackets combined with Angled Patch Panels� Eliminates the need for horizontal cable managers� Uses unused valuable cabinet space - increase of cabinet density� Toolless cable management maintenance � increase of operational
speed
August, 2008page 61 /Tyco Electronics Confidential and Proprietary
Down Time Is A Cost Driver
Real-time Overview & Documentation are KeyIn many industries data network down time is business critical! Depending on the
importance of the network as a daily business tool, hourly costs can be extremely high (e.g. in banking and insurance, airports, ISP's, etc)
� �72% of all business critical systems experience9 hours of down time each year!�(The Standish Group)
� �There�s an average loss of $90,000 per hourof down time for business critical systems!�(Contingency Planning Research)
� �59% of network problems are directly relatedto the physical infrastructure and its connections!�(The Gartner Group)
� �70% of all network failures are attributableto network cabling!� (LAN Technology)
Enterprise Downtime Cost/hr
Error Recovery without IIM
August, 2008page 62 /Tyco Electronics Confidential and Proprietary
IntroductionRequirements of today�s rapidly growing and changing enterprise
infrastructures are no longer fully satisfied by traditional process-based IT infrastructure management but now need intelligent, real-time management.
The main deficiencies of traditional IT Infrastructure management are:
� Maintenance of connectivity records is time consuming & expensive� Inaccurate records and long cycle times� Difficulty in managing and tracking work and change orders� Possible & likely deterioration in infrastructure flexibility, effectiveness
and response times
The AMP NETCONNECT AMPTRAC Intelligent Infrastructure Management System (IIMS) has been designed to address the limitations described above.
August, 2008page 63 /Tyco Electronics Confidential and Proprietary
Many department representatives, including:� Help desk� Planning / Facilities� Network Administration / IT� Supervision / Management� Technicians
And provides real-time accountability and tracking of:� network utilization (Cable Plant and Network Electronics)� budgeted costs of MACs (planned and actual)� device access to network (Authorized and Unauthorized)� circuit connectivity and routing (graphical and textual)� network device asset information (user definable)
AMPTRAC� Supports
August, 2008page 64 /Tyco Electronics Confidential and Proprietary
Intelligence is good in my Infrastructure
Or Simply said, Let the AMPTRAC IIMS
turn this,Into This