Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data...
Transcript of Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data...
![Page 1: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/1.jpg)
Updating TIA & ISO data centerstandards to reflect industry
changes
Jonathan JewJonathan JewJ&M Consultants, Inc.
Co-editor TIA-942-A Data Centers Telecom InfrastructureCo-chair BICSI Data Center Subcommittee
Vice-chair TIA TR-42.6 Telecom AdministrationVice-chair TIA TR-42.1 Commercial Buildings
www.j-and-m.com
![Page 2: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/2.jpg)
What are the standards?
• ANSI/TIA-942-A Telecommunications Infrastructure Standardfor Data Centers (US, approved April 2012, publish in Aug 2012)
• ISO/IEC 24764 Information Technology – Generic Cabling forData Centre Premises (2010, international). Addendum 1 willbe published in 2012be published in 2012
• ANSI/BICSI-002 Data Center Design and Implementation BestPractices (2011, international)
• NMX- I- xxx-ONNCE Energy Efficient & Sustainable HighPerformance Data Centers (México, in progress probable 2012publication)
![Page 3: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/3.jpg)
Fit TIA-942 into new TIA cablingstandards structure
• In TIA-568-C series, cabling standards werereorganized to permit premises specificstandards to be developed withoutstandards to be developed withoutduplication of content.
• TIA-942-A was rewritten to fit into the newstructure
![Page 4: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/4.jpg)
Reorganization of TIA standards
![Page 5: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/5.jpg)
Reorganization of content• Reference generic cabling topology, terms, and MICE
(mechanical, ingress, climatic, electromagnetic)environmental classifications from TIA-568-C.0
• Move bonding & grounding content to TIA-607-B
• Move administration & labeling to TIA-606-B• Move administration & labeling to TIA-606-B
• Move racks & cabinets, power and telecomseparation, and temperature/humidity requirementsto TIA-569-C
• Move outside plant pathways to TIA-758-B
• ISO/IEC 11801 series also being reorganized
![Page 6: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/6.jpg)
Reorganization of ISO/IEC standards
![Page 7: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/7.jpg)
TIA Harmonization with ISO• LC connector for up to 2 fibers
(ISO/IEC specifies angled LC for SM at theExternal Network Interface)
• MPO connector for more than two fibers
LC
• MPO connector for more than two fibers
• Removed 100m channel length limitationfor horizontal cabling of fiber
Now limited by application lengthrestrictions of the type of fiber used
MPO
![Page 8: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/8.jpg)
Higher bandwidths• Specify higher bandwidth cabling types to
support
– Higher performance systems and applications
– Higher performance networks (switch fabrics)
– LAN & SAN convergence– LAN & SAN convergence
![Page 9: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/9.jpg)
Benefits of LAN/SAN Convergence• Reduce number of server connections
• Allow use of small servers (blade & 1U)that can’t support large number ofadapters
• Reduce cost and administration (fewer• Reduce cost and administration (feweradapters, fewer switches, less cabling )
• Simplify support – Ethernet only (noseparate Fibre Channel infrastructure)
• But it requires high bandwidth and lowlatency data center LAN
![Page 10: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/10.jpg)
TIA-942-A Higher Bandwidth Copper• Retained 734/735 coax cable for T-3/E-3
• Removed support for Category 3 and 5e forhorizontal cabling, but retained for backbonecabling (WAN, voice, console)
• Category 6 min for horizontal cabling• Category 6 min for horizontal cabling
• Category 6A or higher is recommended (e.g.Category 7A from ISO standards)
• Category 6A is the minimum in ISO/IEC 24764
734 coax Cat 7A Cat 6A
![Page 11: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/11.jpg)
10GBase-T• To support 10GBaseT, Category 6A or better
recommended in TIA-942-A and minimum inISO/IEC 24754
• 10GBase-T will be widely adopted in 2012 andis predicted to be the most widely shippedis predicted to be the most widely shippedversion of 10G Ethernet in 2014
![Page 12: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/12.jpg)
Factors in adoption of 10Base-T• Rapidly declining cost
• Improved power efficiency, now <4W, lesswith short-reach mode
• Auto-negotiation - backward-compatible with100M & 1G Ethernet100M & 1G Ethernet
• Familiar balanced-pair cabling & RJ45 thattechnicians know how to troubleshoot andterminate
• Flexible configuration and lengths: 2 to 4connectors, up to 100m
![Page 13: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/13.jpg)
Next Generation Cabling
• TIA & ISO developing next generationbalanced twisted-pair cabling standards
• >= 40Gbps (for 40GBase-T)
• Length target to be determined – probably• Length target to be determined – probably40m to 60m
• No corresponding IEEE project yet, butmembers from IEEE 802.3 are involved
![Page 14: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/14.jpg)
Higher bandwidth optical fiber
• Removed OM1 and OM2 (62.5 µm and non-laser-optimized 50/125 µm multimode fiber)
• OM3 (50/125 µm laser optimized) is theOM3 (50/125 µm laser optimized) is theminimum requirement
• Higher bandwidth OM4 is recommended
![Page 15: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/15.jpg)
Ethernet channel lengths over multimode fiber
FiberType
1G 10G 40G 100G 100G
# fibers 2 2 8 20 8 (possibly 2 with
WDM)
OM1 275 m 26 m - - -OM1
OM2 550 m 82 m - - -
OM3 800 m 300 m 100 m 100 m 20-50m*
OM4 1040 m 550 m 150 m 150 m 70-150 m*
Distances in red are specified by manufacturers but not in IEEEstandards. *Length targets for 100G over 4-lanes are not yetspecified.
![Page 16: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/16.jpg)
40G / 100G over Optical Fiber
• Multimode fiber more cost effective thansingle-mode for lengths <=150 m
• 40G uses 8 MM fibers and MPO
• 100G uses 20 MM fibers and either two 12• 100G uses 20 MM fibers and either two 12fiber MPOs or one 24 fiber MPO
• 4-lane/8-fiber 100G will probably cost less andsupport at least 100m over OM4 with OM3reach 20-50m or left for future definition
![Page 17: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/17.jpg)
Single-mode fiber• Single-mode needed for:
– Delivery of circuits from carriers
– Campus cabling
– Backbone for large data centers
• Backbone cable lengths >150m will need singlemode fiber for 40/100G Ethernetmode fiber for 40/100G Ethernet
• 2 SM fibers for current standard (10/40km)
• 4-lane100G SMF standard in development(500m - 2 km, 8 fibers or 2 fibers with WDM)
• Future 400G/1000G (Terabit Ethernet)
![Page 18: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/18.jpg)
TIA-942-A Energy Efficiency• New section on energy efficiency
• Wider range of temperaturesand humidity (see TIA-568-C)based on 2011 ASHRAE TC 9.9guidelinesguidelines
• Other guidelines for energyefficiency relating to cabling,pathways, and spaces
![Page 19: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/19.jpg)
Energy Efficiency• New 2011 ASHRAE guidelines being
considered (ANSI/TIA-569-C-1)
– Temperature: 15 – 35 oC (59 – 95 oF)
– Relative humidity (RH): 20 – 80%
• ESD could be a problem with low humidity• ESD could be a problem with low humidity(<44% at 18 oC, <25% at 27 oC)
• These are temperatures in the cold-aisles. Inhot aisles temperatures could be 55 oC
• Attenuation of balanced pair cables increasesat high temperatures
![Page 20: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/20.jpg)
Energy Efficiency• Overhead cabling
if under floorspace is used forcooling
• Route cables and• Route cables andpathways tominimizeinterfering withproper airflow
![Page 21: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/21.jpg)
Energy Efficiency• Avoid exterior windows and sky lights because
they create additional heat load
Beautiful, but not a good building for a data center
![Page 22: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/22.jpg)
Energy Efficiency• Cable routing should not
compromise efficiency ofenclosures - blankingpanels, brushes &grommets for cableopeningsopenings
• Equipment should matchairflow design of cabinets(front to back, use bafflesfor side-to-side cooledequipment)
![Page 23: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/23.jpg)
Energy Efficiency• Enclosures or enclosure systems:
– cabinets with isolated supply or return
– cabinet cooling system
– hot or cold aisle containment
Cabinets withIsolated return
Aisle-Containment
![Page 24: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/24.jpg)
Energy Efficiency• Separate areas for equipment with high power
density needs
• Build in phases (modular construction)
![Page 25: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/25.jpg)
Energy Efficiency - Lighting• Energy efficient lighting such as LED
• 3 Level lighting protocol by zone for data centersdepending on human occupancy
– Level 1 – no occupants - enough light for cameras
– Level 2 – motion to initiate higher level access toprovide safe passage & ID by camerasprovide safe passage & ID by cameras
– Level 3 – occupied for work – full lighting 500 lux
![Page 26: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/26.jpg)
ISO/IEC JTC 1 SC 39• New ISO/IEC JTC 1 Subcommittee 39
Sustainability for & by Information Technology
• First working group will address energyefficient data centers
– Terminology– Terminology
– Metrics
– Best practices
– Energy management system
• 1st meeting Jun 12th -14th in Redwood City,California
![Page 27: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/27.jpg)
Larger and modular data centers• New TIA-942 space Intermediate Distribution
Area (IDA) containing the Intermediate Cross-Connect (IC)
• Adopted in ISO/IEC 24764 Addendum 1 asIntermediate Distributor (ID)Intermediate Distributor (ID)
• Elimination of TIA requirement thatcentralized optical fiber topologies be limitedto one building to accommodate modular datacenters using outdoor containers & modules
![Page 28: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/28.jpg)
IDA Example
![Page 29: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/29.jpg)
Modular data centers• Outdoor using containers/modules
• Indoor using manufactured modules or bysimply building rooms/areas in phases
• Reduce capital expenditures (Capex) andoperating expenditures (Opex) by building onlywhat you need when you need it.what you need when you need it.
• Do not need to pay maintenance andpower/cool equipment and rooms that aren’tneeded yet.
• Modularity reduces the risk created by anuncertainty about the future
![Page 30: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/30.jpg)
Data center fabrics• TIA-942-A addendum 1 being developed to
provide guidance regarding cabling for datacenter fabrics
• No common definition or implementation
• Generally, low-latency switch fabric that is• Generally, low-latency switch fabric that isnon-blocking, permitting any port tocommunicate with any other port at the fullcapacity of the port
• Virtualization and LAN/SAN convergence aremajor driving factors
![Page 31: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/31.jpg)
Traditional switch architecture
• Backup connections inactive due to spanning tree protocols• Connections between switches are over-subscribed• As many as 3 hops between servers
![Page 32: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/32.jpg)
Full-Mesh
• All switches connected to all other switches in full mesh• For non-blocking sum of backbone >= sum of switch bandwidth• All connections active• Not suitable for large # of switches
![Page 33: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/33.jpg)
Fat-Tree
• Each access switch connected to every interconnection switch• Non-blocking if sum backbone bandwidth >= sum server port bandwidth• All connections active• More scalable than full-mesh• Another layer of inter-connection switches can be added to supportvery large data centers
![Page 34: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/34.jpg)
Fat-Tree with Port Extenders
• Same as fat-tree, but with port extenders at top of racks• May be non-blocking if bandwidth from access switch toport extender >= sum of bandwidth of server ports
![Page 35: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/35.jpg)
Data center fabrics• Much more cabling and higher bandwidth
requirements for data center backbones
• Possible need for longer cable runs to connectswitches
• Fabrics can be accommodated using the• Fabrics can be accommodated using thecabling topology in ANSI/TIA-942-A
– May need IDA-to-IDA & HDA-to-HDA cabling
– Point-to-point cabling in EDAs should be less
than or equal 15 m and within a cabinet row
![Page 36: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/36.jpg)
Other convergence• Structured cabling is being used to support
other building systems and should beconsidered when planning data centers:
– IP cameras
– Security systems– Security systems
– Building automation/monitoring
– Possibly even lighting
• See BICSI Electronics Safety & Security DesignReference Manual & TIA-862-A
![Page 37: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/37.jpg)
Summary• Standards are being updated to support industry
changes and new technologies
• Consider future needs to support 10/40/100G tosupport LAN/SAN convergence, data center fabrics,virtualization
– Cat 6A or better (e.g. Cat 7/7A) for balanced-pair cable– Cat 6A or better (e.g. Cat 7/7A) for balanced-pair cable
– OM4 for multimode fiber (with LC & MPO connectors)
• Consider energy efficiency recommendations
• Build modularly (in phases)
• Consider the impact of data center fabrics
• Consider other systems (e.g. cameras & security) thatcan use structured cabling
![Page 38: Updating TIA & ISO data center standards to reflect ... · PDF fileUpdating TIA & ISO data center standards to reflect industry changes Jonathan Jew J&M Consultants, Inc. Co-editor](https://reader034.fdocuments.net/reader034/viewer/2022050902/5a7dddb37f8b9a49588dee93/html5/thumbnails/38.jpg)
Jonathan Jew
President J&M Consultants, Inc.
Website: www.j-and-m.com
Email: [email protected]
Co-chair BICSI data center subcommittee
Co-editor TIA-942-A
Vice-Chair TIA TR-42.6 telecom administration subcommittee
Vice-Chair TIA TR-42.1 commercial building cabling
Editor ISO/IEC TR 14763-2-1 telecom administration identifiers
US National Committee Project Manager ISO/IEC 24764 datacenter standard
Data Center & Administration Section Editor – ISO/IEC 14763-2cabling planning & installation