All-Optical Networks for Grids: Dream or Reality?
description
Transcript of All-Optical Networks for Grids: Dream or Reality?
All-Optical Networks for Grids:
Dream or Reality?Payam Torab
Lambda Optical Systems CorporationSeptember 28, 2005
www.lambdaopticalsystems.com
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
2
Grids – Tflops vs. Tbps. Emergence of grids is the result of the synergism between
communications and computing, just like cybernetic systems that came out of synergism between communications and control
Role of the network in Grids: to provide throughput– Application-aware networks, or network-aware applications?– Network providing services, or network as a services?– Throughput is the theme unifying connectivity, delay and
bandwidth Balanced growth of networking and computing results in Grids
Networking power (Tbps)
Com
pu
tin
g
pow
er
(Tfl
op
s)
Clusters
Internets
Grids North European Grid
NEESgrid
TeraGrid
Surfnet
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
3
Need for High Throughput Throughput is a grid resource: Uniform
grid growth requires growth in throughput Throughput growth requires improvement
in bandwidth, delay and availability Examples of throughput requirements
– GridFTP applications– Large Hadron Collider (LHC) at CERN
Brookhaven National LabLong Island, NY
OC-48 link to ESNET
ESNET outage?
Source: www.cerncourier.com/main/article/45/7/15
Year Production Experimental Remarks
2001 0.155 0.622-2.5 SONET/SDH
2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.
2003 2.5 10 DWDM; 1 + 10 GigE Integration
2005 10 2-4 X 10 Switch; Provisioning
2007 2-4 X 10 ~10 X 10; 40 Gbps
1st Gen. Grids
2009 ~10 X 10 or 1-2 X 40
~5 X 40 or ~20-50 X 10
40 Gbps Switching
2011 ~5 X 40 or
~20 X 10
~25 X 40 or ~100 X 10
2nd Gen Grids Terabit Networks
2013 ~Terabit ~MultiTbps ~Fill One Fiber
PHENIX experiment – Used GridFTP to transfer 270 TB of data from Long
Island, NY to Japan
Source: Larry Smarr, “The Optiputer - Toward a Terabit LAN ,” The On*VECTOR Terabit LAN Workshop Hosted by Calit2,University of California, San Diego - January 2005
Transpacific 10 Gbps line to
SINET in Japan
Relativistic Heavy Ion Collider RHIC at Brookhaven:600 Mbps peak250 Mbps average
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
4
Photonic Switching: Key to End-to-End Transparency
WDM + Photonic switching– End-to-end transparency
• Bitrate transparency (10 Gbps, 40 Gbps, …)• Payload transparency (SONET, SDH, Ethernet, …)
– Transmission robustness• Simplification or even elimination of windowing• No packet loss due to congestion/buffer overrun• Simpler transport protocols, higher throughput
From: “Development of a Large-scale 3D MEMS Optical Switch Module,” T. Yamamoto, J. Yamaguchi and R. Sawada, NTT Technical Review, Vol. 1, No. 7, Oct. 2003
Electrical Cross-
Connect (EXC)
Photonic Cross-
Connect (PXC)
Photonic Cross-
Connect (PXC)WDM and electrical
switchingSeparate WDM and
optical switchingIntegrated WDM
and optical switching
~O(102) waveleng
ths
O-E-O O-E-O
Full transparency
O-O-O
~O(102) wavelength
s~O(102) Gbps per
wavelength
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
5
Wavelength Switching Scalability Grid-scale applications will ultimately press
even wavelength switching – Example:
Year Production Experimental Remarks
2001 0.155 0.622-2.5 SONET/SDH
2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.
2003 2.5 10 DWDM; 1 + 10 GigE Integration
2005 10 2-4 X 10 Switch; Provisioning
2007 2-4 X 10 ~10 X 10; 40 Gbps
1st Gen. Grids
2009 ~10 X 10 or 1-2 X 40
~5 X 40 or ~20-50 X 10
40 Gbps Switching
2011 ~5 X 40 or
~20 X 10
~25 X 40 or ~100 X 10
2nd Gen Grids Terabit Networks
2013 ~Terabit ~MultiTbps ~Fill One Fiber
Source: Larry Smarr, “The Optiputer - Toward a Terabit LAN ,” The On*VECTOR Terabit LAN Workshop Hosted by Calit2,University of California, San Diego - January 2005
Require too many
optical ports to provide
non-blocking
connectivity!
Similar to any other switching technology, aggregation is essential for scalability of wavelength switching – hence the emergence of transparent multigranular (wavelength and waveband) switching architectures
PXC PXC PXCPXC
Wavelength switching4 wavelengths over 4 hops 32 optical ports
PXC PXC PXCPXC
Waveband
multiplexer
Waveband demultiplex
er
Waveband switching4 wavelengths over 4 hops 8 optical ports
From: “A Graph Model for Dynamic Waveband Switching in WDM Mesh Networks,” M. Li and B. Ramamurthy, IEEE ICC 2004, Vol. 3, June 2004, pp. 1821-1825.
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
6
Waveband Switching Efficiency
buh
h
n
ne
w
b 1
1
11
0 2 4 6 8 100
2
4-100
-80
-60
-40
-20
0
20
40
60
Physical hops in waveband
path (h)
Waveband-
switched circuits (bu)
Sw
itch
ing
eff
icie
ncy
(%)
Waveband switching efficiency: Relative saving in the total number of optical ports in a network when waveband switching is used instead of wavelength switching
nw = number of ports under wavelength switching
nb = number of ports under waveband switching
h = average number of physical hops in each wavebandb = average number of wavelengths in a wavebandu = average waveband utilization (used wavelengths)
1 2 3 4 5 6 7 8 9 101
2
3
4
Wave
ban
d-s
wit
ch
ed
cir
cu
its
(bu
)
Waveband-switching
efficient region
Physical hops in waveband path (h)
Increased
waveband
utilization
Increased waveband
path length (hops)
Waveband
switching gets more
efficient
Waveband switching becomes only more efficient (more saving in optical ports) as more wavelength circuits are carried over longer paths
Example: GridFTP using 4 parallel TCP streams over 4x40 Gbps circuits carried over 6 hops More than 0.1 Tbps throughput over 6 hops using only 30 ports
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
7
2085 circuits@40Gbps ~ 85 Tbps~28400 ports - wavelength switching~21800 ports - waveband switching
998 circuits@40Gbps ~ 40 Tbps~11800 ports - wavelength switching~9700 ports - waveband switching
615 circuits@40Gbps ~ 2.5 Tbps~6200 ports - wavelength switching~5500 ports - waveband switching
5000
10000
15000
20000
25000
30000
0 20 40 60 80 100Network throughput (Tbps)
Req
uired
optica
l por
ts
Wavelength-switching
Waveband-switching
More on Waveband Switching Efficiency
Example: WDM WAN ~80 nodes, ~140 links This simple analysis
does not consider the extra scalability from the increase in bitrate (160Gbps and beyond, OTDM).
1 2 3 4 5 6 7 8 9 101
2
3
4
Wave
ban
d-s
wit
ch
ed
cir
cu
its
(bu
)
Waveband-switching
efficient region
Physical hops in waveband path (h)
Waveband
switching gets more
efficient
2.5 Tbps
40 Tbps 80 TbpsTransmission breakthrough
s Increase in throughput
without increase in
ports
More to appear in:P. Torab and V. Hutcheon, “Waveband switching efficiency in all-optical networks: analysis and case study,” in preparation for OFC 2006.
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
8
Hierarchical Transparent Switching Waveband switching adds another level of
switching to the transparent switching hierarchy Multigranular switching Logical WDM
topologies
Wavelength XC
Wavelength Interfaces
Waveband XC
Wavelength XC
Waveband XC
Wavelength Interfaces
Waveband XC
1 2 h
h physical hops – one logical hop
Waveband Multiplexer
Bandpathbp1
bp1
Waveband Demultiplexer
Two lightpaths with the same routes
Node A Node B
Node A Node B
Wavelength XC
Wavelength Interfaces
Waveband XC
Wavelength XC
Waveband XC
Wavelength Interfaces
Waveband XC
1 2 h1
h1 physical hops – one logical hop
Bandpathbp1
bp1
Node A Node B
Node A Node B
Wavelength XC
Waveband XC
Wavelength Interfaces
Waveband XC
1 2 h2
Bandpathbp2
Waveband Demultiplexer
Node C
Waveband Multiplexer
h2 physical hops – one logical hop
Node C
bp2
lp1 lp2Two lightpaths with partially overlapping routes
FiberWavebandWavelengt
hIP/TDM
Payload-transpar
ent Switchin
g
Several physical hops are lumped into one logical WDM link, requiring switching only at the link endpoints Fast and still flexible dynamic wavelength service over reduced number of hops
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
9
Logical (Virtual) WDM Combined wavelength and waveband switching allows dynamic
configuration of transparent optical topologies supporting dynamic lambdas (from connection on-demand to topology on-demand)
Example: During the next 14 days, computing facility at site A, the storage center at site B, and the visualization room at site C will participate in an experiment that will require multiple dynamic lambdas (e.g., timescale in seconds)
Computing - AStorage - B
Visualization - C
Computing - AStorage - B
Visualization - CDynamic lambdas(fast setup and
teardown)
Logical WDM Topology
Waveband
connections
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
10
Lambda OpticalSystems Solutions Dedicated to transparent switching
technology
Addressing research community and carrier needs
Deployed at U.S. Naval Research Lab (NRL) and Starlight
LambdaNode 200Transparent 64x64 full duplex portsGMPLS, CLI and web interface5.25 inches tall
LambdaNode 2000Integrated WDM and photonic switchingMultigranular switching for maximum scalabilityProvides waveband and wavelength switchingGMPLS, CLI, TL1 and web interface
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
11
NL 101, NL 103 Demos at iGrid 2005
vangogh 6
E600
nud05
x y
a b
iGRID A iGRID B
HDXc
2 x GbE
circuits
Qwest / other wave service
Qwest / other wave service
CENICCENIC
2 x GbE
circuits
VMT Controller
San Diego/UCSD (SAN) Chicago/SL (CHI) Amsterdam/NL (AMS)
**E600
nud06
**or other L2 switch
AAA/DRAC AAA/DRACAAA/DRAC
GbEOC192STM64
vangogh 5
VMT visualization host
HDXc
/2
vm vmvm
vh
4003(2)
HDXc
X /2
X
LambdaNode200
2/12 2/13
2/18 2/19
E120012/2 12/3
VLAN 350 VLAN 350
OME1 3
2 4
5 6
GbEOC192STM64
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
12
Control Plane: Enabler of Transient Services
Grid’s balanced growth needs dynamic on-demand high network throughput
What do we need to provide high throughput?1. Dynamism: Make optimum use of all network resources for the tasks at
hand• Example: If 1.0 Tbps throughput is needed between A and B for one hour, fill up
the network with 25x40Gbps connections and kill them an hour later.
2. Availability: The ability to maintain high throughput through fast recovery• Network failures do happen, therefore high bandwidth does not guarantee high
throughput• In a transient service environment protection is not as expensive
– Telco thinking: 1+1 protection is expensive- I need to plan for twice the capacity, therefore I need to charge my customer twice as much (bronze service, silver service, platinum service, …)
– Grid thinking: Provide as much protection that your schedule allows. The connections will not be there in an hour. The more network resources the more protected circuits.
• (Dynamic) restoration can also add to reliability when (dedicated) protection is unavailable
Key effort needed: Integrating traditional service levels (1+1 protection, 1:N protection, shared mesh restoration, …) into Grid services– Can a GridFTP application ask for transfer over 1+1 connection?– Trade-off between replication/migration and network recovery– Where does the optimal performance stand?
Application
intelligence
(replication,
migration)
Network intellige
nce (protecti
on, restorati
on)
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
13
IP-based control plane paradigm to control packet, time slot (TDM), wavelength, waveband and space (fiber) switching across multiple switching layers, and across multiple domains.
Developed by IETF – CCAMP workgroup with liaison work with OIF and ITU-T
Mature standard now (RFC 3945) with various extensions for different switching technologies (Layer 2, wavelength/waveband, SONET/SDH,…)
Basic functionalities/protocols– Neighbor discovery/link management (Link Management Protocol -
LMP)– Routing with traffic engineering extensions (OSPF-TE, ISIS-TE)– Signaling (RSVP-TE with GMPLS extensions)
Applications/solutions– Recovery (protection, restoration)– Make-before-break– Layer 1 VPN (L1VPN working group)
Generalized Multiprotocol Label Switching (GMPLS)
Cross-connect set upon receiving the PATH message
Bidirectional LSP
PATH PATH
RESV RESVIngress Node A
Transit Node B
Egress Node C
Bidirectional data plane
Cross-connect set upon receiving the RESV message
Both cross-connects set upon receiving the PATH message
Bidirectional data plane
RFC 3473 bidirectional
LSP setup
More efficient bidirectional
LSP setup
PATH
PATH
RESV
RESVCONF (optional)
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
14
Generalized Multiprotocol Label Switching (GMPLS)
New directions– Separation of path computation as a service– Attention to Ethernet as a Layer2 transport– Inter-domain traffic-engineering
• Good work at NSF’s DRAGON project– Inter-domain circuit setup, path computation element (Network Aware
Resource Broker –NARB)– The next step is interoperability with other networks
NARB
End Syste
m
NARB
NARB
End System
AS 1
AS 2
AS 3
Transport Layer Capability Set Exchange
Source: Jerry Sobieski, Tom Lehman, Bijan Jabbari, “Dynamic Resource Allocation via GMPLS Optical Networks (DRAGON),” Presented to the NASA Optical Network Technologies Workshop, August 8, 2004
Enabling Data-Intensive Grid Applications with Advanced Optical Technologies - 9/28/2005
15
Key word for Grid networks is high throughput Lambda Grids are the only way to keep up with throughput demand
– Reality When is access to dark fiber going to be cheap? Dream
– Starting as islands of transparency• Regional Optical Networks (RONs)• Fiber sharing is critical, RONs have to have transparent access to each other• Wavebands as highways between RONs
– Islands growing as optical reach/transmission improves• Digital wrapper, FEC
High throughput needs end-to-end transparency– Data plane transparency
• WDM and photonic switching
– Control plane transparency• Inter-domain end-to-end circuit setup
Availability and recovery are the new QoS for lambda grids Ethernet will be the dominant end-to-end payload
– Transparent networks are ready for payload change
Conclusions: Dream or Reality?
Photonic access to super highway for
RONs?
HOPI Node