Post on 05-Jan-2016
Requirements
Mechanisms + Policies
API
Domain 1 Domain 2 Domain 32
BA
GNSA
LDG
1 3
GNSA GNSA
LDG
LDG
control plane control plane control plane
Ser
vice
pla
ne
ChicagoAmsterdam
Require fat unidirectional pipesTight QoS requirements (jitter, delay, data loss)
Simultaneous connectivity to multiple sitesMulti-domainDynamic connectivity hard to manageUnknown sequence of connections
Request to receive data from 1 hour up to 1 day Innovation for new bio-scienceArchitecture forces optimization of BW utilizationTradeoff between BW and storage
Network Issues
Personnel required at remote location
Remote instrument access (Radio-telescope)
N* previous scenarioAccess multiple remote DB
Copy from remote DB: Takes ~10 days (unpredictable)Store then copy/analyze
Point-to-Pointdata transfer of multi-TB data sets
Current MOPApplication Scenario
2 PB/YearComputational fluid dynamics
1 PB/yearMagnetic fusion
5 PB/YearPlasma physics
3 PB/YearNuclear physics
1 PB/yearSLAC (BaBar experiments)
10 PB/yearCERN LHC (Higgs boson search)
5 PB/yearRHIC (Quark-gluon plasma experiments)
<10 PB/yearCEBAF (Hadron structure experiments)
Estimated 2008 Data Generation
Processor
Performance
Traffic Growth
2x/12 months
x 16
20052010
2015
x 4
2020
x 32
2x/18 months
Data Transmission Plane
optical Control Plane
1 n
DB
1
n
1
n
Storage
Optical Control Network
Optical Control Network
Network Service Plane
Data Grid Service Plane
NRS
DTS
Compute
NMI
Scientific workflow
Apps Middleware
Resource managers
PRA
CreateReservationFromRequestSRA1
SRA2
SRA3
(ConstructProposals)
GetWindowsForRequest
ProposeReschedule
etc.
ConstructProposalForWindow
Fa
bric
UDP
ODINResources
Grid FTP
BIRN Mouse
Apps Middleware
TCP/HTTP
Grid
Layered
Arch
itecture
Lambda Data Grid
IP
Co
nn
ec
tivity
Ap
plic
atio
nR
es
ou
rce
Co
llab
ora
tive
BIRN Workflow
NMI
NRS
BIRN Toolkit
Lambda
Resource managers
DB
Storage Computation
Optical Control
WSRF
Optical protocols
Optical HW
OGSA
Availability: Abundant Optical Bandwidth
Requirements: Data-Intensive e-Science apps
Lambda Data Grid
Applications
Our emphasis
Grid work’s emphasis
Applications Middleware
Network Middleware
Network
• Terabit/s• 100Gb/s• 10Gb/s
1Gb/s
Fiber transmission
Edge computer limitations
Optical Control Network
Optical Control Network
Network Service Request
Data Transmission Plane
OmniNet Control PlaneODIN
UNI-N
ODIN
UNI-N
Connection Control
L3 router
L2 switch
Data storageswitch
DataPath
Control
DataPath Control
DATA GRID SERVICE PLANEDATA GRID SERVICE PLANE
1 n
1
n
1
n
DataPath
DataCenter
ServiceControl
ServiceControl
NETWORK SERVICE PLANENETWORK SERVICE PLANE
GRID Service Request
DataCenter
DWDM-RAM Service Control Architecture
Optical Control Network
Optical Control Network
Network Service Request
Data Transmission Plane
Optical Control PlaneOpticalControl
UNI-N
Optical Control
UNI-N
Connection Control
L3 router
L2 switch
Data storageswitch
DataPath
Control
DataPath Control
DATA GRID SERVICE PLANEDATA GRID SERVICE PLANE
1 n
1
n
1
n
DataPath
DataCenter
ServiceControl
ServiceControl
NETWORK SERVICE PLANENETWORK SERVICE PLANE
GRID Service Request
DataCenter
DWDM-RAM Service Control Architecture
From 100 Days to 100 Seconds
Lambda Data Grid - Globus Services
Fab
ric
SABUL
UDP
ODIN
OMNInetStorage Bricks
Grid FTP
GRAM
GSI
e-Science applications
Multidisciplinary Simulation
SOAP
TCP/HTTP
NRS
Storage Service
DTS
IP
Co
nn
ec
tivity
Ap
plicatio
nR
esou
rceC
olla
bo
rativ
e
GARA
Problem Solving Environment
Applications and Supporting Tools
Application Development Support
Common Grid
Services
LocalResources
Gri
d
Info
rmat
ion
S
ervi
ce
Un
ifo
rmR
eso
urc
eA
cces
s
Bro
keri
ng
Glo
bal
Q
ueu
ing
Glo
bal
Eve
nt
Ser
vice
s
Co
-S
ched
uli
ng
Dat
a C
atal
og
uin
g
Un
ifo
rm D
ata
Acc
ess
Co
mm
un
icat
ion
Se
rvic
es
Au
tho
riza
tio
n
Grid Security Infrastructure (authentication, proxy, secure transport)
Au
dit
ing
Fau
lt
Man
ag
emen
t
Mo
nit
ori
ng
Communication
ResourceManager
CPUs
ResourceManager
Tertiary Storage
ResourceManager
On-Line Storage
ResourceManager
Scientific Instruments
ResourceManager
Monitors
ResourceManager
Highspeed Data
Transport
ResourceManagernet QoS
Grid access (proxy authentication, authorization, initiation)
Grid task initiation
Collective Grid
Services
Fabric
Dat
a R
epli
cati
on
High performance computing and Processor memory co-allocationSecurity and Generic AAAOptical NetworkingResearched in other programlinesImported from the Globus toolkit
value
time
Window
value
time
Increasing value
time
Decreasing
value
time
Peak
value
time
Level
value
time
Asymptotic Increasing
value
time
Asymptotic Increasing
value
time
Step
Application Application
Services Services Services
data
control
data
control
Chicago Amsterdam
AAA
LDG LDGLDG
AAA AAA AAA
LDG
OMNInetOMNInetODIN Starligh
t
Starlight
Netherlight
Netherlight UvAUvA
ASTNASTNSNMPSNMP
Multi-EndPoint Communication
• Network Transfers Faster than Individual Machines– A Terabit flow? A 100Gbit flow? A 10Gbps flow w/ 1Gbps NIC’s
– Clusters are Cost-effective means to terminate Fast transfers
– Support Flexible, Robust, General N-to-M Communication– Manage Heterogeneity, Multiple Transfers, Data Accessibility
Uh-oh!
λData Receiver Data Source
FTP client FTP server
DMS NRM
Client App
Data Management Service
A
D
B
C
X
7:00-8:00
A
D
B
C
X
7:00-8:00
Y
Data service
Scheduling logic
Replica service
NMI /IF
Apps mware I/F
Proposal evaluation
NRS I/F
GT4 /IF
Data calc
DTS
Topology map
Scheduling algorithm
Proposal constructor
NMI /IF
DTS IF
Scheduling service
Optical control I/F
Proposal evaluator
GT4 /IF
Network allocation
Net calc
NRS
VisualizationX 1,000
Storage X 400Computation
X 500
Few-to-few 10Gbps connectivity with C=500, S=400, and V=1,000.
Will require budget of 100 trillion dollars a year
VisualizationX 1,000
Storage X 400Computation
X 500
Enabling new degrees of App/Net coupling
• Hybrid Optical Packet– Use ephemeral optical circuits to steer the herd of elephants (few to few)– Mice or individual elephants go through packet technologies (many to many)– Either application-driven or network-sensed; hands-free in either case– Other hybrid networks being explored (e.g., wireless + wireline)
• Application-engaged networks– The application makes itself known to the network– The network recognizes its footprints (via tokens, deep packet inspection)– E.g., storage management applications
• Workflow-engaged networks– Through workflow languages, the network is privy to the overall “flight-plan”– Failure-handling is cognizant of the same– Network services can anticipate the next step, or what-if’s– E.g., healthcare workflows over a distributed hospital enterprise
BIRN With the OptIPuter we are BIRN With the OptIPuter we are Addressing the Challenges of Large and Addressing the Challenges of Large and
Distributed DataDistributed Data
Each Brain is Big Data Each Brain is Big Data and Comparisons Must and Comparisons Must
be Made Between be Made Between Many!Many!
. ~5um
} 512 x 512x100,000
Invisible Nodes, Elements,
Hierarchical,Centrally Controlled,
Fairly Static
Traditional Provider Services:Invisible, Static Resources,
Centralized Management
Distributed Device, Dynamic Services, Visible & Accessible Resources, Integrated As Required By Apps
Limited Functionality,Flexibility
Unlimited Functionality,Flexibility
OptIPuter Paradigm Shift
fc *
Parallelism Has Come to Optical Networking (WDM)
Source: Steve Wallach, Chiaro Networks
“Lambdas”Parallel Lambdas Will Drive This Decade
The Way Parallel Processors Drove the 1990s
From: Smarr Talk “The Beginning of the Access Grid” April 15, 1999 www.jacobsschool.ucsd.edu/~lsmarr/talks/ACCESS.4.99_files/frame.htm
P-CSCFPhys. PCSCF
Session Convergence &
NexusEstablishment
End-to-endPolicy
DRAC Built-inServices
(sampler)
WorkflowLanguage
3rd PartyServices
AAA
Access
Value-AddServices
Sources/Sinks
Topology
Metro
Core
Proxy Proxy ProxyProxyProxy
P-CSCFPhys. P-CSCF
Proxy
Grid CommunityScheduler
•smart bandwidth management •Layer x <-> L1 interworking
•Alternate Site Failover
•SLA Monitoring and Verification •Service Discovery
•Workflow Language Interpreter
Bird’s eye View of the Service Stack
</DRAC>
<DRAC>
LegacySessions
(Management & Control Planes)
ControlPlane A
ControlPlane B
OAMOAMOAMPOAMOAMOAMPOAMOAMOAMPOAMOAM
OAMOAMOAMOAM
OAMOAM
How It Works: A Notional View
Admin.
Application
connectivity plane
virtualization plane
dynamic provisioning plane
Alert, Adapt,Route, Accelerate
Detect
supplyevents
eventssupply
AgileNetwork(s)
Application(s)
AAA
NE
from/to peering DRACs
demand
Negotiate
DRAC, portable SW
Routed IP NetworkRouted IP Network
GE
Customer A
CustomerAnetwork
CustomerAnetwork
Customer B
CustomerBnetwork
CustomerBnetwork
High-cap user High-cap user
GE
PP8600
PP8600
DRAC-driven Bypass in Action
VLAN XRouted IP
VLAN XRouted IP
Layer 1Bandwidth
Layer 1Bandwidth
ControlPlane
“DRAC”
UNI
VLAN YCloud Bypass
Flows differentiatedOn IP Subnet or port
Application Application
Services Services Services
Going multi-domain: the SC2004 Demonstrator
data
control
data
control
Chicago Amsterdam
• finesse the control of bandwidth across multiple domains
• while exploiting scalability and intra- , inter-domain fault recovery
• thru layering of a novel SOA upon legacy control planes and NEs
AAA
DRAC DRACDRAC
AAA AAA AAA
DRAC
OMNInetOMNInetODIN Starligh
t
Starlight
Netherlight
Netherlight UvAUvA
ASTNASTNSNMPSNMP
2nd Case Study: DataCenter•CPU + DATA + NET Orchestration•Impact of Virtualization on the End-to-End Session
Site 1 Site 2 Site 3
Horizontal IT IntegrationThe way a provider gainfully operates
for many paying customersover a geographical footprint
What Joe Smith thinks that he’s getting in exchange for a
monthly check to a provider
Joe Smith’sown Mainframeand good Apps
100 - 10,000 blades
vs.
Joe Smith’s Virtual Machines run herew/ Apps, Licenses (SAN not shown)
w/ right-sized bandwidth, 24x7, at 100% disaster-free Zip code
““Horizontal IT Integration” has multiple facets: Horizontal IT Integration” has multiple facets: Virtualization, SOA, Grid Computing, e-Utilities, Service GridsVirtualization, SOA, Grid Computing, e-Utilities, Service Grids
Virtual Compute PlaneVirtual Compute Plane
Virtualized Data-Centers w/ integrated resource control
AppVM
AppVM App
VM
Multi-domain Virtual Network, Security & Services PlaneMulti-domain Virtual Network, Security & Services Plane
Policies Policies Policies
Met
a-S
ched
ule
r
Mu
lti-
reso
urc
e C
oo
rdin
atio
n P
lan
e
Secure RouterNortel VR5000
L3 SwitchNortel ERS8600
L2 SwitchNortel ES5500
Metro OE GatewayNortel OM35/5/65
Application SwitchNortel AS2424SSL
User PlaneUser Plane
WS
Computing RM
WS
Device RM
WS
Storage RM
In Focus: Multi-Resource Coordination Plane
WS = Web ServicesRM = Resource Manager
• Instruments• Sensors• SCADA• RFID infr.
Met
a-S
ched
ule
r
Mu
lti-
reso
urc
e C
oo
rdin
atio
n P
lan
e
DRAC
WS
Execution Engines
DRAC
WS
DRAC
WS
Computation at the Right Place & Time!We migrate live Xen VMs, unbeknownst to
applications and clients, with dynamic cpu+data+net orchestration
Computation at the Right Place & Time!We migrate live Xen VMs, unbeknownst to
applications and clients, with dynamic cpu+data+net orchestration
Seattle
Netherlight
Amsterdam
NYC
Toronto
SC|2005
UvA
Starlight
Chicago
VMs
DynamicLightpaths
hitless remote rendering
The SC05 “VM Turntable” Demonstrator
The whole IT Industry is on a journey
Old World
Static
Silo
Physical
Manual
Application
New World
Dynamic
Shared
Virtual
Automated
Service
© GGF
WHAT ARE WEB SERVICES?• Web services are simple XML-based messages for
machine-machine messaging
–Web services don’t necessarily involve web browsers
–Think of web services as XML-based APIs
• Web services use standard internet technologies to interact dynamically with one another
–Well understood security model
–Loosely coupled
–Can be combined to form complex services–Open agreed standards connect disparate platforms
• Middleware based on web services has enjoyed tremendous success in the past five years
–eBay/PayPal, Amazon and Google all big users of web services
Google’s web service offerings:>Search Google’s eight billion web page database>Dictionary lookup
eBay’s usage of web services:>1 billion web service transactions per month>40% of listings now generated via web services
Web services rapidly becoming an essential part of many IT services in both B2B and B2C market categories
Example: Lightpath Scheduling
• Request for 1/2 hour between 4:00 and 5:30 on Segment D granted to User W at 4:00
• New request from User X for same segment for 1 hour between 3:30 and 5:00
• Reschedule user W to 4:30; user X to 3:30. Everyone is happy.
Route allocated for a time slot; new request comes in; 1st route can be rescheduled for a later slot within window to accommodate new request
4:30 5:00 5:304:003:30
W
4:30 5:00 5:304:003:30
X
4:30 5:00 5:304:003:30
WX
☺
4:30 5:00 5:304:003:30
W
4:30 5:00 5:304:003:30
X
4:30 5:00 5:304:003:30
WX
A
B
C
Scheduling Example - Reroute • Request for 1 hour between nodes A and B between
7:00 and 8:30 is granted using Segment X (and other segments) is granted for 7:00
• New request for 2 hours between nodes C and D between 7:00 and 9:30 This route needs to use Segment E to be satisfied
• Reroute the first request to take another path thru the topology to free up Segment E for the 2nd request. Everyone is happy
A
D
B
C
X7:00-8:00
A
D
B
C
X7:00-8:00
Y
Route allocated; new request comes in for a segment in use; 1st route can be altered to use different path to allow 2nd to also be serviced in its time window
☺
x.x.x.1
y.y.y.1
Optical cut-through
x.x.x.1
y.y.y.1Optical cut-through
e-Science example Application Scenario Current MOP Network Issues
Pt – Pt Data Transfer of Multi-TB Data Sets
Copy from remote DB: Takes ~10 days (unpredictable)Store then copy/analyze
Want << 1 day<< 1 hour, innovation for new bio-scienceArchitecture forced to optimize BW utilization at cost of storage
Access multiple remote DB
N* Previous Scenario Simultaneous connectivity to multiple sitesMulti-domainDynamic connectivity hard to manageDon’t know next connection needs
Remote instrument access (Radio-telescope)
Cant be done from home research institute
Need fat unidirectional pipesTight QoS requirements (jitter, delay, data loss)
Other Observations:• Not Feasible To Port Computation to Data• Delays Preclude Interactive Research: Copy, Then Analyze• Uncertain Transport Times Force A Sequential Process – Schedule Processing After Data Has Arrived• No cooperation/interaction among Storage, Computation & Network Middlewares•Dynamic network allocation as part of Grid Workflow, allows for new scientific experiments that are not possible with today’s static allocation