TR-GRID ULUSAL GRID OLUŞUMU GRID PROJELERİ ve GRID UYGULAMALARI Dr. Burcu Ortakaya
OFC 2009 – Grid vs Cloud Computing and Why This Should … - Challenges in Enabling Grid... · 16...
Transcript of OFC 2009 – Grid vs Cloud Computing and Why This Should … - Challenges in Enabling Grid... · 16...
Cees de Laat!GLIF.is founding member!
OFC 2009 – Grid vs Cloud Computing and Why This Should Concern the Optical Networking Community
Challenges in Enabling Grid Computing
over Optical Networks
100000 flops/byte!
Telescopes
Input nodes
Correlator nodes
Output node
..... To equal the hardware
correlator we need:
16 streams of 1Gbps
16 * 1Gbps of data
2 Tflops CPU power
2 TFlop / 16 Gbps =
1000 flops/byte
THIS IS A DATA FLOW
PROBLEM !!!
SCARIe: a research project to create a Software Correlator for e-VLBI. VLBI Correlation: signal processing technique to get high precision image from
spatially distributed radio-telescope.
The SCARIe project
The “Dead Cat” demo" SC2004 & iGrid2005!
SC2004,
Pittsburgh,
Nov. 6 to 12, 2004
iGrid2005,
San Diego,
sept. 2005
Produced by:
Michael Scarpa
Robert Belleman
Peter Sloot
Many thanks to:
AMC
SARA
GigaPort
UvA/AIR
Silicon Graphics,
Inc.
Zoölogisch Museum
US and International OptIPortal Sites!
NCMIR SIO
UIC
USGS EDC
TAMU UCI
SARA KISTI
NCSA &
TRECC
CALIT2
AIST RINCON & Nortel
IJKDIJK!
Sensor grid: instrument the dikes!
30000 sensors (microphones) to cover all Dutch dikes!
First controlled breach occurred on sept 27th ‘08:!
e-F
oo
d!
e-B
ioS
cie
nce!
e-B
iob
an
kin
g!
e-C
OA
ST!
e-E
co
log
y!
e-D
ata
-
inte
nsiv
e
scie
nc
es!
.....!
Virtual Laboratory!
generic e-Science services!
High Performance & Distributed Computing!
Web & Grid services!
GLIF 2008! Visualization courtesy of Bob Patterson, NCSA Data collection by Maxine Brown.
Management
Visualisation
Mining
Web2.0
Media
Backup
Security
NetherLight!Meta
CineGrid
Medical
TV
Gaming
Conference
RemoteControl
Clouds
Distributed Simulations
Predictions
EventProcessing StreamProcessing
Workflow
DataExploration
Who (just to name a few) working Interfaces - similar
solutions addressing the previous challenges
•! CANARIE - UCLP
•! ESnet = OSCARS
•! G-LAMBDA - GNS-WSI
•! HPDM - VLAN based lightpaths
•! NORTEL - DRAC
•! UvA - Token based service, NDL, etc
•! Phosphorous - G2MPLS, UCLP2, NRPS - > NSP
•! GN2 - JRA3 - AutoBahn - IDM
•! I2 - DRAGON/ HOPI - DCN
•! FermiLab - Lambda Station
•! DoE - LambdaStation, TeraPaths
Slide: Gigi Karmous Edwards!
•! Harmony architecture (I) !
•!15!
Grid-aware GMPLS (G2MPLS) for Grid Network Services
!! Two models for the layering between Grid and Network resources
G2MPLS overlay model
different scope with respect to the IETF GMPLS Overlay & Peer
G2MPLS integrated model
My view!
GLIF!
GN2/3!
Federica !
Phosphorus!
Onelab!
Planetlab!
FIRE!
etc.!
OSI!
7!
6!
5!
4!
3!
2!
1!
0!
•! needs
repeatable
experiments!
•! needs QoS &
lightpaths!
•! needs capacity
and capability!
•! needs
infrastructure
descriptions!
TeraThinking!
•! What constitutes a Tb/s network?!
•! CALIT2 has 8000 Gigabit drops ?->? Terabit Lan?!
•! look at 80 core Intel processor!
–! cut it in two, left and right communicate 8 TB/s!
•! think back to teraflop computing!!
–! MPI turns a room full of pc’s in a teraflop machine!
•! massive parallel channels in hosts, NIC’s!
•! TeraApps programming model supported by!
–! TFlops ! !-> !MPI / Globus!
–! TBytes ! !-> !OGSA/DAIS!
–! TPixels ! !-> !SAGE!
–! TSensors !-> !LOFAR, LHC, LOOKING, CineGrid, ...!
–! Tbit/s ! !-> !?!
ref Larry Smarr & CdL!
The Problem!
A
G
F E
D
C
B
H
I want HC and AB!
Success depends on the order!
Wouldn’t it be nice if I could request [HC, AB, ...]!
?!
Network Description Language
Object Subject Predicate
•! From semantic Web / Resource Description Framework.
•! The RDF uses XML as an interchange syntax.
•! Data is described by triplets:
Location! Device! Interface! Link!
name! description! locatedAt! hasInterface!
connectedTo! capacity! encodingType! encodingLabel!
Object
Subject Subject
Object
Subject
Object
Subject
Object
Subject
Network Description Language
Choice of RDF instead of XML syntax
Grounded modeling based on G0805 description:
Article: F. Dijkstra, B. Andree, K. Koymans, J. van der Ham, P. Grosso, C. de Laat, "A Multi-Layer Network
Model Based on ITU-T G.805"
NDL + PROLOG!
•!Reason about
graphs!
•!Find sub-graphs
that comply with
rules !
Research Questions:!
•!order of requests!
•!complex requests!
•!Usable leftovers!
139.63.145.0
139.63.145.1
139.63.145.15
139.63.145.16139.63.145.17
139.63.145.18
139.63.145.2
139.63.145.3
139.63.145.31
139.63.145.32
139.63.145.33
139.63.145.34
139.63.145.38
139.63.145.40
139.63.145.41
139.63.145.42139.63.145.43
139.63.145.44
139.63.145.45
139.63.145.46
139.63.145.49
139.63.145.50
139.63.145.51
139.63.145.52
139.63.145.63
139.63.145.64
139.63.145.65
139.63.145.66
139.63.145.68
139.63.145.69139.63.145.70139.63.145.71
139.63.145.72
139.63.145.73139.63.145.74
139.63.145.75
139.63.145.79
139.63.145.81
139.63.145.82
139.63.145.83
139.63.145.84
139.63.145.85
139.63.145.86
139.63.145.87
139.63.145.88
139.63.145.94 192.168.0.1 192.168.0.2
192.168.0.3
192.168.0.4
192.168.0.5
192.168.0.6
192.168.1.1
192.168.1.2
192.168.1.3
192.168.1.4
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
Visualisation
Initialization and BFS discovery of NEs
95.9
94.5
95.8
96.
95.9
99.9
99.
100.
100.
94.599.9
99.9
100.
95.899.
96.
99.9
100.
100.
99.8
100.
100.
98.9
99.8
100.
100.
100.
100.
100.
98.9
101.
100.
101.100.
100.
100.
139.63.145.94192.168.0.1192.168.0.2
192.168.0.3
192.168.0.4
192.168.0.5
192.168.0.6
192.168.1.1
192.168.1.2
192.168.1.3
192.168.1.4
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
Network flows using real-time bandwidth measurements
nodePath = ConvertIndicesToNodes[
ShortestPath[ g,
Node2Index[nids,"192.168.3.4"],
Node2Index[nids,"139.63.77.49"]],
nids];
Print["Path: ", nodePath];
If[NetworkTokenTransaction[nodePath, "green"]==True,
Print["Committed"], Print["Transaction failed"]];
Path:
{192.168.3.4,192.168.3.1,139.63.77.30,139.63.77.49}
Committed
Transaction on shortest path with tokens
Topology matters can be dealt with algorithmically
Results can be persisted using a transaction service built in UPVN
Needs["WebServices`"]
<<DiscreteMath`Combinatorica` <<DiscreteMath`GraphPlot`
InitNetworkTopologyService["edge.ict.tno.nl"]
Available methods:
{DiscoverNetworkElements,GetLinkBandwidth,GetAllIpLinks,Remote, NetworkTokenTransaction}
Global`upvnverbose = True;
AbsoluteTiming[nes = BFSDiscover["139.63.145.94"];][[1]]
AbsoluteTiming[result = BFSDiscoverLinks["139.63.145.94", nes];][[1]]
Getting neigbours of: 139.63.145.94
Internal links: {192.168.0.1, 139.63.145.94} (...)
Getting neigbours of:192.168.2.3
Internal links: {192.168.2.3}
Mathematica enables advanced graph queries, visualizations and real-
time network manipulations on UPVNs"
ref: Robert J. Meijer, Rudolf J. Strijkers, Leon Gommans, Cees de Laat, User Programmable Virtualiized Networks, accepted for publication to the IEEE e-Science 2006 conference Amsterdam.
Interactive programmable networks
Multi Layer Service Architecture!
Network
layers
Application
layers Use Interface
Control Interface
(protocols API’s)
Network
Service
Application
5b of 6!
n.a.v. interview met Kees Neggers (SURFnet) & Cees de Laat (UvA) "
•!BSIK projects
GigaPort &!
•!VL-e / e-
Science!
cookreport.com!
Questions ?!
Thanks: Paola Grosso & Jeroen vd Ham & Freek
Dijkstra & team for several of the slides.!
A Declarative Approach to Multi-Layer Path Finding Based on Semantic Network Descriptions.!
http://delaat.net:/~delaat/papers/declarative_path_finding.pdf!