DAS-1 DAS-2 DAS-3 · Outline • Introduction • DAS-1, DAS-2, DAS-3 systems • Impact of DAS •...
Transcript of DAS-1 DAS-2 DAS-3 · Outline • Introduction • DAS-1, DAS-2, DAS-3 systems • Impact of DAS •...
Henri BalVrije Universiteit Amsterdam
Faculty of Sciences
DAS-1 DAS-2 DAS-3
Outline
• Introduction• DAS-1, DAS-2, DAS-3 systems• Impact of DAS• Collaborations with
- VL-e- SURFnet- Grid’5000
• New users
Distributed ASCI Supercomputer
• Joint infrastructure of ASCI research school• Clusters integrated in a single
distributed testbed • Long history (10 years)
and continuity
DAS is a Computer Science grid
• Motivation: CS needs its own infrastructure for- Systems research and experimentation- Distributed experiments- Doing many small, interactive experiments
• DAS is simpler and more homogeneous than other grids- Single operating system - “A simple grid that works’’
Typical heterogeneous Grid
DAS-1 (1997-2002)VU (128) Amsterdam (24)
Leiden (24) Delft (24)
6 Mb/sATM
Configuration
200 MHz Pentium Pro64-128 MB memory2.5 GB local disk
Myrinet interconnectBSDI => Redhat Linux
DAS-2 (2002-2007)VU (72) Amsterdam (32)
Leiden (32) Delft (32)
SURFnet1 Gb/s
Utrecht (32)
Configuration
two 1 GHz Pentium-3s>= 1 GB memory20-80 GB disk
Myrinet interconnectRedhat Enterprise LinuxGlobus 3.2PBS => Sun Grid Engine
DAS-3 Procedure
• NWO proposal (ASCI, VL-e, MultimediaN), Sep’04• Funding from NWO/NCF, Apr’05• Steering group + implementation group• European tender (with Stratix and TUD/GIS)• Selected ClusterVision, Apr’06
• Optical wide-area network bySURFnet (GigaPort-NG project)
VU (85)
DAS-3 dual AMD Opterons4 GB memory250-1500 GB diskMore heterogeneous:
2.2-2.6 GHzSingle/dual core nodes
Myrinet-10G (exc. Delft)Gigabit EthernetScientific Linux 4Globus, SGE
TU Delft (68) Leiden (32)
UvA/MultimediaN (46)
UvA/VL-e(40)
SURFnet6
10 Gb/s lambdas
Performance DAS-1 DAS-2 DAS-3
# CPU cores 200 400 792
SPEC CPU2000 INT (1 core) 78.5 454 1445
SPEC CPU2000 FP (1 core) 69.0 329 1858
1-way latency MPI (μs) 21.7 11.2 2.7
Max. throughput (MB/s) 75 160 950
Wide-area bandwidth (Mb/s) 6 1000 40000
Outline
• Introduction• DAS-1, DAS-2, DAS-3 systems• Impact of DAS• Collaborations with
- VL-e- SURFnet- Grid’5000
• New users
Usage of DAS• ~200 users, 34 Ph.D. theses• Clear shift of interest:
Cluster computing
Distributed computing
Grids & peer-to-peer computing
Virtual laboratories for e-Science
Opening workshops• DAS-1 opening (1998):
- only 1 talk (VU) about distributed computing
• DAS-2 opening (2002):- only 1 talk (VU) about local computing
• DAS-3 opening (NOW):- talks about grids, e-Science, optical networks,
international embedding, distributed multimedia computing, P2P/gossiping, co-allocating Grid Scheduler
Impact of DAS
• Major incentive for VL-e 20 M€ BSIK funding- Virtual Laboratory for e-Science
• Collaboration SURFnet on DAS-3- SURFnet provides multiple 10 Gb/s light paths
• Collaboration with French Grid’5000- Towards a European scale CS grid?
Grid’5000
VL-e: Virtual Laboratory for e-Science
Some VL-e results• VLAM-G workflow system• KOALA scheduler (see Mohamed’s talk)• Zorilla peer-to-peer system (see Drost’s talk)• Java-centric grid computing (see demo)
- Ibis: Grid communication library- Satin: divide&conquer on grids- JavaGAT: Gridlab Application Toolkit
• Applications of Ibis, Satin, JavaGAT:- Automatic grammar learning (UvA)- Protein identification (AMOLF)- Brain image analysis (VUmc)
Dordrecht1
Breda1
Tilburg1
Sub network 4: Blue Azur
Subnetwork 3: Red
Sub network 1: Green
Subnetwork 2: Dark blue
Subnetwork 5: Grey
Zwolle1
Enschede1
Nijmegen1
Dwingeloo1
Amsterdam2
Den Bosch1
Utrecht1
Maastricht1
Eindhoven1
Rotterdam1
Groningen1
Delft
Leiden Hilversum
Amsterdam1
Den Haag
• 4 DAS-3 sites happen to be on Subnetwork 1 (olive green)
• Only VU needs extra connection
Source: Erik-Jan Bos
SURFnet6
Adding 10G waves for DAS-3
Delft
Amsterdam1Amsterdam2
Amsterdam3
G5 G5
G5 G5
WSS <-
split
select
select
split
WSS ->
WSS <-
WSS ->
WSS <-
split
select
select
split
WSS ->
WSS <-
WSS ->
G3
Delft
G5G3
G3G3
GMD
GMD
G5
Utrecht
G1
Delft
G1
G1G1
GMD
GMD
G9
G9
G9
G9
GMD
GMD
Hilversum
G6
Delft
G6
G6G6
GMD
GMD
G5
G5
Leiden
G4
G4
G4
G4
GMD
GMD
Den Haag
G5 G5
G5 G5 G5 G5G5
G5
• Band 5 added at all participating nodes
• WSS added for reconfigurability
• “Spur” to connect VU• Full photonic mesh
possible• Source: Erik-Jan
Bos
• Also see De Laat’s talk
“spur” –hardwired connections
WSS
UvA
Leiden
Delft
MultiMedian
VU
International grid experiments
• Distributed supercomputingon GridLab- Real speedups on a very
heterogeneous testbed
• Crossgrid experiments- N-body simulations (astronomy)- Grid-based interactive
visualization of medical images
Grid’5000
Grenoble
RennesLyon
Toulouse
Sophia
Orsay
Bordeaux
1000
500
500
500
500
500
500
500
500
Grid’5000
• Optical 10 Gb/sec link between Paris and Amsterdam, funded by VL-e
• See Cappello’s talk (and keynote @ CCGrid2007)
Connecting DAS-3 and Grid’5000
Users of DAS-3• ASCI, VL-e• MultimediaN
- See MultimediaN talk + demo• NWO projects
- StarPlane, JADE-MM, GUARD-G, VEARD, GRAPE Grid, SCARIe, AstroStream, CellMath,MesoScale, …..
• NCF projects (off-peak hours)
New users
• DAS is intended primarily for computer scientists from ASCI, VL-e, MultimediaN
• How do we define “Computer Scientist”?• Need a light-weight admission test …
Proposed test
There are only 10 typesof people in the world:
Those who understand binaryand those who don't.
☺= Pass as Computer ScientistSource:
Acknowledgements• Lex Wolters• Dick Epema• Cees de Laat• Frank Seinstra• Erik-Jan Bos• Henk van der Vorst
• Andy Tanenbaum• Bob Hertzberger• Henk Sips• Aad van der Steen• Many others
• NWO• NCF• SURFnet• VL-e• MultimediaN• VU, UvA, TUD, Leiden
• ASCI• ClusterVision• TUD/GIS• Stratix
Special acknowledgement ….
• Reasons why DAS is ``a simple grid that works’’
• 1) simple, homogeneous design
• 2) Kees Verstoep