Grid’5000 Grid'5000 - DAS-3 workshop 104/12/06 Grid’5000 * DAS-3 – Grid'5000 workshop December...

Grid'5000 - DAS-3 workshop 104/12/06

Grid’5000

Grid’5000Grid’5000*

DAS-3 – Grid'5000 workshopDecember 4th, 2006

www.grid5000.frwww.grid5000.fr

*5000 CPUs

Pierre NEYRON - INRIAPierre NEYRON - INRIAHead of Grid’5000Head of Grid’5000 Technical Committee Technical Committee

mailto:mailto: [email protected]@imag.fr


Grid’5000

Grid'5000 hardwareOverviewHardware summary

Grid'5000 NetworkBackbone networkSecurity architectureRouting tables

Grid'5000 Software stackAccounts managementGrid tools

Grid'5000 PeopleCommitteesTechnical staffCollaborative work

OutlineOutline


Grid’5000

Grid’5000 platform overviewGrid’5000 platform overview

1000 (684)518 (518)

500 (198)

500 (116)

500 (252)

500 (270)

500 (322)

500 (146)

500 (94)

Nation wide Grid:9 sites, 500 CPUs/site(1000 for Orsay)

Currently:2600 CPUs

Interlink network:10GB/s dark fibre


Grid’5000

Grenoble

RennesLyon

Toulouse

Sophia

Orsay

Bordeaux


Grid’5000

Grid’5000 platform hardwareGrid’5000 platform hardwareGrid'5000 hardware summary

see: https://www.grid5000.fr/mediawiki/index.php/Special:HardwareAbstract

Manufacturers:• HP• Dell• IBM• Sun• AppleCPU types:• AMD Opteron [80%]• Intel Xeon64• Intel Itanium2• Intel Xeon32• PowerPC G5Network:• Gigabit Ethernet• Myrinet• Infiniband


Grid’5000


Grid'5000 NetworkBackbone networkSecurity architectureRouting tables

Grid'5000 Software stacksAccounts managementGrid tools


OutlineOutline


Grid’5000

Grid’5000 backbone networkGrid’5000 backbone network

Grid'5000 dedicated backbone networkRenater4: Sites connected to a single layer-2 VLAN

RENATER

Rennes

RENATER

Lille

RENATER

Nancy

RENATER

BordeauxParis Lyon

RENATER

RENATER

Grenoble

Lyon

RENATER

Sophia

RENATER

Toulouse

Black fibreDedicated LambdaFully isolated traffic!

10 Gbps

10 Gbps

10 Gbps

10 Gbps

10 Gbps

10 G

bps

10 Gbps

10 Gbps

10 Gbps

1Gb throughRenater3

(MPLS)Renater4 VLAN 550

192.168.4.0/24

RENATER

Orsay

DAS-3 via Geant and Surfnet


Grid’5000

Grid'5000 router

Site's Grid'5000access machineSecure access

Grid'5000 site

Grid’5000 security architectureGrid’5000 security architecture

2 fibers:1 dedicated to Grid’5000

Grid'5000site

Grid'5000site

Grid'5000 network is isolated from Renater general purpose network.Access is granted thru secure gateways

Lab

Reginal Node

Regional node

Regional Node

Cluster

Lab's router

SSH

Regional Node

Internet

Renater general purposenational

network backboneGrid'5000 dedicated

isolatednetwork backbone

Lab network

Grid'5000 site

Site servers(DNS, LDAP,

NFS, DHCP/PXE,Console, Reboot)

Cluster nodesworkstation

Renater

ds


Grid’5000

Grid’5000 routing tablesGrid’5000 routing tablesSite-to-Site routing tablesBordeaux 192.168.133.0 /24 (255.255.255.0)

Grenoble (idpot ) 129.88.70.0 /24 (255.255.255.0)

(icluster2) 129.88.98.0 /24 (255.255.255.0)

(icare ) 192.168.11.0 /24 (255.255.255.0)

Lille 192.168.159.0 /24 (255.255.255.0)

Lyon 10.69.0.0 /17 (255.255.128.0)

Nancy 172.28.52.0 /22 (255.255.252.0)

Orsay 172.24.0.0 /16 (255.255.0.0 )

Rennes 131.254.202.0 /23 (255.255.254.0)

Sophia 138.96.20.0 /22 (255.255.252.0)

Toulouse 192.168.22.0 /24 (255.255.255.0)

On site, not routed local networksSeveral private networks are used but not routed from site to site: admin network, IPMA/RSA network, Myrinet/Infiniband network...

Routing for experiments using virtualization10.<site>.0.0 address space used for virtual nodes (work in progress)

=> DAS-3 connection: Need to avoid IP conflicts


Grid’5000


Grid'5000 NetworkBackbone networkRouting tablesSecurity



OutlineOutline


Grid’5000

Cluster

A single account for every user on every sites:•Every site runs a LDAP server•Every site manages its LDAP branch of user accounts•Every site replicates the LDAP branches of the other sites•Home directories structure automatically createdAll sites share the same global Grid'5000 account directory

Grid'5000 account managementGrid'5000 account management

Grid'5000site

SSH

Home directories/home/<site>/<user>

LDAP accountssite local branch +replicated remote branches

Grid'5000 site

SSH

Grid'5000site

remote accounts replication+

home directory creation & on demand synchronization

Unique user accounts on Grid'5000


Grid’5000

Cluster

Grid'5000 grid toolsGrid'5000 grid tools

Grid'5000site

SSH

Grid'5000 siteSSH

Resource management & deployment

homesLDAP

Grid'5000site

OAR

KadeployOAR

Kadeploy

OARGrid

•Every cluster is managed by a OAR server•On user's demand, every cluster can be redeployed with the user's provided environment•OAR grid allows grid submission over the whole grid


Grid’5000


Grid'5000 NetworkBackbone networkRouting tablesSecurity



OutlineOutline


Grid’5000

Grid’5000 committeesGrid’5000 committees

• Steering Committee (CP): Pr. Franck Cappello, Pr. Thierry Priol, Pr. Brigitte Plateau & al

• Technical Committee (CT): 20 engineers + researchers, about 50 persons

• Users: more than 200 active users, located on Grid'5000 sites/nearby/in foreign countries (Netherlands, Brazil, Africa, Japan...)


Grid’5000

Grid’5000 technical staffGrid’5000 technical staff

Grid'5000 technical staff is dispatched on all 9 sites, France wild.National: coordination + development (2 engineers)Sites: 1 or 2 engineers per sites for the administrationPeople hosted in INRIAresearch labs or universities. Bordeaux – Grenoble – Lille – Lyon –

Nancy - Orsay (Paris) – Rennes – Sophia-Antipolis (Nice) - Toulouse

Grid'5000 as acomputer grid, andas a grid of People


Grid’5000

Grid’5000 collaborative workGrid’5000 collaborative work

Communication means:• Mailing lists (CP, CT, users, site-users, site-

staff, network-staff)• Instant messaging (Grid'5000 dedicated

Jabber server)• Phone + audio-conf (monthly CT audio-

meeting + task dedicated meeting... )• Physical meeting every 4 months Tools aiding collaborative work:• Wiki: public web site + user's portal +

committees portal• Bugzilla: ticket tracking, task assignment• SVN repository / GForge• incident tracking tools/cross admin

logs/monitoring


Grid’5000

QUESTIONS ?QUESTIONS ?

Grid’5000 Grid'5000 - DAS-3 workshop 104/12/06 Grid’5000 * DAS-3 – Grid'5000 workshop December...

Documents

Transcript of Grid’5000 Grid'5000 - DAS-3 workshop 104/12/06 Grid’5000 * DAS-3 – Grid'5000 workshop December...