Fabrizio Gagliardi EU DataGrid Project Leader EGEE Project Coordinator [email protected]

Fabrizio GagliardiEU DataGrid Project LeaderEGEE Project [email protected]

EGEE is proposed as a project funded by the European Union under contract IST-2003-508833

From Grid Prototypes to Grid Infrastructures

NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003

NORDUnet 2003 - Reykjavík, Iceland – 24/27 August 2003- 2

Introduction

• International computing networks are major enablers of new computing models

• GRID computing is one of them, probably the most promising

• Effective implementation of a truly distributed computing model across different non uniform computing administrative domains

• Many prototype projects based on RN Geant and similar international initiatives


Through collaborative Grid projects, there is thepotential for a truely global scientific applications grid

Some DataGrid Projects


• Connections of the different nodes of the EDG testbed are made possible by the EU-funded GEANT project

connecting more than 30 countries across Europespeeds of up to 10 Gbit/shigh data throughput Quality of Service

• EDG and GEANT: the first major production quality tests of the network

EDG & GEANT

speedreliabilitymonitoring capabilities


• Genomic Exploration• Earth Observation• High Energy Physics

Applications

more and more scientists begin to use the Grid computing model and existing Grid testbeds, relying on Grid technology to solve huge data challenges


LHC – Resources Requirements

CMSATLAS

LHCb

Storage – Raw recording rate 0.1 – 1 GByte/sec

Accumulating data at 5-8 PetaBytes/year

10 PetaBytes of disk

Processing – 100,000 of today’s fastest PCs


DataGrid prototypes: DataGrid (I)

• 9.8 M Euros EU funding over 3 years (twice as much from partners)

• 90% for middleware and applications (High Energy Physics, Earth Observation, Genomic Exploration)

• Total of 21 partners, over 150 programmers from research and academic institutes as well as industrial companies

• Three year phased developments & demos (2001-2003)

• Several improved versions of middleware software (final release end 2003)

• Several components of software integrated in the large Particle Physics Production LHC Computing Project (LCG)

• Software used by partner projects: DataTAG, CROSSGRID, GRACE


DataGrid prototypes : DataGrid (II)

• DataGrid testbed: more than 1000 CPUs at more than 15 sites (up to 40)

• Connections made possible by the EU-funded GEANT project • connecting more than 30 countries across Europe

• speeds of up to 10 Gbit/s

• high data throughput

• quality of Service

Site Country CPUs Storage

CC-IN2P3* FR 620 192 GB

CERN* CH 138 1321 GB

CNAF* IT 48 1300 GB

Ecole Poly. FR 6 220 GB

Imperial Coll. UK 92 450 GB

Liverpool UK 2 10 GB

Manchester UK 9 15 GB

NIKHEF* NL 142 433 GB

Oxford UK 1 30 GB

Padova IT 11 666 GB

RAL* UK 6 332 GB

SARA NL 0 10000+ GB

TOTAL 5 1075 14969 GB

*also Dev. TB; +200 TB including tape


The next step: go production

• Similarly to research networks few years ago, after many prototype projects we need to go production

• Major issues: Security, (im)maturity of M/Ware toolkits, difficult user interface for average use, cost and complexity of operations, etc…


EGEE vision:Enabling Grids for E-science in Europe

• Goal•Create a wide European Grid production quality infrastructure on top of present and future EU RN infrastructure

• Build on•EU and EU member states major investments in Grid Technology

•International connections (US and AP)•Several pioneering prototype results •Larg Grid development team (>60 people in EDG)

•Requires major EU funding effort

• Approach•Leverage current and planned national and regional Grid programmes

•Work closely with relevant industrial Grid developers, NRENs and US-AP projects

EGEE

Applications

Geant network


Why EGEE? Impact on Society

• Access to a production quality grid will change the way science and business is done in Europe

An international network of scientist will be able to model a new flood of the Danube in real time, using meteorological and geological data from several centers across Europe

A team of engineering students will be able to run the latest 3D rendering programs from their laptops using the Grid

A geneticist at a conference, inspired by a talk she hears, will be able to launch a complex biomolecular simulation from her mobile phone


Why EGEE? Political context

• Current Grid R&D projects run out within few months

• The EGEE partners have already made major progress in aligning national and regional Grid R&D efforts, in preparation for EGEE

• EGEE will preserve the current strong momentum of the European Grid community, and the enthusiasm of the hundreds of young European researchers already involved in EU Grid projects (>150 in EDG only)


Why EGEE? Historical analogy

• Prior to the EU Geant programme, there was in Europe a multitude of exploratory projects in networking technology. Geant was truly production oriented, and brought European telecom operators actively into the picture

• In a similar way, EGEE can ensure preservation of current investments in European Grid R&D, extending the present infrastructure and focussing all activities towards establishing a production quality Grid


EGEE Partner Federations

• Integrate regional Grid efforts

• Represent leading grid activities in Europe


EGEE Activity Areas

• Services• Deliver “production level” grid services (manageable, robust, resilient to failure)• Ensure security and scalability

• Middleware• Professional Grid middleware re-engineering activity in support of the production

services• Networking

• Proactively market Grid services to new research communities in academia and industry

• Provide necessary education


ResourceCenter

(Processors, disks)

Grid server Nodes

ResourceCenter

ResourceCenter

ResourceCenter

OperationsCenter

Regional SupportCenter

(Support for Applications

Local Resources)

Regional Support

Regional Support

Regional Support

EGEE Operations Structure


EGEE Service Activity

• Create, operate, support and manage a production quality infrastructure

• Structure: • EGEE Operations Management at CERN • EGEE Core Infrastructure Centres in the UK, France,

Italy and CERN (leveraging HEP LCG at the start), responsible for managing the overall Grid infrastructure

• Regional Operations Centres, responsible for coordinating regional resources, regional deployment and support of services in all other countries

• Offered services:• Middleware deployment and installation• Software and documentation repository• Grid monitoring and problem tracking• Bug reporting and knowledge database• VO services• Grid management services


EGEE Networking Activity

• Dissemination and outreach

• User training and induction

• Application identification and support

• Two pilot application centers (for high energy physics and biomedical grids)

• One more generic component dealing with longer term recruitment and support of other communities

• Policy and International cooperation

rely on a supporting network in the partner regions


EGEE Middleware Activity

• Hardening and re-engineering of existing middleware functionality, leveraging the experience of partners

• Activity concentrated in few major centers

• Key services: Resource Access• Data Management• Information Collection and Accounting• Resource Brokering (Italy)• Quality Assurance• Grid Security• Middleware Integration• Middleware Testing


EGEE & Industry

• Industrial participation encouraged both as potential end-users and IT technology and service suppliers

• Normally through national and regional Grid EGEE federations

• EGEE will maintain an Industry Forum to keep selected Industrial and Commercial interested parties in close contact

• Services developed in first EGEE 2 years phase (2004-5) might be tendered to Industry in second phase (2006-7)


EGEE Timeline

• May 2003: proposal submitted

• July 2003: proposal accepted

• September 2003: start negotiation

• April 2004: start project


Conclusions

• The EU DataGrid project has successfully fulfilled its role of EU Grid flagship project in collaboration with several other EU and international projects

• Essential to keep the momentum and the current lead in production Grids in Europe

• Important to build an international cooperation between European and US/AP Grid infrastructure projects

• The scientific user communities are already international (HEP is an excellent example) and so the computing resources and most of the experimental instruments

• EGEE proposes the right framework and plans to accomplish the above objectives leveraging present and future international research networks


More information

• More information on the EU DataGrid project on:www.edg.org

• More information on EGEE on:www.cern.ch/egee

Or mail me at: [email protected]

Fabrizio Gagliardi EU DataGrid Project Leader EGEE Project Coordinator [email protected]

Documents

Transcript of Fabrizio Gagliardi EU DataGrid Project Leader EGEE Project Coordinator [email protected]