UK DTI Mission – 29 June 2004 - 1 Grid Deployment Ian Bird LCG Deployment Area Manager & EGEE...

UK DTI Mission – 29 June 2004 - 1

Grid DeploymentGrid Deployment

Ian Bird

LCG Deployment Area Manager &EGEE Operations Manager

IT Department, CERN

Presentation to UK DTI mission29th June 2004


OverviewOverview

Grid Deployment Scope and responsibilities Organisation

Deployment activities Deployment in LCG EGEE


Grid Deployment: Scope of ResponsibilitiesGrid Deployment: Scope of Responsibilities

Certification activities Certification of middleware as a coherent set of services Preparing that package for deploying

Operational and support activities Coordinating and supporting the deployment to collaborating computer

centres Coordinating Grid Operations activities Providing Operational support Providing Operational security support Providing User support CA management VO registration and management

Policy CA and user registration policies Operational policy Security policies Resource usage and access policies


DeploymentArea ManagerDeployment

Area ManagerGrid Deployment

BoardGrid Deployment

Board

CertificationTeam

CertificationTeam

DeploymentTeam

DeploymentTeam

ExperimentIntegration

Team

ExperimentIntegration

Team

Testing groupTesting group

Security group

Security group

Storage group

Storage group

GDB task forces

ad-hoc collaborations

HEPiX

GGF

Grid Projects:EGEE,Trillium,Grid2003/OSG,etc

Regional Centres

LHC Experiments

LCG Deployment Area

LCG Deployment Organisation and Collaborations

OperationsCentres- RAL

OperationsCentres- RAL

Call Centres- FZK

Call Centres- FZK

Advises, informs,Sets policy

Set requirements

Set requirements

Col

labo

rativ

e ac

tiviti

es

participate

participate


Certification activitiesCertification activities


Certification, Testing and Release CycleCertification, Testing and Release Cycle

CERTIFICATIONTESTING

CERTIFICATIONTESTING SERVICESSERVICES

Integrate

BasicFunctionality

Tests

Run testsC&T suitesSite suites

RunCertification

Matrix

Releasecandidate

tag

PR

E-P

RO

DU

CT

ION

PR

OD

UC

TIO

N

APPINTEGR

APPINTEGR

Certifiedrelease

tag

DE

VE

LO

PM

EN

T &

IN

TE

GR

AT

ION

UN

IT &

FU

NC

TIO

NA

L T

ES

TIN

G

DevTag

HEPEXPTS

BIO-MED

OTHERTBD

APPSSW

Installation

DE

PL

OY

ME

NT

PR

EP

AR

AT

ION

Deploymentrelease

tag

DEPLOYDEPLOY

Productiontag

Developers


Operational activitiesOperational activities


The LCG Deployment BoardThe LCG Deployment Board

Grid Deployment Board (GDB) set up to address policy issues requiring agreement and negotiation between resource centres

Members: country representatives, applications, and project managers Sets up working groups

Short term or ongoing Bring in technical experts to focus on specific issues

GDB approves recommendations from working groups Groups:

Several that outlined initial project directions (operations, security, resources, support)

Security – standing group – covers many policy issues Grid Operations Centre task force User Support group Storage management and other focused issues Service challenges


Operations services for LCGOperations services for LCG

Operational support Hierarchical model

• CERN acts as 1st level support for the Tier 1 centres• Tier 1 centres provide 1st level support for associated Tier 2s

Grid Operations Centres (GOC)• Provide operational monitoring, troubleshooting, coordination of incident

response, etc.• RAL (UK) led sub-project to prototype a GOC• 2nd GOC in Taipei now in operation

– Together providing 16hr coverage– Expect 3rd centre in Canada/US to help achieve 24hr coverage

User support Central model

• FZK provides user support portal– Problem tracking system web-based and available to all LCG participants

• Experiments provide triage of problems CERN team provide in-depth support and support for integration of

experiment sw with grid middleware


SecuritySecurity

LCG Security Group (led by Dave Kelsey (RAL) LCG usage rules – proposed as general Grid usage guidelines Registration procedures and VO management

• Agreement to collect only minimal amount of personal data

• Currently registration is only valid for 6 month (procedures will change) Initial audit requirements are defined Initial incident response procedures

• Site security contacts etc. are defined Set of trusted CAs (including Fermilab online KCA) Security policy (to be finished by end of year)

This group is now a Joint Security group covering several grid projects/infrastructure


Deployment: LCG Deployment: LCG EGEE EGEE


Sites in LCG-2/EGEE-0 : June 28 2004Sites in LCG-2/EGEE-0 : June 28 2004

Austria U-Innsbruck

Canada Triumf

Alberta

Carleton

Montreal

Toronto

Czech Republic

Prague-FZU

Prague-CESNET

France CC-IN2P3

Clermont-Ferrand

Germany FZK

Aachen

DESY

GSI

Karlsruhe-U

Wuppertal

Greece HellasGrid

Hungary Budapest

India TIFR

Israel Tel-Aviv

Weizmann

Italy CNAF

Frascati

Legnaro

Milano

Napoli

Roma

Torino

Japan Tokyo

Netherlands NIKHEF

SARA

Pakistan NCP

Poland Krakow

Portugal LIP

Russia SINP-Moscow

JINR-Dubna

Spain PIC

UAM

USC

UB-Barcelona

IFCA

CIEMAT

IFIC

Switzerland CERN

CSCS

Taiwan ASCC

IPAS

NCU

UK RAL

Birmingham

Cavendish

Glasgow

Imperial

Lancaster

Manchester

QMUL

RAL-PP

Sheffield

UCL

UCL-CCC

US BNL

FNAL

HP Puerto-Rico

• 22 Countries• 63 Sites (49 Europe, 2 US, 5 Canada, 6 Asia, 1 HP)

• Coming: New Zealand, China, other HP (Brazil, Singapore)

• 3800 cpu


LCG and EGEE OperationsLCG and EGEE Operations

EGEE is funded to operate and support a research grid infrastructure in Europe

The core infrastructure of the LCG and EGEE grids will be operated as a single service, will grow out of LCG service LCG includes US and Asia, EGEE includes other sciences Substantial part of infrastructure common to both

LCG Deployment Manager is the EGEE Operations Manager CERN team (Operations Management Centre) provides coordination,

management, and 2nd level support Support activities are expanded with the provision of

Core Infrastructure Centres (CIC) (4) Regional Operations Centres (ROC) (9) ROCs will be coordinated by Italy, outside of CERN (which has no ROC)


LCG LCG EGEE EGEE

Operational support: The LCG GOC is the model for the EGEE CICs

• CIC’s replace the European GOC at RAL• Also run essential infrastructure services• Provide support for other (non-LHC) applications• Provide 2nd level support to ROCs

User support: Becomes hierarchical Through the Regional Operations Centres (ROC)

• Act as front-line support for user and operations issues• Provide local knowledge and adaptations

Coordination: At CERN (Operations Management Centre) and CIC for HEP


PolicyPolicy


LCG Security and Availability PolicyLCG Security and Availability Policy

Prepared jointly with GOC group Objectives

Agreed set of statements Attitude of the project towards security and availability Authority for defined actions Responsibilities on individuals and bodies

Promote the LHC science mission Control of resources and protection from abuse Minimise disruption to science Obligations to other network (inter- and intra- nets) users Broad scope: not just hacking

Maximise availability and integrity of services and data Resources, Users, Administrators, Developers (systems and

applications), and VOs Does NOT override local policies Procedures, rules, guides etc

contained in separate documents


Resource access policyResource access policy

Resource negotiation Each participating site might have constraints – funding policy, levels of

support, user communities, etc. Those part of EGEE have committed to provide resources to EGEE

applications within those constraints

The Operations group together with applications groups, and Regional Operations Centres Negotiate access to resources at sites on behalf of application community Each site might have local access policies – users, applications, etc The grid infrastructure should not override local site policies

• However, in the context of an EU funded project a commitment to the project may change the policy


Approach to SLA’sApproach to SLA’s

In LCG: Formal MoU will be made between CERN and the Tier 1 centres for

services to the LHC experiments Countries commit to a certain level of resources to be provided

In EGEE: A goal of project is to understand what SLA’s mean in a grid framework (we

do not know what they are yet): Envisage:

• SLA between ROC and resource centres in a region– Resources, support levels, backup support, access policies, etc.

• SLA between regions (via ROC)– Accumulation of individual SLA’s

• SLAs with network providers• Performance against targets will be public information


InteroperabilityInteroperability

Several grid infrastructures for LHC experiments: LCG-2/EGEE, Grid2003/OSG, NorduGrid, other national grids

LCG/EGEE explicit goals to interoperate One of LCG service challenges Joint projects on storage elements, file catalogues, VO management, etc.

Most are VDT (or at least Globus-based) Grid2003 & LCG use GLUE schema

Issues are: File catalogues, information schema, etc at technical level Policy and semantic issues

UK DTI Mission – 29 June 2004 - 1 Grid Deployment Ian Bird LCG Deployment Area Manager & EGEE...

Documents

Transcript of UK DTI Mission – 29 June 2004 - 1 Grid Deployment Ian Bird LCG Deployment Area Manager & EGEE...