Introduction to Grid Technologies in EGEE

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

•iASTRO MC MEETING&WORKSHOP, 27-30, APRIL, 2005,SOFIA, BULGARIA

Introduction to Grid Technologies in EGEE

Emanouil Atanassov,

Aneta Karaivanova and Todor Gurov

Institute for Parallel Processing - BAS

Grid Day Nis 31 Jan 2006 2


INFSO-RI-508833

Overview

• Evolvement of Grids• What is Grid?• Grid Services• Goals of the EGEE project• Building a production Grid for e-Science• Grid applications in EGEE and SEE-GRID• The Grid Challenges



INFSO-RI-508833

Evolvement of Grids

Historical perspective• Local Computing

– All computing resources at single site.– People move to resources to work.

• Remote Computing– Resources accessible from distance.– All significant resources still centralized.

• Distributed Computing– Resources geographically distributed.– Specialized access; largely data transfers.

• Grid Computing– Resources and services geographically distributed.– Standard interfaces; transfers of computations and data.

• Web Services and Grid Computing – Grid Services– Industry adopts Grid technology



INFSO-RI-508833

What is GRID?

“Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I.Foster)– Resources are controlled by their owners– The Grid infrastructure provides access to collaborators

A Virtual Organization is:– People from different institutions working to solve a common goal– Sharing distributed processing and data resources

Enabling People to Work Together on Challenging Projects– Science, Engineering, Medicine… - e-Science, e-Health– Public service, commerce… - e-Government, e-Business

The Grid could be the “new age” Internet– ‘[The Grid] intends to make access to computing power, scientific data

repositories and experimental facilities as easy as the Web makes access to information.’, UK PM, 2002



INFSO-RI-508833

The GRID vision

• On one hand:– Researchers/employees perform their activities

regardless of geographical location, interact with colleagues, share and access data

• On the other hand:– Scientific instruments and experiments provide huge

amount of data, incl. national databases• And in the middle:

– The Grid: networked data, processing centres and ”grid middleware” as the “glue” of resources.



INFSO-RI-508833

Grid Services

• Basic unit of computation – job• Basic unit of storage – file• Information systems – BDII, Globus-mds, R-GMA, file

catalogues, metadata catalogues• Authorization, authentication, accounting (AAA)–

based on PKI (Public key infrastructure)• Every Grid site provides basic Grid services• Advanced Grid Services: MPI jobs, Mass Storage

Facilities accessed via SRM, Fine grained AAA (VOMS, DGAS).



INFSO-RI-508833

Grid Services - schema



INFSO-RI-508833

Grid Services in gLite



INFSO-RI-508833

EGEE Partner Federations

All work in EGEE will be carried out by the 70 partners grouped in 12 federations.



INFSO-RI-508833

Goals of the EGEE project

• Goal in one sentence:– Allow scientists from multiple domains to use,

share, and manage geographically distributed resources transparently.

• The EGEE project brings together experts from over 27 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure, available to scientists 24 hours-a-day.

• The project aims to provide researchers in academia and industry with access to major computing resources, independent of their geographic location. The EGEE project will also focus on attracting a wide range of new users to the Grid.



INFSO-RI-508833

Scientific disciplines to run Grid applications

• EGEE aims to establish production quality sustained Grid services – 3000 users from at least 5 disciplines– integrate 50 sites into a common

infrastructure– offer 5 Petabytes (1015) storage

• Demonstrate a viable general process to bring other scientific communities on board Pilot New



INFSO-RI-508833

EGEE – building a production Grid for e-Science

• Operations Management Centre (OMC):– At CERN – coordination etc

• Core Infrastructure Centres (CIC)– Manage daily grid operations –

oversight, troubleshooting– Run essential infrastructure services– Provide 2nd level support to ROCs– UK/I, Fr, It, CERN, + Russia (M12)– Taipei also run a CIC

• Regional Operations Centres (ROC)– Act as front-line support for user and

operations issues– Provide local knowledge and

adaptations– One in each region – many distributed

• User Support Centre (GGUS)– In FZK – manage PTS – provide single

point of contact (service desk)– Not foreseen as such in TA, but need is

clear



INFSO-RI-508833

Components of a production Grid

• A production Grid consists of stable interoperating Grid sites (Resource centres), which enable access to Grid users from various Virtual Organizations

• Every Grid site provides basic Grid services and follows strict operational procedures.

• Monitoring allows fast detection of problems and their resolution or isolation.



INFSO-RI-508833

BG01-IPP setup

UI

- PKI X.509 certificate keys- JDL files

Terminals

enterGrid

enterGrid

enterGrid

enterGrid

UI WN

WN

WN

WN

WN

WNRB/II

CESE

BDII



INFSO-RI-508833

Structure of EGEE operations• The grid is flat, but

• Hierarchy of responsibility– Essential to scale the operation

• CICs act as a single Operations Centre– Operational oversight (grid

operator) responsibility

– rotates weekly between CICs

– Report problems to ROC/RC

– ROC is responsible for ensuring problem is resolved

– ROC oversees regional RCs

• ROCs responsible for organising the operations in a region– Coordinate deployment of

middleware, etc

• CERN coordinates sites not associated with a ROC

CIC

CICCIC

CICCIC

CICCIC

CICCIC

CICCIC

RCRC

RCRC RCRC

RCRC

RCRC

ROCROC

RCRC

RCRC

RCRCRCRC

RCRCRCRC

ROCROC

RCRC

RCRC RCRC

RCRC

RCRC

ROCROC

RCRC

RCRC

RCRC

RCRC

ROCROC

OMCOMC

RC = Resource Centre



INFSO-RI-508833

Operations monitoring maps

•In LCG-2:

• 137 sites, 34 countries

• >12,000 cpu

• ~5 PB storage

•Includes non-EGEE sites:

• 9 countries, 18 sites

•In LCG-2:

• 137 sites, 34 countries

• >12,000 cpu

• ~5 PB storage

•Includes non-EGEE sites:

• 9 countries, 18 sites



INFSO-RI-508833

Selection of Monitoring tools

GIIS Monitor GIIS Monitor graphs Sites Functional Tests

GOC Data BaseScheduled Downtimes Live Job Monitor

GridIce – VO view GridIce – fabric view Certificate Lifetime Monitor

Note: Those thumbnails are links and are clickable.



INFSO-RI-508833

Example: LHC at CERN



INFSO-RI-508833

CMS LHC Experiment



INFSO-RI-508833

Example biomedical app: gPTM3D

• One data set is – DICOM files: 100MB – 1GB– One radiological image: 20MB – 500MB

• Complex interface: optimized graphics and medically-oriented interactions

• Physician interaction is required at and inside all stepsPoorly discriminant data, pathologies, medical windowing

Interaction

RenderExplore Analyse InterpretAcquire



INFSO-RI-508833

Figures

Small body

Medium body

Large body

Lungs

Dataset

87MB

210MB

346MB

87MB

Input data

3MB18KB/slice

9.6 MB25KB/slice

15MB22KB/sclice

410KB4KB/slice

Output data

6MB106KB/slice

57MB151KB/slice

86MB131KB/slice

2.3MB24KB/slice

Tasks

169

378

676

95

StandaloneExecution

5min15s1min54s

33min11min5s

18min

36s

EGEEExecution 14 procs.

37s18s

2min30s1min15s

2min03

24s



INFSO-RI-508833

Example: The MAGIC Telescope

• Ground based Air Cerenkov Telescope

• Gamma ray: 30 GeV - TeV• LaPalma, Canary Islands

(28° North, 18° West)• 17 m diameter• operation since autumn 2003

(still in commissioning)• Collaborators: IFAE Barcelona, UAB Barcelona,

Humboldt U. Berlin, UC Davis, U. Lodz, UC Madrid, MPI München, INFN / U. Padova, U. Potchefstrom, INFN / U. Siena, Tuorla Observatory, INFN / U. Udine, U. Würzburg, Yerevan Physics Inst., ETH Zürich

Physics Goals: Origin of VHE Gamma raysActive Galactic NucleiSupernova RemnantsUnidentified EGRET sourcesGamma Ray Burst



INFSO-RI-508833

~ 10 kmParticleshower

Ground based γ-ray astronomy

~ 1o

Ch

eren

kov

ligh

t

~ 120 m

Gammaray

GLAST (~ 1 m2)

Cherenkov light Image of particle shower in telescope camera

reconstruct: arrival direction, energyreject hadron background



INFSO-RI-508833

MAGIC – Hadron rejection

• Based on extensive Monte Carlo Simulation– air shower simulation program CORSIKA– Simulation of hadronic background is very CPU consuming

to simulate the background of one night, 70 CPUs (P4 2GHz) needs to run 19200 days

to simulate the gamma events of one night for a Crab like source takes 288 days.

– At higher energies (> 70 GeV) observations are possible already by On-Off method (This reduces the On-time by a factor of two)

– Lowering the threshold of the MAGIC telescope requires new methods based on Monte Carlo Simulations



INFSO-RI-508833

BG application in SEE-GRID VO - SALUTE

The Problem: ultra-fast semiconductor carrier transport

femtosecond relaxation of hot electrons by phonon emission in presence of electric field.

Barker-Ferry equation and Monte Carlo approach• Application in nanotechnologies: innovative results for

GaAs:

collision broadening and memory effects of quantum kinetic model;

Intra-collision field effect: quantum scattering - retarding and accelerating field.

• “NP-hard” problem concerning the evolution time• Parallel and Grid implementation



INFSO-RI-508833

Wigner function

800 x 260 points

150 fs



INFSO-RI-508833

Energy relaxation process:collisional broadening

Accumulation

From 10 fs up to 250 fs



INFSO-RI-508833

BG application in ESR VO – air pollution prediction

• Under development by Tzvetan Ostromsky from IPP• Transition from HPC to Grid computing



INFSO-RI-508833

Challenges before new sites

• Install middleware and follow security and middleware upgrades in a timely fashion

• Present valuable resource to the Virtual Organizations that the site supports

• Participate in the various challenges. So far we have seen the HEP and the Biomed VO challenges, and the security challenges

• Participate in innovation efforts – development of middleware and/or grid applications

• Attract new users• The Grid is about people



INFSO-RI-508833

BG Grid support centre contact information

Contact persons: • Emanouil Atanassov,

SA1 Activity Leader,

[email protected]

• Aneta Karaivanova,

NA2 Activity Leader,

[email protected]

• Todor Gurov, Alternate EGEE SEE-ROC

and SEE-GRID manager, [email protected]

• Ivan Dimov, EGEE & SEE-GRID Project

manager for BG

[email protected]

•http://www.grid.bas.bg/

Introduction to Grid Technologies in EGEE

Documents

Transcript of Introduction to Grid Technologies in EGEE