The Interactive European Grid Project

The Interactive European Grid Project

Paul HeinzlreiterGUP, University Linz

CoreGrid Summer School, Budapest, 7.9.2007

CoreGrid Summer School, Budapest, 7.9.2007 2

Outline

General Project Overview Research Activities Infrastructure Applications


i2g Partners


i2g History

Precursor:Crossgrid Project

First Project to deliver Interactivity for Grids

i2g improves:Key ComponentsIntegrationAdaptation to new MiddlewareNew applications


i2g Aims

Timeframe: May 2006 – April 2008 Interactivity User-Friendliness MPI Support Support of Demanding Applications Integration of all Components Provision of a Good User Experience Interoperability (with EGEE / gLite)


i2g Joint Research Activity

Interactivity & VisualizationResource BrokerInteractive ConnectionsVideo Streaming

CrossBrokerReduced Job Startup Times

Graphical User InterfaceJob ControlData HandlingJob Output

MPI SupportIntra-Site and Inter-Site

Active Security


(i2)glogin

Workernode

Workernode

Workernode

Workernode

Workernode

Workernode

Workernode

Workernode

Workernode

Workernode

GatekeeperGatekeeper

on the Grid

ClientClient

gloginPoint ofContact

job submit

glogin’

fork

socket

glogin’’

fork

inherited socket

return

return

Problem: GASS cache

interactivebidirectional connection

trafficforwarding


GVid

vtk

App

MP4 Encoder

Event Decoder

TCPTransport

SocketTransport

FileTransport

PipeTransportre

mo

teG

lut

Ap

plic

atio

nG

Vid

SD

LO

utpu

tclie

nt

MP4 Decoder

Event Encoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

Grid UI

glogin gloginGSS secured $ grid-proxy-init

$ glogin -x –C altix1


GUI – Migrating Desktop

User-Friendly Platform Independent

Java Running Sequential & MPI

Jobs Roaming Access Server

interfaces with the UI Data Management Easily Extendable

Plugin Mechnism for new Applications

Extension for Interactive Remote Visualization


GUI – Migrating Desktop


SchedulingAgent

ResourceSearcher

ApplicationLauncher

Condor-G DAGMan

CE

WN WN

LCG

CE

WN WN

gLite

MigratingDesktop

InformationIndex

ReplicaManager

CrossBroker

Interactive Job BrokerAutomatic Job Managementfor Parallel Applications

Resource SearchingJob ConditioningLaunching, Monitoring, RetryResult Retrieval

Workflows, Interactive & Batch Jobs, MPI Support

JDL ExtensionsCompatible

Best Effort Approachfor Failures / ProblemsImproved Job Startup Time


MPI Support

OpenMPIJoint Effort / OSSBest of FT-MPI, LA-MPI, LAM-MPI, PACX-MPI

PACX-MPIInter-Cluster / SiteSupports Vendor-MPI

MPI-StartShell ScriptsAbstraction LayerWMS / Schedulers and MPI-ImplementationsFlexibility (relocateable Shellscripts)

PACX-MPI

LAM/MPI

LA-MPI

FT-MPI

MPIImplementations Scheduler

WMS (CrossBroker)

MPI-STARTPACX-MPI

Vendor MPI

Application

Intra Cluster Communication

Inter Cluster Communication

PACX-MPI MPI-START


Active Security

Security measures for interactive frameworkActive Security Tools:

Track Grid InteractionsRespond to Unwanted Security Events

Extended Security Monitoring & TrackingCoordination among Security Tools & Sites

Architecture Divided in 2 LayersMonitoring & Control Layer

Security Monitoring & Control EngineAnalysis Layer

Alert AnalysisAlert Correlation Engine

R-GMACommunication Layer


Integration

Most Advanced: Fusion Application – Integrates Components from JRA

MPI SupportCrossbrokerVisualizationVideostreamingGUI SupportInteractivity


Interactivity & Visualization

i2glogin GVid CrossBroker Migrating

Desktop Full Integration + MPI Support

Job Submission Services

CrossBroker

Logging&

Bookkeeping

Roaming Access Server

Computing Element

LRMS

Gatekeeper

JDL

Migrating Desktop

,

WorkerNode

gLogin

Simulation +visualisation

WorkerNode

vtk Ap

p

MP4 Encoder

Event Decoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

remo

teGlu

tA

pp

lica

tion

GV

id

Java Video

Pla

yer

MP4 Decoder

Event Encoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

Job Submission Services

CrossBroker

Logging&

Bookkeeping

Roaming Access Server

Computing Element

LRMS

Gatekeeper

JDLJDL

Migrating Desktop

,,

WorkerNode

gLogin

Simulation +visualisationSimulation +visualisation

WorkerNode

vtk Ap

p

MP4 Encoder

Event Decoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

remo

teGlu

t

MP4 Encoder

Event Decoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

remo

teGlu

tA

pp

lica

tion

GV

id

Java Video

Pla

yer

MP4 Decoder

Event Encoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

Java Video

Pla

yer

MP4 Decoder

Event Encoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

Java Video

Pla

yer

MP4 Decoder

Event Encoder

TCPTransport

SocketTransport

FileTransport

PipeTransport

Java Video

Pla

yer

MP4 Decoder

Event Encoder

TCPTransport

SocketTransport

FileTransport

PipeTransport


Integration: Fusion Application

Application is integrated in Migrating Desktop

Communication Channel to MPI Master over the Roaming Access Server

Communication MD – RAS can also go over TCP/IP


The i2g Infrastructure

12 Sites 430 CPU Cores 21 TB of Storage Multiple

Architectures i2g Middleware

Componentsavailable forApplications


The i2g Infrastructure (Production)


i2g Applications

Plasma Fusion Simulation & Visualization First Images of the Universe Brain Diagnostic Control of Pollution …


Interactive Plasma Fusion Simulation

Most Advanced Application

MPI-Parallel Simulation Remote Visualization Full Interaction (GUI,

Mouse) GUI Integration

Distinguished as best Application Demo at OGF in Manchester


Fusion Application in MD


First Images of the Universe

Simulation of the Measurements of the Planck Satellite to be launched in 2008.

Model ValidationRadiation Values should have Gaussian DistributionIf not, Model Error

Calculating Autocorrelations:

Sky gets divided100000 Simulations, 5 Minutes each

Output gets visualized


Brain Diagnostic

Developing and Testing new Diagnostic Methods for Brain Images

Requires large DatasetsDICOM ImagesCorresponding Brain MapsFinal Diagnostic Map

E.g. Perfusion Studies (Stroke Diagnosis)

Visualization and Annotation


Control of Pollution

Developing of a Model Suite to analyze the Evolution of Pollution Clouds.

Compute Intensive Models

Interaction enables steering of Simulation Parameters

Impact Analysis MPI-Parallel Simulation


i2g Offer

Support for external developers and users’ application

s

The Interactive European Grid project helps application developers in adapting their applications to work in a Grid environment as well as in parallelization of these applications. If you would like us to consider running your application in the int.eu.grid infrastructure or creating a new VO, please contact:

Isabel Campos IFCA-CSIC*, Santander, Spain

email: iscampos AT ifca.unican.es

Gridinfrastructu

re

The Project helps to integrate new clusters to the int.eu.grid infrastructure as well as to establish agreements between research communities and infrastructure providers.

If you would like to join the int.eu.grid infrastructure with your cluster, please contact:

Project support teamemail: i2g.support AT lip.pt

Education

Tutorials: for Grid beginners, Grid site administrators, application developers; some tutorials on MPI and other Grid specific.

If you would like to join the tutorial organized at our site or to organize our tutorial at your facilities, please contact:

Celso Martínez RiveroIFCA-CSIC,

Santander, Spainemail: mrivero AT

ifca.unican.es

Seminars for groups of interested people or consultancy.

If you need a consultancy or you are interested in taking part in a seminar on Grid technology, please contact:

Project Office Santander, Spain

email: inteugrid-office AT ifca.unican.es

specifying the subject, product or problem.

Disseminative

materialsGeneral information about the Project – brochures, posters, etc. General int.eu.grid presentation.

If you are interested in receiving them, please contact:

Robert PajakCYFRONET*, Kraków, Poland

email: r.pajak AT cyfronet.krakow.pl


New Applications?

http://dissemination.interactive-grid.eu/how_to_use_grid

Description of interactive tools Use our infrastructure for your application Online user questionaire http://dissemination.interactive-grid.eu/how_to_

use_grid/user_questionnaire/User Data formOur offer (infrastructure, tools, …)


i2g Information & Contact

http://www.interactive-grid.eu http://dissemination.interactive-grid.eu

Brochures, Posters, Information ... Contact Persons:

(New) Applications:Isabel Campos; [email protected](New) Infrastructure:[email protected]:Celso Martinez Rivero; [email protected]:Project Office; [email protected]

The Interactive European Grid Project

Documents

Transcript of The Interactive European Grid Project