Terena conference, June 2004, Rhodes, Greece Norbert Meyer The effective integration of scientific...

Terena conference, June 2004, Rhodes, Greece

Norbert Meyer

The effective integration of scientific instruments in the

Grid

Cezary Mazurek, PSNC, Access to Grid and Digital Library Services 2


High Performance Computing and

Visualisation with the SGI Grid

for Virtual Laboratory Applications

co-funded by

State Committee for Scientific Research

and Silicon Graphics

Project No. 6 T11 0052 2002 C/05836Project No. 6 T11 0052 2002 C/05836



Project duration:R&D – December 2002 .. November 2004Deployment – 1 year

Partners:ACK CYFRONET AGH (Kraków) – LEAD PARTNER

PŁ (Łódź)

PSNC (Poznań)

TASK (Gdańsk)

WCSS (Wrocław)

ATM S.A. (Warsaw)

Silicon Graphics

Institute of Bioorganic Chemistry PAS (Poznań)

Institute of Meteorology and Water Management (Warsaw)



PIONIER paradigms

VLAB



• Building a nation wide HPC/HPV infrastructure:

• Connecting the existing infrastructure with the new testbed.

• Dedicated Application Servers

• Resource Management, User Account Management, Global Accounting

• Data access optimisation

• tape subsystems

• Access to scientific libraries

• Checkpoint restart

• kernel level / IA64 architecture

• Advanced security system

• Advanced visualisation

• Distributed & Remote visualization

• Programming environment supporting the end user

• How to simplify the process of making parallel applications

R&D activities



Architecture

APPLICATIONS

GRID M IDDLEWARE

VIRTUAL LABORATORY HIGH PERFORMANCE

VISUALISATION APPS EMERGENCY COMPUTING

CENTER

ADVANCED NETWORK INFRASRUCTURE (PIONIER) HPC/HPV INFRASTRUCTURE



„The Grids that evolve along these lines will need to exhibit all the properties, most importantly, interoperability. In this way a collection of Grids, each providing specialized facilities, will interoperate to form a single Grid at the interface level.

We believe that addressing these research priorities (with the representative balance between the pure technical and the social/business oriented issues) in the 2005 to 2010 timeframe will enable Europe to take the global lead in Grid technology and its application in the Information Society of the future” .

Next Generation Grid(s), European Grid Research 2005 - European Grid Research 2005 - 20102010Expert Group Report, ftp://ftp.cordis.lu/pub/ist/docs/ngg_eg_final.pdf

Next Generation Grids (NGG)



Virtual Laboratory

(VLab)



virtual, remote, ...virtual, remote, ...Grid-Grid-enabledenabled

VERYVERY limited accesslimited access

Main reason -Main reason - COSTSCOSTS

Main Main GOALGOAL - to make - to make commonly accessiblecommonly accessible

Added ValueAdded Value

Why Vlabs ??



• Access via Internet

• Universal architecture

• Allows to use the same approach for various types of instruments

• To keep the information

• Experiments, results

• Publications

Guidelines



• Remote usage of expensive and unique facilities

• Better utilisation

• Joint venture and on-line co-operation of scientific teams

• Shorter deadlines, faster work

• eScience – closer

• Equal chances

• Tele –work, -science

The Goal



• Access various facilities • Unique, expensive, universal architecture

• Choose the best suitable site

• Define measurement scenarios

• Digital libraries

• Collaborative work

• Accounting issues

• Enhanced security level

• On-line progress monitoring

• Platform independent

Differences between Virtual Laboratory and pure remote access



Difference between Virtual Laboratory and pure remote access – CONT.

• Advanced time reservation

• Bearing in mind the ‘human factor’



Example:

Preprocessing PostprocessingExperiment VisualisationData Results

• Allow to flexibly define the measurement scenarios, which include all experiment parts from pre-processing tasks, through the execution of experiment to post-processing

• Possibility to add new scenario elements• Defining conditions between ‘black boxes’• Clearly define the data flow• Make simpler the control process

Dynamic Measurement scenarios (DMS)



In Out

Dynamic Measurement scenarios



• Checking conditions and constrains (earlier defined inside the

‘black box’)

• Conditions between connections (e.g. OR, AND)

• Input and Output conditions

Complexity of DNM - connections



Complexity of DMS - conversions

• Made in case the OUTPUT of App1 is incompatible with INPUT

of App2

• Developed several conversion schemas

• The behaviour of Measurement parts is defined by

DMSL (ang. Dynamic Measurement Scenario Language)



Dynamic Measurement scenarios



• Scientific instrument = Grid resource• Submitting the jobs to device (e.g. to Spectrometer) as

well as to other computational systems• ‘Wider definition’ of Grid Resource

Resources in the Grid



• Defining limits for available resources• A reporting system, which takes into consideration the

type of laboratory, projects and users• Integrated with Grid Accounting

Global Accounting



... more than remote access

Testbed infrastructure

• Virtual Laboratory of Nuclear Magnetic Resonance Spectroscopy – cooperation with Institute of Bioorganic Chemistry PAS

• Virtual Laboratory of Radiotelescopy – Radioastronomy Department of Mikołaj Kopernik University



Globus

Fabric Layer

Vlab Layer(specific services)

Middleware Layer

Applications Access services

Collaborative work environmentj

Measurement Scenarios Data visualizations

Labour administration

Grid gateway

Global Vlab resourcebroker

Local resource broker

Authorization

Data Management System

MonitoringAccounting

HPC/HPV systemsScientific instruments Storage

GridFTP

Grid Information Service

GRMS

Virtual Laboratory environment Grid environment

Applications

Application A

Networking



Jobs management

Digital library

Communicationtools

Educationalarea

Userprofile

Submit new

scenario

Monitoring

Accounting

NMR database

Radio-astronomydatabase

Publicationsdatabase

Chat

Audio

Video

User information

Customizingtools

Depends on laboratory

profile

User Interface - components



Summary

Grid-enabled

Universal architecture

User-friendly

The remote facility becomes a critical resource !!



Subcomponents



THANK YOU !

[email protected]ń Supercomputing and Networking Center

http://vlab.psnc.pl

http://www.wcss.wroc.pl/pb/sgigrid/

VLAB

Terena conference, June 2004, Rhodes, Greece Norbert Meyer The effective integration of scientific...

Documents

Transcript of Terena conference, June 2004, Rhodes, Greece Norbert Meyer The effective integration of scientific...