SEE-GRID and EGEE Portal applications

M. Kozlovszky, A. Balasko

Computer and Automation Research Institute of the Hungarian Academy of Sciences

PUCOWO,Zurich, Switzerland 10-11/06/2010

www.lpds.sztaki.hu

2

Content1. SEE-GRID-SCI Tools & Applications

1. Tools1. USGIME - User specific grid infrastructure monitoring

extension 2. CWRE - Workflow Repository

2. Applications1. NMMC3D2. SRA

2. SEE-GRID2 Applications1. FEM2.5D2. EMMIL

3. EGEE Applications1. OMNeT++2. ABC, MPI-FD-FDTD, ...

3

Content1. SEE-GRID-SCI Tools & Applications

1. Tools1. USGIME - User specific grid infrastructure monitoring

extension 2. CWRE - Workflow Repository

2. Applications1. NMMC3D2. SRA

2. SEE-GRID2 Applications1. FEM2.5D2. EMMIL

3. EGEE Applications1. OMNeT++2. ABC, MPI-FD-FDTD, TINKER, ...

4

USGIME - User/application specific grid infrastructure monitoring extension of P-GRADE Portal

•Aim: Allow users of P-GRADE Portal to test: • The validity of their own certificate• Accessibity between

•Computing Elements•Storage Elements (using Remote Files and Logical File Catalog)

•Solution: portlet development with new functionalities•Functions: shows availability/compatibility of specified resources or makes query of the whole grid infrastructure (from user point of view).•Advantage: Always up-to-date information about the infrastructure for developers/end users.•Targeted: mainly end users and not grid administrators•Gridification:

•LPDS @ SZTAKI•Used special feature: Google visualization toolkit•More detailed informationhttp://wiki.egee-see.org/index.php/JRA1_Commonalities#USGIME_.28User.2Fapplication_specific_grid_infrastructure_monitoring_extension_of_P-GRADE_Portal.29

P-GRADE Portal 2.7

+ List of Computing Elements,and Storage Elements

Pre-developed workflowJob generation is done from CE-list

List of SEs

Usage scenario

Implemented GUI 1/3

Implemented GUI 2/3

Implemented GUI 3/3

•Aim: to make available workflow applications for the whole P-GRADE portal user community •Solution: Integrating P-GRADE portal with DSpace repository•Functions:

•App developers can publish their ready-to-use and half-made applications in the repository•End-users can download, parameterize and execute the applications stored in the repository

•Advantage: • Appl. developers can collaborate with appl. developers and with end-users•Portal user community members can share their WFs• Different portal user communities can share their WFs

Portal

DSpace repository

Portal

End-users

App developers

Portal

• Gridification: LPDS @ SZTAKI, WPI /US/• DSpace was developed by HP and MIT Libraries (http://www.dspace.org.)

• More detailed informationhttp://wiki.egee-see.org/index.php/JRA1_Commonalities#CWRE_.28Common_Workflow_Repository_Extension_of_P-GRADE_Portal.29

CWRE - Common Workflow Repository Extension

• DSpace View• DSpace Download• DSpace Upload

New portlets

11

Integrating P-GRADE portal with DSpace repository

DSpace Repository

Upload WF to DSpace

Download WF from DSpace

12

Content1. SEE-GRID-SCI Tools & Applications

1. Tools1. USGIME2. CWRE

2. Applications1. NMMC3D2. SRA

2. SEE-GRID2 Applications1. FEM2.5D2. EMMIL

3. EGEE Applications1. OMNeT++2. ABC, MPI-FD-FDTD, TINKER, ...

13

Seismo VO’s applications

• NMMC3D - Numerical Modeling of Mantle Convection

• MDSSP-WA – process and analysis of seismic signals

• SRA - Seismic Risk Assessment

• ELF – Earthquake Location finder

Country 1

Applications

(ELF,FPS,MDSSP-WA,NMMC3D,SDS)

Programming tool (SDS data I terators)

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Country 2 Country n...

Distributed storage and indexing of data on grid (by SDSAS upload scripts)

Country 1

Applications

(ELF,FPS,MDSSP-WA,NMMC3D,SDS)

Programming tool (SDS data I terators)

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Earthquake and seismic waveform data

Country 2 Country n...

Distributed storage and indexing of data on grid (by SDSAS upload scripts)

"We shape our buildings, and afterwards our buildings shape us”

-- Churchill

GRNET.gr

MTA SZTAKI .hu

IPP .bg

ICI .ro

TUBITAK.tr

UPT.al

UKIM.mk

UoBL.ba

RBI.hr

UOB.rs

RENAM.md

UOM.me

GRENA.ge

IIAP-NAS-RA.am

SEEFIRE + Geant Support

SEE-GRID

SeismologyVO

MeteorologyVO

EnvironmentalVO

SeismologyVO

MeteorologyVO

EnvironmentalVO

SEE-SCI

e-Infrastru

cture

= P-GRADE portal was used during development

14

NMMC3D - Numerical Modeling of Mantle Convection

• Aim: better understand the influences of parameters on the behavior of upwellings.•Function: The application solves the equations of thermal convection with a partly finite difference, partly spectral scheme.•Gridification:

•GGRI /Geodetic and Geophysical Research Institute/•LPDS @ SZTAKI•Bogazici University /Turkey/

•Important used/developed features•Application specific complex portlet structure•Embedded picture rendering service •Parameter Study type solution

•More detailed informationhttp://www.lpds.sztaki.hu/gasuc/index.php?m=7&s=16

15

NMMC3D – application profile and Workflow structure

• Fortran based binaries• No internal communication• Small input size (1MB)• Moderate large output size (100MB)• Output rendered by external application (embedded visualization)• End users are able to configure parameter• Extra parameter configuration and submission portlet for end-user • Workflow structure:

• 1 Autogenerator • 1 Parameter Study job

• 5 direct input parameters + one generated parameter• 1 binary in the PSJob

16

17

18

19

20

21

22

Example result file converted to an image

Application output

SRA - Seismic Risk Assessment• Aim:

•Application for public safety and hazards mitigation.

•Gridification:• METU /Ankara, Turkey/

•Important used/developed features•Application specific portlet •Embedded Google maps •Embedded picture rendering service •Parameter Study solution

•More detailed informationhttp://wiki.egee-see.org/index.php/SRA_gridification_guide

25

SRA: Seismic Risk Assessment

• developed as a portlet• Google-powered map

26

Content1. SEE-GRID-SCI Tools & Applications

1. Tools1. USGIME2. CWRE

2. Applications1. NMMC3D2. SRA

2. SEE-GRID2 Applications1. FEM2.5D2. EMMIL

3. EGEE Applications1. OMNeT++2. ABC, MPI-FD-FDTD, TINKER, ...

27

FEM2.5D

•Aim: Finite difference (FD) modeling (for solving partial differential equations). •Gridification:

• Miskolc University• LPDS @ SZTAKI support

•Important used/developed features• Application specific portlet for end users• Embedded picture rendering service • Parameter Study type solution

•More detailed informationhttp://www.lpds.sztaki.hu/gasuc/index.php?m=7&s=2

28

FEM 2.5D end-user GUI and workflow output

29

EMMIL : E-Marketplace Model Integrated with Logistics

•Aim: Supply chain efficiency improvement with numerical model. Three sided negotiation between:

• Buyers• Sellers• Third party logistics providers

•Gridification:•IBS /International Business School/•LPDS @ SZTAKI support

•Important used/developed features• Application specific portlets• Embedded Google maps• Embedded picture rendering service• Parameter Study type solution• Complex independent user views

•More detailed information

http://www.lpds.sztaki.hu/gasuc/index.php?m=7&s=1

30

EMMIL Workflow structure

GEN

SEQ

COLL

SEQSEQSEQ

Parameter study grid jobs

Grid job: parameter field

generator

Collector

P-GRADE Portal

31

Profile Page

Every user has its own Profile

32

Buyer: Product Management

33

Buyer: Create new Auction

34

List of available Auctions

35

3PL’s bid taking

36

Downloading Results

37

Set up the most optimal supplier chain

Log in as buyer supplier, or logistic service provider. Create auctions or take your bids

EMMIL P-GRADE Procurement Portal views

Parameter study optimization workflow is automatically generated and executed

Best buyer-seller-logistic service provider pairs discovered, result

presented in an Excel file

Download rendered images

38

EMMIL P-GRADE Procurement Portal•Buyers profile•Buyer’s product management•Buyer starts new auction•Supplier places bid•Supplier auction is closing•3PL places bid•3PL auction is closing

Results

39

Content1. SEE-GRID-SCI Tools & Applications

1. Tools1. USGIME2. CWRE

2. Applications1. NMMC3D2. SRA

2. SEE-GRID2 Applications1. FEM2.5D2. EMMIL

3. EGEE Applications1. OMNeT++2. ABC, MPI-FD-FDTD, TINKER, ...

40

OMNET++

OMNeT++ is a public-source, component-based, modular, discrete event simulation environment. •Aim: Use of grid infrastructure for computing network simulations, queuing network simulations•Gridification:

•OpenSim Ltd.•LPDS @ SZTAKI

•Important used/developed features• Application specific portlets• Parameter Study type solution• Parrot usage• Service like, easy-to-use login (non-gridsphere based authentication), captcha supported automatic user creation

•More detailed informationhttp://www.lpds.sztaki.hu/gasuc/index.php?m=6&r=4

41

OMNET++ workflow

42

Two types of OMNET portal usageOMNeT++ user portal•User accounts exist for 1 week•Can handle NED files that use INET and Queuing module sets from OMNeT++•No binary come from end user

•Portal does grid operations with one fixed certificate

OMNeT++ developer portal•Permanent user accounts•Can handle custom OMNeT++ modules and NED files that handle such modules•Binaries come from end users

•Users are authenticated based on their personal certificates

43

OMNeT++ portal

44

•ABC - Solving the Schrodinger equation for triatomic systems using time independent method (Department of Chemistry, University of Perugia, Italy)•RWavePR - Solving the Schrodinger equation for triatomic systems using time dependent method (Department of Chemistry, University of Perugia, Italy)•MPI-FD-FDTD - Numerical modeling of ElectroMagnetic field distribution in human tissues (University of Manchester, United Kingdom)•TINKER Conformer Generator - Molecular modeling for QSAR studies for drug development with TINKER external library (LPDS @ SZTAKI)•…

Other applications/services examples based on P-GRADE Portal

45

Thank you for your attention!Questions?