Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal

P. Kacsuk*, G. Sipos* and T. Kiss***MTA SZTAKI** Univ. of Westminster


INFSO-RI-508833

Properties of the P-GRADE Portal

• What is P-GRADE portal?– General purpose, graphical, workflow-oriented Grid portal– Supports the development and execution of workflow-based Grid

applications– Components of the workflows can be

Sequential jobs Parallel jobs (MPI, PVM) Legacy code (GEMLCA) services

– Easy porting of legacy and other applications to the Grid– Enables the exploitation of two levels of parallelism

intra-job inter-job

– Enables to control and observe the execution of the workflow applications

– Enables fault-tolerant workflow execution


INFSO-RI-508833

Properties of the P-GRADE Portal

• Enables application-specific extensions by new portlets (E-GRID, Traffic simulation Grid, etc.)

• Hides the low-level Grid details • Enables the access of every important feature of the

underlying Grid, e.g.: – brokering service– information service, etc.

• Can serve as a bridge between different Grids to solve the Grid interoperability problem at the workflow level:– Enables easy porting of legacy and other applications

between various Grids (e.g. from LCG to gLite, from GT2 to GT4, etc.)

– Hides middleware changes for the end-user– Enables parallel execution of a workflow in several Grids


INFSO-RI-508833

Proxy management

P-GRADE portal in a nutshell

Grid resources management

Workflow creation

Job mapping to Grid resources

Workflow management and

execution visualization


INFSO-RI-508833

Two levels of parallelism by the P-GRADE workflow

• Semantics of the workflow enables two levels of parallelism:

The job can be a parallel program

– Parallel execution inside a workflow node– Parallel execution among workflow nodes

Multiple jobs can run parallel

• The P-GRADE Portal workflow concept enables the efficient parallelization of complex problems

www.cpc.wmin.ac.uk/GEMLCAwww.cpc.wmin.ac.uk/GEMLCA

Grid Execution Management for Legacy

Code Applications

OGSA-DAIIntegration with P-GRADE

Tamas Kukla and Tamas KissCentre for Parallel Computing

University of [email protected], [email protected]


OGSA-DAI portlets

4 portlets implemented

1. Data Service Manager

2. Database Browser

3. Query Manager

4. Data Manipulation


Data Service Manager portlet

Add/remove OGSA-DAI services

List available resources

of a service

List available

tables

Select and describe

tables


Database Browser portlet

Select service /

resource / table

Display the

selected table or XML

resource

Order by the

selected column


Query Manager portletDisplay results on screen

Run query on selected database

See next slide

Display results

Display results


Query Manager portletDeliver results to file

Data can be sliced up

and compressed

before delivery

Define target

location

Define query

statement


Data Manipulation portletManipulate data of selected database

Select database

Define update

statement

See next slide

Update


Data Manipulation portletDeliver results from file to database

Define target table

Define files to be delivered

Extracts and

merges

Select target

service / resource


A possible user scenario

• The new OGSA-DAI portlet allows to browse an OGSA-DAI database and deliver query results to a set of files.

• These files can be used in normal or parameter study workflows as input ports.

• The workflow output can be converted and transferred back to the database.

1. Select the data service and resource

2. Run a query and deliver results to a (set of) file(s)

3. Run the (parameter study) workflow on the data-sets

4. Transfer the results back into an OGSA-DAI database


Extended PG-Portal

DataBrowserWorkflow

Step1: Select the OGSA-DAI service and DataBase resource with the DataBrowser

OGSA-DAIservice

Oracle DataBaseresource

User

eXist XMLDataBaseresource

RequestList of resources

Request

List of resources

Storage resource (local or remote)


Step1: Select the OGSA-DAI service and DataBase resource with the DataBrowser

Select service and resource


Extended PG-Portal

DataBrowserWorkflow

Step2: Run a query and deliver results to a (set of) file(s)

OGSA-DAIservice


User


Query Results

Query

ResultsResults

Query



Step2: Run a query and deliver results to a (set of) file(s)

Define query

statement

Deliver results to a set of files


Extended PG-Portal

DataBrowserWorkflow

Step3: Run (parameter study) workflow (this step does not use the OGSA-DAI portlets!)

OGSA-DAIservice


User


Create workflow Workflow

output


Input data


Extended PG-Portal

DataBrowserWorkflow

Step4: Transfer results back into an OGSA-DAI database

OGSA-DAIservice


User


Workflow output


Workflow output

Workflow output

Confirmation

Confirmation

Confirmation Request


Step4: Transfer results back into an OGSA-DAI database

Define target

databaseDefine target table

Define files to be transferred

and converted

Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal

Documents

Transcript of Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal