1

2010-04-27

EDGIEDGIEEuropeanuropean DDesktopesktop GGridrid IInitiativenitiativeEDGIEDGIEEuropeanuropean DDesktopesktop GGridrid IInitiativenitiative

Advanced EDGI technologies

Jozsef KovacsComputer and Automation Research Institute,

Hungarian Academy of Sciences (MTA SZTAKI)Budapest, Hungary

kovacs.jozsef@sztaki.mta.hu

EDGI is supported by the FP7 Capacities Programme under contract nr RI-261556

ContentsContentsContentsContents

•MetaJobMetaJob

Submission of job collection in EDGISubmission of job collection in EDGI

•GBACGBAC

Virtualization on the BOINC resourcesVirtualization on the BOINC resources

•AtticAttic

A P2P file system supporting the EDGI A P2P file system supporting the EDGI infrastructureinfrastructure

•SpeQuloSSpeQuloS

Dynamic extension of BOINC with CloudsDynamic extension of BOINC with Clouds

2

DG site

Volunteer/Campus

Resources

Software architecture developed by EDGISoftware architecture developed by EDGISoftware architecture developed by EDGISoftware architecture developed by EDGI

UNICORE grid

gLite grid

3GBridge

attic

monitor

AR

UserIF

BridgeIF

Attic FS DG client

attic

Monitor

UI

UI

DGPro-ject

submit

inspect

upload

select

submit

cloud

DG client

attic

ARC grid

submit

Service Grid Desktop Grid EDGI bridge components

A high level scenario for grid usersA high level scenario for grid usersA high level scenario for grid usersA high level scenario for grid users

• Step 1: Step 1: Select Select an application you want to execute on Gridsan application you want to execute on Grids

•Use any from the EDGI repositoryUse any from the EDGI repository

•Bring your own applicationBring your own application

• Step 2: Step 2: EDGI project EDGI project performs the necessary preparations for performs the necessary preparations for you and as a result the application appears in the EDGI you and as a result the application appears in the EDGI application repositoryapplication repository

•application portingapplication porting

• registering the application in the AR and in several DGsregistering the application in the AR and in several DGs

• infrastructure setup for your access (by the operators of the infrastructure setup for your access (by the operators of the connecting grids)connecting grids)

• Step 3: Go to our Step 3: Go to our EDGI AR EDGI AR and collect info for your and collect info for your submissionsubmission

• Step 4: Prepare your Step 4: Prepare your input filesinput files, create and submit , create and submit jdljdl

• Step 5: Query the Step 5: Query the statusstatus and download the and download the outputsoutputs

4

Step 3 (cont.): Collect informationStep 3 (cont.): Collect informationfrom the EDGI AR (files)from the EDGI AR (files)Step 3 (cont.): Collect informationStep 3 (cont.): Collect informationfrom the EDGI AR (files)from the EDGI AR (files)

5

Step 4, 5: submit jdl and get results Step 4, 5: submit jdl and get results Step 4, 5: submit jdl and get results Step 4, 5: submit jdl and get results

• Create your my.jdl (as usual):Create your my.jdl (as usual):

Executable = "dsp";Executable = "dsp";

Arguments = "-f 22 -i 22 -p 723 -n pools.txt";Arguments = "-f 22 -i 22 -p 723 -n pools.txt";

InputSandbox = { InputSandbox = {

"gsiftp://dev17-portal.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/"gsiftp://dev17-portal.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp" , dsp" ,

"pools.txt","pools.txt",

OutputSandbox = {"cost.txt"};OutputSandbox = {"cost.txt"};

SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo";SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo";

• Submit, Status query, Output downloading can be performed by Submit, Status query, Output downloading can be performed by the well-known gLite commands: the well-known gLite commands: glite-wms-job-submitglite-wms-job-submit, glite-wms-, glite-wms-job-status, glite-wms-job-logging-info, job-status, glite-wms-job-logging-info, glite-wms-job-outputglite-wms-job-output, etc., etc.

6

METAJOBMETAJOBSubmission of job collection in EDGISubmission of job collection in EDGI

7

DG site

Volunteer/Campus

Resources

Software architecture developed by EDGISoftware architecture developed by EDGISoftware architecture developed by EDGISoftware architecture developed by EDGI

UNICORE grid

gLite grid

3GBridge

attic

monitor

AR

UserIF

BridgeIF

Attic FS DG client

attic

Monitor

UI

UI

DGPro-ject

submit

inspect

upload

select

submit

cloud

DG client

attic

ARC grid

submit

Unfolding Huge

number of jobs

Step 4b: create and submit jdlStep 4b: create and submit jdlStep 4b: create and submit jdlStep 4b: create and submit jdl

• Create your jdl:Create your jdl:

Executable = "dsp";Executable = "dsp";

Arguments = "-f 22 -i 22 -p 723 -n pools.txt";Arguments = "-f 22 -i 22 -p 723 -n pools.txt";

InputSandbox = { InputSandbox = {

"gsiftp://dev17-portal.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp" , "gsiftp://dev17-portal.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp" ,

"pools.txt","pools.txt",

"_3gb-metajob-dsp-10000" };"_3gb-metajob-dsp-10000" };

OutputSandbox = {"cost.txt"};OutputSandbox = {"cost.txt"};

SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo";SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo";

• Submit:Submit:

glite-wms-job-submit -o idglite-wms-job-submit -o id edgi-metajob-10000.jdl edgi-metajob-10000.jdl

9

MetaJob definition as Extra input file

MetaJob: input filesMetaJob: input filesand metajob definitionand metajob definitionMetaJob: input filesMetaJob: input filesand metajob definitionand metajob definition

• Upload your individual input files to web server:Upload your individual input files to web server:

•http://somewhere.com/pools1.txthttp://somewhere.com/pools1.txt

•……

•http://somewhere.com/pools10000.txthttp://somewhere.com/pools10000.txt• Create the description of your metajob:Create the description of your metajob:

%Comment pools1.txt%Comment pools1.txt

Arguments = "-i 0 -n pools.txt -f 22 -p 723“Arguments = "-i 0 -n pools.txt -f 22 -p 723“

Input = pools.txt=http://somewhere.com/pools1.txt=7b7eb86bf50c58cbf92dc12ff5adf7f4=9652Input = pools.txt=http://somewhere.com/pools1.txt=7b7eb86bf50c58cbf92dc12ff5adf7f4=9652

QueueQueue

……

……

%Comment pools10000.txt%Comment pools10000.txt

Input = Input = pools.txt=http://somewhere.com/pools10000.txt=7b7eb86bf50c58cbf92dc12ff5adf7f4=9652pools.txt=http://somewhere.com/pools10000.txt=7b7eb86bf50c58cbf92dc12ff5adf7f4=9652

QueueQueue

10

MetaJob: Query the status and loggingMetaJob: Query the status and loggingMetaJob: Query the status and loggingMetaJob: Query the status and logging

11

Location of the status description

MetaJob: Query the detailed statusMetaJob: Query the detailed statusMetaJob: Query the detailed statusMetaJob: Query the detailed status

12

Metajob handling rules

Actual status of jobs

Location of the mapping

MetaJob: Download the resultsMetaJob: Download the resultsMetaJob: Download the resultsMetaJob: Download the results

13

•Download result as usual:glite-wms-job-output -i id --dir outputs

./outputs/cost.txt

•Extract it:tar zxvf cost.txt ./outputs/<subjobid1>/cost.txt./outputs/<subjobid2>/cost.txt…./outputs/<subjobid10000>/cost.txt

•See the mapping between your individual job definition and the jobids (which gives the

name of directories storing the output files of your app)

one subjob id

GBACGBACVirtualization on the BOINC resourcesVirtualization on the BOINC resources

14

Virtualisation on the worker nodes…Virtualisation on the worker nodes…Virtualisation on the worker nodes…Virtualisation on the worker nodes…

• Current situationCurrent situation

• Only registered applications can be executed on BOINCOnly registered applications can be executed on BOINC

• Applications need to be ported to BOINCApplications need to be ported to BOINC

• Applications need to be validatedApplications need to be validated

• Aim of virtualisation/sandboxing:Aim of virtualisation/sandboxing:

• To execute application under BOINC To execute application under BOINC without without registrationregistration

• To execute an application on a BOINC client machineTo execute an application on a BOINC client machine in in a secure waya secure way

• To do this To do this withoutwithout any any portingporting effort in a very similar effort in a very similar way as an application is submitted and executed on way as an application is submitted and executed on Linux, for example.Linux, for example.

• Execute applications that are Execute applications that are not registered at the not registered at the BOINC project BOINC project and that were not validated for BOINCand that were not validated for BOINC

• Majority of Desktop Grid resources are Windows based Majority of Desktop Grid resources are Windows based (68.9%) - however majority of scientific applications (68.9%) - however majority of scientific applications run on Linux...run on Linux...

15

Virtualisation by GBAC Virtualisation by GBAC for Desktop Grids (BOINC)for Desktop Grids (BOINC)Virtualisation by GBAC Virtualisation by GBAC for Desktop Grids (BOINC)for Desktop Grids (BOINC)

16

Worker machine (Windows)

(SZTAKI) Desktop Grid Server

GBAC

client

Linux OS

APP

Worker machine

client

…..

1

2

3

4

EDGIDemoBOINC project

…

OverviewOverview

3G Bridge

…

BOINC Client

GBAC

Application and Inputs

BOINC Client

GBAC

Application and Inputs

Application and Inputs

1. A User submits her application and inputs via gLite.

2. 3G Bridge detects that the application is “legacy” (not BOINC native) and redirects the binaries and inputs to the GBAC native BOINC application at EDGIDemo.

3. Clients (who have VirtualBox installed) will download the BOINC native GBAC application with the submitted application and its inputs.

4. GBAC starts a Linux Virtual Machine (using VirtualBox).

5. GBAC copies the applications and inputs into the Virtual Machine.

6. The application is executed in the Linux VM.7. The result is fetched from the VM by GBAC.8. The VM is shut down and discarded.9. GBAC finishes and the result is returned to

EDGIDemo from the Client.10.The results are returned to gLite through

3G Bridge from EDGIDemo.11.

Desktop Grid resource

BOINC Client VirtualBox

GBACVirtual Machine

BOINC Server

Application and inputs

Application and inputs

Output filesOutput files

G-Lite

18

Details: job execution on a virtualized DG resourceDetails: job execution on a virtualized DG resource

2. Clients (with VirtualBox installed) will download the BOINC native GBAC application with the submitted application and its inputs.

1. A user submits an Application with inputs through gLite to BOINC. 3. GBAC starts a Linux Virtual Machine (using VirtualBox).4. GBAC copies the applications and inputs into the Virtual Machine.5. The application is executed in the Linux VM.

7. The VM is shut down and discarded.8. GBAC finishes and the result is returned to BOINC from the Client.

9. The result is returned to gLite through 3G Bridge from the BOINC Server.

…

6. The result is fetched from the VM by GBAC.

GBAC: VM Guest and requirementsGBAC: VM Guest and requirementsGBAC: VM Guest and requirementsGBAC: VM Guest and requirements

• Only Linux supportOnly Linux support

•Licensing issues with WindowsLicensing issues with Windows

• 32 bit applications preferred32 bit applications preferred

•Linux image is 32 bit, later 64 bit one will be Linux image is 32 bit, later 64 bit one will be addedadded

• VM is a standard Linux systemVM is a standard Linux system

•Same requirements as for applications to be Same requirements as for applications to be executed in a Gridexecuted in a Grid

•Static linking preferred, all non standard libraries Static linking preferred, all non standard libraries must be bundled, etcmust be bundled, etc. .

19

VirtualBox and BOINC Manager and Virtual Machine VirtualBox and BOINC Manager and Virtual Machine booting via GBAC booting via GBAC VirtualBox and BOINC Manager and Virtual Machine VirtualBox and BOINC Manager and Virtual Machine booting via GBAC booting via GBAC

20

NOTE: This screenshot of GBAC is using a special “DEBUG” mode for demonstration purposes.

Normally no windows are presented at all and the VM always runs in the background (in so-called “headless mode” ).

ATTICATTICA P2P file system supporting the EDGI A P2P file system supporting the EDGI

infrastructureinfrastructure

21

Data handling under BOINCData handling under BOINCData handling under BOINCData handling under BOINC

BOINC ClientApplication

Project task

localstorage

Computing resource

projectdata

Volunteer PC

Web Server

Input

Output

Retrieve WorkUnit

Upload Handler

EDGI – Data Access

= Einstein@Home data mirroring locations

BOINCScheduler

Task

Decentralization of data centersDecentralization of data centersDecentralization of data centersDecentralization of data centers

23

P2P Cloud

DG ClientApplication

(XtremWeb/BONIC)

Project task

localstorage

Computing resource

projectdata

Management service

Volunteer PC

Project server

Inpu

t

Input

LFN1

LFN1

LFN1

1

2

3

EDGI – Data Access

24

PublishingPublishingPublishingPublishing

Publisher

Scheduler

DataLookupServer

DC

attic://voldemort.cs.cf.ac.uk:7000/data/<ID>

XML message (over HTTP)

DC

DC

Periodically query for replication requests

DataCenter

Register as replica; added to locations list

attic://voldemort.cs.cf.ac.uk:7000/data/<ID>

EDGI – Data Access

25

PublishingPublishingPublishingPublishing

Publisher

Scheduler

DC

attic://voldemort.cs.cf.ac.uk:7000/data/<ID>

XML message (over HTTP)

DC

DC

Periodically query for replication requests

DataCenter

Register as replica; added to locations list

attic://voldemort.cs.cf.ac.uk:7000/data/<ID>

PublisherAny entity that publishes a file

DataLookupServer

EDGI – Data Access

SchedulerKeeps track of managing number of replicas for a new request.

DataCenterCaches data on the network.

DataLookupServerManages number of replicas for a new request and responding to cache requests

DG site

Volunteer/Campus

Resources

Attic in the EDGI infrastructureAttic in the EDGI infrastructureAttic in the EDGI infrastructureAttic in the EDGI infrastructure

UNICORE grid

gLite grid

3GBridge

attic

monitor

AR

UserIF

BridgeIF

DG client

attic

Monitor

UI

Attic UI

DGPro-ject

submit

inspect

upload

select

submit

cloud

DG client

attic

ARC grid

submit

DC PUB LS

lfn

SpeQuloSSpeQuloSDynamic extension of BOINC with CloudsDynamic extension of BOINC with Clouds

27

Motivation for using cloudsMotivation for using cloudsMotivation for using cloudsMotivation for using clouds

• Desktop GridDesktop Grid

• Variable amount of resources, volatility, Variable amount of resources, volatility, unpredictability, unannounced departureunpredictability, unannounced departure

• Low QoS compare to classical DCI Low QoS compare to classical DCI

• Tail Effect Tail Effect

• We define QoS as a level of confidence in Bag of We define QoS as a level of confidence in Bag of Task (BoT) execution.Task (BoT) execution.

• Question: how do we provide QoS to users given the Question: how do we provide QoS to users given the dynamism and volatility of the computing dynamism and volatility of the computing resources ?resources ?

• Intrinsic approach : improve DG scheduler for QoS Intrinsic approach : improve DG scheduler for QoS ability ability

• Extrinsic approach : provide additional dedicated Extrinsic approach : provide additional dedicated computing resourcescomputing resources

28

• SolutionSolution

• Allow users to express QoS needs for their BoTAllow users to express QoS needs for their BoT

• Provision resources from Cloud to satisfy these needsProvision resources from Cloud to satisfy these needs

SpeQuloS: a middleware for QoS to SpeQuloS: a middleware for QoS to Desktop GridsDesktop Grids

SpeQuloS: a middleware for QoS to SpeQuloS: a middleware for QoS to Desktop GridsDesktop Grids

29

When to trigger QoS ?When to trigger QoS ?When to trigger QoS ?When to trigger QoS ?

Estimates the completion Estimates the completion time :time :•Without the tailWithout the tail

•With the tailWith the tail

•With QoSWith QoS

Triggers QoSTriggers QoS• If there is creditsIf there is credits

•At 90% of the BoT At 90% of the BoT executionexecution

• if there is an expected if there is an expected benefitbenefit

RI-261556 WP1version: 02

30

Thank you for your attention…Thank you for your attention…Thank you for your attention…Thank you for your attention…

Contact:Contact:

Jozsef Kovacs (Jozsef Kovacs (smith@sztaki.hu))

Project website:Project website:

www.edgi-project.eu

Acknowledgements:Acknowledgements:

EDGI EU FP7 project (EDGI EU FP7 project (RI-261556)RI-261556)

31