Grid Computing @ BIFI 1

Grid computing at BIFI:Description of resources &

examples of use

seminar 8th June 2005Guillermo Losilla Anadón

Grid Computing @ BIFI 2

BIFI: grid-related projects

• EGEE ("Enabling Grids for E-sciencE")– European grid for scientific use open 24

hours-a-day

• IrisGrid– National initiative

• I4Grid

Grid Computing @ BIFI 3

EGEE: predecessors

• DataGrid– First European initiative to create a pan-European

Grid

• LCG (LHC Computing Grid)– Built to deal with the huge

computing needs required to process the data that will be produced by the LHC

taken from GridCafé ®

Grid Computing @ BIFI 4

EGEE: project structure

• Extremely hierarchized:– Three main areas organised into 11 activities:

• NA: Networking Activities (5 acts)• SA: Specific Service Activities (2 acts)• JRA: Joint Research Activities (4 acts)

– 11 partner federations

• BIFI SA1 SWE federation(Spain & Portugal)

Grid Computing @ BIFI 5

EGEE: current status

• Currently using LCG grid middleware

• EGEE grid midleware (gLite) still in development

• Some figures:

taken from GSTAT (http://goc.grid.sinica.edu.tw/gstat/) 19:01h 7th June 2005

EGEE is currently the most powerful Grid infrastructure but...

...it is not enough for LHC!!!

Grid Computing @ BIFI 6

EGEE/LCG: architecture & services

• WMS (Workload Management System)– Manages the whole process of submitting a job to the

Grid– Uses JDL (Job Description Language) for describing

jobs

• DMS (Data Management System)– Manages the storing of data in the Grid– Identification(GUID, LFN, SURL) & copy of files– RLS(Replica Location Service) & LFC(LCG File

Catalog)

Grid Computing @ BIFI 7

EGEE/LCG: architecture & services

• IS (Information System)– Provides information about the Grid resources

and their status– Compliant with the GLUE schema– MDS (Monitoring & Discovery System) vs

RGMA (Relational Grid Monitoring Architecture)

– Source of information for Monitoring & Accounting services

Grid Computing @ BIFI 8

EGEE/LCG: node types

• UI (User Interface): point of entrance to the Grid• CE (Computing Element): access point to a farm of

computing nodes• WN (Worker node): computing • SE (Storage Element): access to storage element (disk,

tape...)• RB (Resource Broker): allocation and monitorization of

jobs• MON-BOX: Information System (RGMA-based)• BDII (Berkeley DB Information Index): Information

System cache• PS (Proxy Server): proxy certificates renewals

Grid Computing @ BIFI 9

EGEE/LCG resources at BIFI

• 1 UI: ui-egee.bifi.unizar.es

• 1 CE: ce-egee.bifi.unizar.es

• 1 SE + 1 Mon-Box: se-egee.bifi.unizar.es

• 1 WN

Using RB, PS & BDII from other sites(PIC & IFAE)

Grid Computing @ BIFI 10

EGEE: security & authorization

• Most services are built on top of GSI (Grid Security Infrastructure)

Every user must own, not only an user account in a UI node, but a valid digital certificate (issued by a suported CA) to be able to use the Grid

• Authorization enabled by Virtual Organization Organizations

Every user must join a virtual organization (CMS, Atlas, Alice, LHCb, SWETEST...)

Grid Computing @ BIFI 11

Examples of use: Acess to a UI

guillermo@lxbifi08:~> ssh -X guillermo@ui-egee.bifi.unizar.es

[guillermo@ui-egee guillermo]$ grid-proxy-initYour identity: /C=ES/O=DATAGRID-ES/O=BIFI/CN=Guillermo

Losilla AnadonEnter GRID pass phrase for this identity:Creating proxy .......................................................... DoneYour proxy is valid until: Wed Jun 8 09:39:11 2005

Grid Computing @ BIFI 12

Examples of use: job submission

[guillermo@ui-egee guillermo]$ globus-job-run ce-egee.bifi.unizar.es /bin/pwd

/home/swetest011

Simplest job submission:Simplest job submission:

Complex job SubmissionComplex job Submission

[guillermo@ui-egee tests]$ more testJob.jdl

Executable = "testJob.sh";

StdOutput = "testJob.out";

StdError = "testJob.err";

InputSandbox = {"./testJob.sh"};

OutputSandbox = {"testJob.out","testJob.err"};

Grid Computing @ BIFI 13

Examples of use: job submission[guillermo@ui-egee tests]$ edg-job-list-match --vo swetest testJob.jdl

Selected Virtual Organisation name (from --vo option): swetestConnecting to host lcgrb02.ifae.es, port 7772

*************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found:

*CEId* ce00.inta.es:2119/jobmanager-lcgpbs-swetest ce01.lip.pt:2119/jobmanager-lcgpbs-swetest ce2.egee.cesga.es:2119/jobmanager-lcgpbs-swetest lcg-ce.usc.cesga.es:2119/jobmanager-lcgpbs-swetest lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-short ramses.dsic.upv.es:2119/jobmanager-pbs-swetest lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-swetest lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-swetestL***************************************************************************

Grid Computing @ BIFI 14

Examples of use: job submission[guillermo@ui-egee tests]$ edg-job-submit --vo swetest testJob.jdl

Selected Virtual Organisation name (from --vo option): swetestConnecting to host lcgrb02.ifae.es, port 7772Logging to host lcgrb02.ifae.es, port 9002

********************************************************************************************* JOB SUBMIT OUTCOME The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:

- https://lcgrb02.ifae.es:9000/6scdvJ4Y1QU8yeMjH1zBhw*********************************************************************************************[guillermo@ui-egee tests]$ edg-job-status https://lcgrb02.ifae.es:9000/6scdvJ4Y1QU8yeMjH1zBhw

*************************************************************BOOKKEEPING INFORMATION:

Status info for the Job : https://lcgrb02.ifae.es:9000/6scdvJ4Y1QU8yeMjH1zBhwCurrent Status: ScheduledStatus Reason: Job successfully submitted to GlobusDestination: lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-shortreached on: Tue Jun 7 19:56:35 2005*************************************************************

Grid Computing @ BIFI 15

Examples of use: job submission[guillermo@ui-egee tests]$ edg-job-get-output https://lcgrb02.ifae.es:9000/6scdvJ4Y1QU8yeMjH1zBhw

Retrieving files from host: lcgrb02.ifae.es ( for https://lcgrb02.ifae.es:9000/6scdvJ4Y1QU8yeMjH1zBhw )

********************************************************************************* JOB GET OUTPUT OUTCOME

Output sandbox files for the job: - https://lcgrb02.ifae.es:9000/6scdvJ4Y1QU8yeMjH1zBhw have been successfully retrieved and stored in the directory: /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw

*********************************************************************************

[guillermo@ui-egee tests]$ ll /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw/total 4-rw-rw-r-- 1 guillermo guillermo 0 jun 7 22:02 testJob.err-rw-rw-r-- 1 guillermo guillermo 168 jun 7 22:02 testJob.out

[guillermo@ui-egee tests]$ more /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw/testJob.outTue Jun 7 21:58:22 CEST 2005wn066.ific.uv.es/home/swetest002/globustmp.wn066.23526.0/

WMS_wn066_023996_https_3a_2f_2flcgrb02.ifae.es_3a9000_2f6scdvJ4Y1QU8yeMjH1zBhw

Grid Computing @ BIFI 16

Examples of use: data management

[guillermo@ui-egee tests]$ edg-gridftp-ls gsiftp://se-egee.bifi.unizar.es/storage

swetestaliceatlascmsdteamlhcbsixt

[guillermo@ui-egee tests]$ edg-gridftp-mkdir gsiftp://se-egee.bifi.unizar.es/storage/swetest/prueba

[guillermo@ui-egee tests]$ edg-gridftp-ls gsiftp://se-egee.bifi.unizar.es/storage/swetest/

prueba

Grid Computing @ BIFI 17

Examples of use: data management

$ lcg-cr -l mi_fichero_en_el_Grid-d sfn://se-egee.bifi.unizar.es/storage/swetest/test/mi_test --vo swetest file://`pwd`/mi_test

Guid: guid:5def36aa-324e-4206-a631-a56093b8b02b

$ lcg-lr --vo swetest guid:5def36aa-324e-4206-a631-a56093b8b02bLocation: sfn://se-egee.bifi.unizar.es/storage/swetest/test/mi_test

$ lcg-lr --vo swetest mi_fichero_en_el_GridLocation: sfn://se-egee.bifi.unizar.es/storage/swetest/test/mi_test

$ lcg-cp --vo swetest guid:5def36aa-324e-4206-a631-a56093b8b02b file://`pwd`/mi_fichero_recuperado

Grid Computing @ BIFI 18

Examples of use: Information System

[guillermo@ui-egee tests]$ lcg-infosites --vo swetest se

**************************************************************These are the related data for swetest: (in terms of SE)**************************************************************

Avail Space(Kb) Used Space(Kb) Type SEs----------------------------------------------------------51354616 21660820 disk se01.lip.pt11781208 134239228 disk se00.inta.es17971988 10066312 disk aker.dsic.upv.es64635064 2473188 disk lcg-se.usc.cesga.es5503300 2331648 disk se2.egee.cesga.es1000000000000 500000000000 mss castorgrid.ific.uv.es0 500000000000 mss castorsrm.ifae.es1000000000000 500000000000 mss castorsrm.ific.uv.es

[guillermo@ui-egee tests]$ ldapsearch -LLL -x -H ldap://ce-egee.bifi.unizar.es:2170 -b "mds-vo-name=bifi,o=grid"

Grid Computing @ BIFI 19

Useful links

• Grid computing at BIFI: http://bifi.unizar.es/research/computing/grid/gridcomputing.html

• EGEE project: http://public.eu-egee.org/

• LCG project: http://lcg.web.cern.ch/LCG/

• LCG-2 User Guide: https://edms.cern.ch/file/454439//LCG-2-UserGuide.html