DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 11

GLUE Schema - Status Report

DataTAG WP4 meetingBologna, 29 July 2003

Sergio Andreozzi INFN-CNAFandreozzi@cnaf.infn.it

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 22

OUTLINE

• The Information Model – open issues

• Computing resources

• Storage resources

• Monitoring extensions

• The LDAP Implementation for MDS

• Integration status

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 33

Computing resourcesIssue 1: CE viewpoint of the cluster

• The subcluster concept is suitable to provide a summary description of the nature of a cluster in term of host characteristics

• It does not properly suits brokering needs

• This leads to the current EDG constraint of one subcluster per cluster

• This is a limitation because:

• Does not accommodate the hypothesis of heterogeneous clusters

• Does not accommodate flexibility in batch system configuration

• MISSING TO EXPRESS DESCRIPTION OF NODES ASSIGNED TO A SPECIFIC QUEUE

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 44

Computing resourcesIssue 1: CE viewpoint of the cluster

CPU:PIIIRAM:0.5GB

OS:Linux

…

Headnode

gatekeeper

infoServic

e

CPU:PIIIRAM:0.5GB

OS:Linux

CPU:PIVRAM:2GBOS:Linux

CMS

CPU:PIVRAM:2GBOS:LinuxATLAS

Batch server

In this cluster, a computing element is configured with a node assigned to CMS VO.A specific software is installed in that node.

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 55

Computing resourcesIssue 1: CE viewpoint of the cluster

CPU:PIIIRAM:0.5GB

OS:Linux

…

Headnode

gatekeeper

infoServic

e

CPU:PIIIRAM:0.5GB

OS:Linux

CPU:PIVRAM:2GBOS:Linux

CMS

CPU:PIVRAM:2GBOS:LinuxATLAS

Batch server

In this cluster, a computing element is configured with a node assigned to ATLAS VO.A specific software is installed in that node.

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 66

Computing resourcesIssue 1: CE viewpoint of the cluster

ComputingElement

UniqueID : stringName : string

Info

LRMSType : stringLRMSVersion : stringGRAMVersion : stringHostName: stringGatekeeperPort : stringTotalCPUs : string

State

Status : stringTotalJobs : intRunningJobs : intWaitingJobs : intWorstResponseTime : intEstimatedResponseTime : intFreeCPUs : int

Policy

MaxWallClockTime : intMaxCPUTime : intMaxTotalJobs : intMaxRunningJobs : intPriority : int

Job

GlobalID : stringLocalID : stringLocalOwner : stringGlobalOwner : stringStatus : stringSchedulerSpecific : string

AccessControlBase

Rule : string[*]

*

*

ParticipatingHost

Host

(See Host)

Cluster

UniqueID : stringName : string

*

SubCluster

UniqueID : stringName : string<homogeneity attribute lists>

Homogeneity

DisjointedHostsSet

*

provides

assigned

this association needsto be introduced and described

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 77

Computing resourcesIssue 2: multiple entry points to a queue

• The CE Unique ID contains the contact point

• For scalability issues, in big cluster it is envisioned:

• To separate gatekeeper from batch server

• To deploy more than one gatekeeper

• This leads to the duplication of CE entities

• Needs to refine the CE definition

• Possible solution:

• refine the CE ID, (informally) from the grid entry point into a queue of a batch system, to the queue of a batch system

• Entry points are properties of the CE

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 88

Computing resourcesIssue 2: multiple entry points to a queue

Worker node

Worker node

Worker node

Worker node…

Head node

Batch server

Access node1

gatekeeper

Access node2

gatekeeper

Access noden

gatekeeper

…

queue

grisCan run on an access node, on theHead node or on another machine

e.g.: from CEUniqueID: accessnode1:2119/jobmanager-pbs-redaccessnoden:2119/jobmanager-pbs-redto CEUniqueID: headnode/jobmanager-pbs-red

(the list of accesspoints are attributes)

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 99

Storage resources

• The Grid Resource is the space being shared

• Glue SE Schema aims at provide a generalization for several Storage Resource Manager

• Glue SE Schema 1.1 is mainly based on

• experience on using disk-servers with a set of protocol to access/transfer data

• design documents related to SRM

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 1010

Storage resources

• Current deployment experience within EDG 2.0 (where SE is the EDG-SE) is providing useful feedback

• Several refinements are needed in order to deal with new use cases

• I’m working on a proposal with use cases; this will take into account most of the features discussed during last CHEP

(ref: http://www.cnaf.infn.it/~sergio/datatag/glue/v11/SE/GlueSE_revision.html)

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 1111

GLUE Schema extended for monitoring

• within DataTAG we have extended the host entity with several metrics

• these have not been proposed for inclusion in the GLUE schema

• any action on this?

DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 1212

GLUE Schema integration in VDT

• Early July, meeting with Alain Roy at CNAF

• Achievement:

• By this week in VDT 1.1.10 there will be:

• Updated GLUE Schema, extended version

• EDG CE Info provider properly configured for CONDOR LRMS