DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 11
GLUE Schema - Status Report
DataTAG WP4 meetingBologna, 29 July 2003
Sergio Andreozzi [email protected]
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 22
OUTLINE
• The Information Model – open issues
• Computing resources
• Storage resources
• Monitoring extensions
• The LDAP Implementation for MDS
• Integration status
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 33
Computing resourcesIssue 1: CE viewpoint of the cluster
• The subcluster concept is suitable to provide a summary description of the nature of a cluster in term of host characteristics
• It does not properly suits brokering needs
• This leads to the current EDG constraint of one subcluster per cluster
• This is a limitation because:
• Does not accommodate the hypothesis of heterogeneous clusters
• Does not accommodate flexibility in batch system configuration
• MISSING TO EXPRESS DESCRIPTION OF NODES ASSIGNED TO A SPECIFIC QUEUE
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 44
Computing resourcesIssue 1: CE viewpoint of the cluster
CPU:PIIIRAM:0.5GB
OS:Linux
…
Headnode
gatekeeper
infoServic
e
CPU:PIIIRAM:0.5GB
OS:Linux
CPU:PIVRAM:2GBOS:Linux
CMS
CPU:PIVRAM:2GBOS:LinuxATLAS
Batch server
In this cluster, a computing element is configured with a node assigned to CMS VO.A specific software is installed in that node.
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 55
Computing resourcesIssue 1: CE viewpoint of the cluster
CPU:PIIIRAM:0.5GB
OS:Linux
…
Headnode
gatekeeper
infoServic
e
CPU:PIIIRAM:0.5GB
OS:Linux
CPU:PIVRAM:2GBOS:Linux
CMS
CPU:PIVRAM:2GBOS:LinuxATLAS
Batch server
In this cluster, a computing element is configured with a node assigned to ATLAS VO.A specific software is installed in that node.
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 66
Computing resourcesIssue 1: CE viewpoint of the cluster
ComputingElement
UniqueID : stringName : string
Info
LRMSType : stringLRMSVersion : stringGRAMVersion : stringHostName: stringGatekeeperPort : stringTotalCPUs : string
State
Status : stringTotalJobs : intRunningJobs : intWaitingJobs : intWorstResponseTime : intEstimatedResponseTime : intFreeCPUs : int
Policy
MaxWallClockTime : intMaxCPUTime : intMaxTotalJobs : intMaxRunningJobs : intPriority : int
Job
GlobalID : stringLocalID : stringLocalOwner : stringGlobalOwner : stringStatus : stringSchedulerSpecific : string
AccessControlBase
Rule : string[*]
*
*
ParticipatingHost
Host
(See Host)
Cluster
UniqueID : stringName : string
*
SubCluster
UniqueID : stringName : string<homogeneity attribute lists>
Homogeneity
DisjointedHostsSet
*
provides
assigned
this association needsto be introduced and described
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 77
Computing resourcesIssue 2: multiple entry points to a queue
• The CE Unique ID contains the contact point
• For scalability issues, in big cluster it is envisioned:
• To separate gatekeeper from batch server
• To deploy more than one gatekeeper
• This leads to the duplication of CE entities
• Needs to refine the CE definition
• Possible solution:
• refine the CE ID, (informally) from the grid entry point into a queue of a batch system, to the queue of a batch system
• Entry points are properties of the CE
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 88
Computing resourcesIssue 2: multiple entry points to a queue
Worker node
Worker node
Worker node
Worker node…
Head node
Batch server
Access node1
gatekeeper
Access node2
gatekeeper
Access noden
gatekeeper
…
queue
grisCan run on an access node, on theHead node or on another machine
e.g.: from CEUniqueID: accessnode1:2119/jobmanager-pbs-redaccessnoden:2119/jobmanager-pbs-redto CEUniqueID: headnode/jobmanager-pbs-red
(the list of accesspoints are attributes)
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 99
Storage resources
• The Grid Resource is the space being shared
• Glue SE Schema aims at provide a generalization for several Storage Resource Manager
• Glue SE Schema 1.1 is mainly based on
• experience on using disk-servers with a set of protocol to access/transfer data
• design documents related to SRM
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 1010
Storage resources
• Current deployment experience within EDG 2.0 (where SE is the EDG-SE) is providing useful feedback
• Several refinements are needed in order to deal with new use cases
• I’m working on a proposal with use cases; this will take into account most of the features discussed during last CHEP
(ref: http://www.cnaf.infn.it/~sergio/datatag/glue/v11/SE/GlueSE_revision.html)
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 1111
GLUE Schema extended for monitoring
• within DataTAG we have extended the host entity with several metrics
• these have not been proposed for inclusion in the GLUE schema
• any action on this?
DataTAG is a project funded by the European Union DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003DataTAG WP4 meeting, Bologna 29/07/2003 – n – noo 1212
GLUE Schema integration in VDT
• Early July, meeting with Alain Roy at CNAF
• Achievement:
• By this week in VDT 1.1.10 there will be:
• Updated GLUE Schema, extended version
• EDG CE Info provider properly configured for CONDOR LRMS
Top Related