Metadata Progress
GridPP1820 March 2007Mike Kenyon
Metadata Team
• Glasgow– Mike Kenyon– Caitriana Nicholson– Tom Doherty– Paul Millar
• CERN/Glasgow– Helen McGlone
ATLAS Event-level Metadata
• Navigation from Tag to events in AOD/ESD/RAW format
• Produced during AOD production– Oracle DBs at Tier0, Tier1– MySQL (probably) at Tier2
• ATLAS are implementing a Tag database to allow efficient identification and selection of events for analysis
• First-level cuts will be possible via queries on relational databases
Tag Navigator Toolhttps://twiki.cern.ch/twiki/bin/view/Atlas/TagNavigat
orTool
• Need to integrate Tag DB with Distributed Data Management (DQ2) and Distributed Analysis (Ganga) tools
• Tag Navigator Tool (TNT) developed to plug that gap
• Aims to take user from query on Tag DB to analysis using results of query
TNT Process Summary
• Runs a query on the database and outputs a local Root collection
• Divides this into a number of sub-collections
• Submits user jobs to LCG, one per sub-collection
• Output files can be registered as new DQ2 dataset
TNT Process Summary
Ganga Plugin
• TNT has also been implemented as a plugin to Ganga
• Ganga already allowed use of Tag files – TNT plugin is complementary to this
• Makes use of existing Ganga functionality for job submission, handling etc
• Will be available in Ganga release 4.3
Future Plans
• Continue development of TNT and GangaTnt in response to user feedback
• Performance testing of Tags in TNT• Scalability and performance
measurements on Tag DB• Continue integration of Tags with AMI
AMI-VOMS solution1. The grid user authenticates
themselves to AMI using a grid certificate.
2. AMI allows the user to launch a Java web start application to create a VOMS proxy.
3. The user chooses the x509 certificate to be sent to the VOMS server
4. The VOMS server sends back a VOMS proxy certificate to the Java application.
5. The VOMS proxy file is uploaded to the server, which hosts a proxy delegation service, via a secure web service attachment.
6. The user’s VO, group and role can be extracted from the proxy and used for authorisation purposes.
SourceForge project: Acacia
12
3
4
5
67
DQ2/AMI integration• Use of delegated VOMS proxy certificate - Contact
application on behalf of user, e.g. DQ2.• Register a dataset simultaneously in AMI and DQ2• Other dq2 commands possible
Tags/AMI integration• Preliminary work on prototype database to allow
physics predicate cut on a collection and catalogue the results within AMI
pyAMI
• AMI Web Service Client for Python• AMI tutorial available at:
http://ami3.in2p3.fr:8080/opencms/opencms/AMI/www/Tutorial/pyAMI.html
• Future development and maintenance• Providing AMI command wrappers for groups/tools such as
POOL Conditions, Muon and Tag collector• Code optimisation – replace SAX with DOM parsing of AMI
output• Reformatting output – interface to format XML
The present: MonAMI v0.6
• “Universal sensor framework”– Capable of monitoring many different services– Can send data to any number of information
systems– Plugin structure
• v0.6 deployed at all ScotGrid sites & Grenoble
• Configuration details available at– http://www.gridpp.ac.uk/wiki/MonAMI
Upcoming: MonAMI v0.7
• v0.7– Support for 64-bit (AMD64) architecture– Improved documentation– Major improvements to R-GMA support
• Robust against temporary R-GMA failures• Support for TLS-based secure upload of data• Support for proxy-certificate based authentication• “Drop in” deployment
– Support for event monitoring (e.g. DPM/GridFTP)
• Released pending solution of minor SL4-related issues
The Future: MonAMI v0.8
• Planned features– New python plugin API– Web & RSS feeds– PostGreSQL monitoring
Top Related