The EDGeS project receives Community research funding XtremWeb-HEP & EGEE CSST-HUST-Wuhan Octobre...
-
Upload
frank-lloyd -
Category
Documents
-
view
217 -
download
0
Transcript of The EDGeS project receives Community research funding XtremWeb-HEP & EGEE CSST-HUST-Wuhan Octobre...
The EDGeS project receives Community research funding
XtremWeb-HEP & EGEEXtremWeb-HEP & EGEE
CSST-HUST-WuhanCSST-HUST-Wuhan
Octobre 11-15th, 2010Octobre 11-15th, 2010
Oleg Lodygensky - LAL - Oleg Lodygensky - LAL - [email protected]@lal.in2p3.fr
2Authors : O. Lodygensky HUST : XtremWeb-HEP 2
EDGeSEDGeSBOINC-based DGs
XtremWeb/XWHEP-based DGs
localXtremWeb DG
IN2P3200 PCs
300 (150%)
new public XtremWeb DGEDGeS@home
1.000 PCs
EGEE VO
publicXtremWeb DG
IN2P3
300 PCs6 (2%)
publicXtremWeb DGAlmereGrid
3.000 PCs?
new public BOINC DG
EDGeS@home
10.000 PCs
publicBOINC DG
SZDG
30.000 PCs77.000 (256%)
publicBOINC DG
Extremadura
70.000 PCs22.500 (32%)
localBOINC DG
Westminster1.500 PCs
1881 (125%)
localBOINC DGCorrelationSystems
publicBOINC DGAlmereGrid
0 PCs1.700
publicBOINC DGIbervicis
0 PCs24.000
3HUST : XtremWeb-HEPAuthors : O. Lodygensky 3
EDGeSEDGeS
desktopgrid.vo
voms
lb wmsbdii
an EGEE VO
BOINC-EGEEbridge
EGEE>DGbridge CE
UoW queueSDZGr queueAlmere queue
Fundecyt queue
BOINC-based DGs
UoW, Correlation
Systems local DGs
SZDGr,
Almere,
Fundecytpublic DGs
AR
UICoreService Resource
lfc
bridge
bridge
bridge
bridge
myproxy
ce
wms
ce
4
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
Bridging EGEE and Desktop Grids
Grid User
X509 proxy
VOMS Proxy
Submits Job with VOMS proxy
Publishes available Resources
Pushes Job
Log
Log
VOMS Admin
Manages VO
Site Admin
Manages Site
Accesses Data with VOMS proxy
Accesses Data with VOMS proxy
Gives Job Status
Gives Accounting and Auditing
VOMS Server
AccountingLogging & Bookkeeping
Meta-scheduler(WMS)
Site Computing Resource
Site Storage
Resource
Grid Admin
Sends back Output Sandbox Sends back
Output Sandbox
VOMS Proxy =X509 proxy with
VOMS extensions
EGEE (Service Grid)Well defined infrastructure:- authentication- authorization- logging and bookkeeping- quality of service (QoS)
A strong effort of standardization
Volunteer PCs
Volunteer PCs
Desktop Grids : Boinc, XWHEPUndefined infrastructure:- anonymity- volatility
How to bridge ? - security - monitoring - logging
DG User
?
Server(scheduler)
5
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
XWHEP->EGEEXWHEP->EGEEresource sharingresource sharing
9
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
XWHEP :XWHEP :XtremWeb by High Energy PhysicsXtremWeb by High Energy Physics
LAL :•LCG Tiers-2•EGEE seed resource• XtremWeb by High Energy Physics (XWHEP) developer & maintainer
XWHEP is based on XtremWeb 1.8.0. by INRIA.
http://www.xtremweb-hep.org/spip.php?rubrique16
XWHEP ensures security at different levels:
• authentication :✓ SSL and X509 certificates
• authorization and confidentiality :✓ user rights management✓ user groups✓ access rights
• volunteer nodes integrity :✓ application repository✓ sandboxing
• monitoring and bookeeping :✓all messages are logged
XWHEP environment➡ multi users➡ multi applications
10
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
XWUser Job
Execution Sandbox
Userdata andapplications
Dynamically downloadedXW user dataandapplication
Externaldata
servers
Server certificate
Server public key
Job Mgt
DeployedXW-Client UI
DeployedXW-Computing
Service
PC
Volunteer PC
Data Sandbox
All communicationsare encrypted
Data Mgt
XW Services
Local I/O
XWHEP : Architecture, authenticationXWHEP : Architecture, authenticationand resource integrityand resource integrity
Authentication: - login/password - X509 cert. Node integrity:
- sandboxing
Authentication: - login/password - X509 cert.
XW Coordinator(scheduler)
XW Data Repository
24
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
VOMS Server
Meta-scheduler(WMS)
DG User
jLite : proxy init
XW Coordinator(scheduler)
jLiteby Oleg Sukhoroslovhttp://code.google.com/p/jlite/
Public worker
Group worker
Private worker
User X.509 Cert
User X.509 proxy
Server certificate
Server public key
Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)
XWHEP->EGEEXWHEP->EGEEbridgingbridging
XW Bridge
Submit Pilot jobw/ user X509 proxy
Retreived signedDG user job
DG user job submissionwith X509 proxy
DownloadDG userX509 proxy
Pilot Job
Private worker
WN
Site Computing Resource
Pilot Job submitted as EGEE Job
DG user job deployment, status, results
28
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
How to port an application fromHow to port an application fromXWHEP to EGEEXWHEP to EGEE
•EGEE is made of Linux computing resourcesEGEE is made of Linux computing resources•XWHEP applications must have a Linux XWHEP applications must have a Linux
versionversion
•There is no specific need to recompile nor There is no specific need to recompile nor relink binariesrelink binaries
29
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
EGEE->XWHEPEGEE->XWHEPresource sharingresource sharing
30
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
Bridging EGEE and Desktop Grids
Grid User
X509 proxy
VOMS Proxy
Submits Job with VOMS proxy
Publishes available Resources
Pushes Job
Log
Log
VOMS Admin
Manages VO
Site Admin
Manages Site
Accesses Data with VOMS proxy
Accesses Data with VOMS proxy
Gives Job Status
Gives Accounting and Auditing
VOMS Server
AccountingLogging & Bookkeeping
Meta-scheduler(WMS)
Site Computing Resource
Site Storage
Resource
Grid Admin
Sends back Output Sandbox Sends back
Output Sandbox
VOMS Proxy =X509 proxy with
VOMS extensions
EGEE (Service Grid)Well defined infrastructure:- authentication- authorization- logging and bookkeeping- quality of service (QoS)
A strong effort of standardization
Volunteer PCs
Volunteer PCs
Desktop Grids : Boinc, XWHEPUndefined infrastructure:- anonymity- volatility
How to bridge ? - security - monitoring - logging
DG User
?
Server(scheduler)
31
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
gLite ensures security:
• authentication :✓ X509 cert. and VOMS ext.
• authorization and confidentiality :✓ VOMS server
• monitoring and bookeeping
EGEE->XWHEP bridge :EGEE->XWHEP bridge :authentication, authorization, logging, authentication, authorization, logging,
monitoringmonitoring
XWHEP ensures security at different levels:
• authentication :✓ SSL and X509 certificates
• authorization and confidentiality :✓ user rights management✓ user groups✓ access rights
• volunteer nodes integrity :✓ application repository✓ sandboxing
• monitoring and bookeeping :✓all messages are logged
35
Volunteer PCs
Volunteer PCsXtremWeb
User
V 5.7.5-1
HUST : XtremWeb-HEPAuthors : O. Lodygensky
QM WS client
QM
XW
plu
gin
WS
Int
erfa
ce
P-GRADE Portal
EDGeSEDGeSARAR
EGEE -> XtremWebEGEE -> XtremWebresource sharingresource sharing
VOMS Server
LCG CE
Meta-scheduler(WMS)
EDGeS CE XW Coordinator(scheduler)