Forschungszentrum Jülich in der Helmholtz-Gemeinschaft UNICORE and Grid Computing in Europe Dietmar...
-
Upload
jesse-james -
Category
Documents
-
view
218 -
download
0
Transcript of Forschungszentrum Jülich in der Helmholtz-Gemeinschaft UNICORE and Grid Computing in Europe Dietmar...
Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UNICORE UNICORE and Grid Computing in and Grid Computing in
EuropeEurope
Dietmar Erwin
Forschungszentrum Jü[email protected]
August 27, 2003
HPN Workshop via AG 2Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
ContentsContents
• Introduction to FZJ
• Objectives and Status of UNICORE
• Grid Projects in Europe
• Eurogrid, GRIP and OGSA
HPN Workshop via AG 3Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
“The Future is Our Mission”
Matter
Energy
Information
Life
Environment
Source FZJ
HPN Workshop via AG 4Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Forschungszentrum Jülich at a GlanceForschungszentrum Jülich at a Glance
Founded:Dec. 1956
Area:2.2 km2
Legal Form:PLC
Operator:•Federal Republic of Germany (90 %)•State of North Rhine-Westphalia (10 %)
Bodies:• Partners Meeting• Supervisory Board• Board of Directors• Scientific and Technical Council
Scientific/Technical Structure:•10 Departments (39 Institutes)• 7 Central Departments• 2 Programme Groups• 3 Project Management Groups
Budget: Institutional: € 236 Mio/yearNon-institutional: € 52 Mio /year
Staff: 4200Including: • 1000 Scientists
• 400 Postgraduate Students• 150 Undergraduate Students• 300 Trainees
Visiting Scientists: more than 700 annually from more than 30 countries
Source FZJ
HPN Workshop via AG 5Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UNICORE: Uniform Interface to UNICORE: Uniform Interface to Computing ResourcesComputing Resources
Developed software for a• seamless, • secure, and • intuitive access to distributed computing resources
http://www.unicore.orgFunded by BMBF, grant: 01 IR 001
ObjectivesObjectives
HPN Workshop via AG 6Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UNICORE HighlightsUNICORE Highlights
• Intuitive, system-independent creation and control of jobs
• Support for multi-system and multi-site jobs
• Dynamic flow control
• Integrated security through X.509 certificates
• Access to remote file stores and archives
• Extensible support for scientific & commercial applications
• Minimal intrusion into site autonomy
HPN Workshop via AG 7Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UNICORE UNICORE ScreenshotScreenshot
HPN Workshop via AG 8Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Batch Subsystem
AJO
User Certificate
Job Preparation Agent (JPA)
Job Monitor Controller (JMC)
Abstract status request
User authentication
List of SitesUNICORE Client
Target System Interface (TSI)
Incarnated job
Commands
User mapping,Resources info
Site-specific authorization
TSI TSI
Batch Subsystem
FZJ LRZ
...
...
TSI
Batch Subsystem
Preparation andControl of jobs
Network Job Superviser(NJS)
Unsafe Internet (SSL/https)
Safe Intranet (TCP/http)
IDB
Job incarnation
Sending jobs toother gateways,data transfer
Status request
SV1 T3E SR8000 files
UUDB IDBIDB UUDB IDB
NJS
UNICORE Gateway
optional firewall
optional firewall
UNICORE Gateway
NJS
UNICORE ArchitectureUNICORE Architecture
HPN Workshop via AG 9Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
CPMD CPMD Application supportApplication support
HPN Workshop via AG 10Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Ad hoc Ad hoc weather predictionweather prediction
C: EUROGRID Projekt
Source DWD
HPN Workshop via AG 11Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UNICORE StatusUNICORE Status
• Project UNICORE Plus successfully completedReport at: http: //www.unicore.org
• UNICORE basis for a German HPC Grid(LRZ, HLRS, FZJ)
• Software available as Open Source for R&Dprojects http: //www.unicore.org/download
• UNICORE commercially supported and marketed by Pallas www.pallas.com
• UNICORE used in European Grid projects• UNICORE selected by Japanese NAREGI project
HPN Workshop via AG 12Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
LRZ: SR 8000, VPP, LC*RUS: SX-5, 2xLC*FZJ: 2xT3E,SV1, LC*ZIB: T3E, SunDWD: IBM SP, SGIRUKA: IBM SPPC²: LC*TUD: T3E, SGIPallas: SunFujitsu: VPP, Sun
*LC = Linux Cluster
UNICORE DeploymentUNICORE Deployment
FZJFZJ
ZIBZIBPCPC22
TUDTUD
LRZLRZ
DWDDWD
RUKARUKA
RUSRUS
HPN Workshop via AG 13Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
EGSOAVO
GRIA
CrossGridGridLab
GRIP
DataTAG
EuroGrid
DAMIEN
DataGrid
GEMSS
MammoGrid
BioGridSeLeNe
OpenMolGrid
COG
FlowGrid
GRACEMOSES
GRASP
Oct 2000 Oct 2001
Wave 1 Projects
Wave 2 Projects
Infrastructure
Applications
Middleware
Oct 2002
European ProjectsEuropean Projects
Source EC
HPN Workshop via AG 14Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
• First actions launched in IST-FP5
• Grid research is a strategic objective in FP6
2000-2002 2002-2006
FP5
FP6
125M€
58M€From FP5 to FP6 Grid research
funding more than doubles
IST-FP6 commitment to Grid research
FP6 Grid FundingFP6 Grid Funding
Source EC
HPN Workshop via AG 15Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
10
6
1 2
9
4
11
3
8
7
5
1
2
3
10
6
9
4
11
8
7
5
Forschungszentrum Jülich, D
Parallab - University of Bergen, N
CNRS - IDRIS, F
Pallas GmbH, D
debis Systemhaus,D
Warsaw University - ICM, P
Victoria University of Manchester, UK
Deutscher Wetterdienst, D
GIE EADS CCR, F
Fujitsu European Centre for Information Technology, UK
ETH Zürich, CH
HPN Workshop via AG 16Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Objectives Objectives
• Establish a European GRID network of leading European HPC centres
• Enhance UNICORE through:– Resource broker– Standard based File Transfer (gridFTP)
• Demonstrate Grid solution in – Bio molecular simulations,– Weather prediction,– Coupled CAE simulations,– Structural analysis.
Funded by EU, grant: IST-1999-20247
HPN Workshop via AG 17Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Globus
Application
Layer
Collective
Layer
Resource
Layer
Fabric
Layer
Grid ArchitectureGrid Architecture
Applications, Portals, Environments
Higher Services, APIs, Protocols
APIs, Protocols, SDKs
Computers, Networks, Devices
UNICORE
HPN Workshop via AG 18Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
GRIP ObjectivesGRIP Objectives
1. To develop interoperability software between UNICORE and Globus based Grid systems
2. To develop and demonstrate techniques for inter-Grid applications in meteorology and bio-molecular science
3. To influence creation of international Grid standards through the Global Grid Forum (GGF)
1. To develop interoperability software between UNICORE and Globus based Grid systems
2. To develop and demonstrate techniques for inter-Grid applications in meteorology and bio-molecular science
3. To influence creation of international Grid standards through the Global Grid Forum (GGF)
Grid Interoperability: Creation of virtual organizations using different technologies based on common standardsGrid Interoperability: Creation of virtual organizations
using different technologies based on common standards
Seamless, secure, intuitive access to different systems and sites to solve complex problems
GRIP partners- lead GGF working/ research groups- work on the definition of OGSA, the Open Grid Service Architecture
HPN Workshop via AG 19Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
GRIP PartnersGRIP Partners
Forschungszentrum Jülich (FZJ)
Pallas
University of Manchester (UoM)
ICM Warsaw (ICM)
Deutscher Wetterdienst (DWD)
University of Southampton (Soton)
Fujitsu European Laboratories
Argonne National Laboratories (ANL)
HPN Workshop via AG 20Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
GRIP StatusGRIP Status
• UNICORE Interoperability Software implemented (FZJ)
• Interoperability UNICORE – Globus demonstrated
• Development of OGSA interfaces in 2003
HPN Workshop via AG 21Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Batch Subsystem
AJO
User Certificate
Job Preparation Agent (JPA)
Job Monitor Controller (JMC)
Abstract status request
User authentication
List of SitesUNICORE Client
Target System Interface (TSI)
Incarnated job
Commands
User mapping,Resources info
Site-specific authorization
TSI TSI
Batch Subsystem
FZJ
...
...
Globus TSI
Globus
Preparation andControl of jobs
Network Job Superviser(NJS)
Unsafe Internet (SSL/https)
Safe Intranet (TCP/http)
IDB
Job incarnation
Sending jobs toother gateways,data transfer
Status request
SV1 T3E files
UUDB IDBIDB UUDB IDB
NJS
UNICORE Gateway
optional firewall
optional firewall
UNICORE Gateway
NJS
UNICORE ArchitectureUNICORE Architecture
HPN Workshop via AG 22Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UNICORE & OGSI UNICORE & OGSI
UnicoreGateway
Unicore ClientNetwork
JobSupervisor
ResourceDatabase
UserDatabase
Organic Server
PrimePower
OGSA Service
ResourceBroker
• Work to be completed in 2003– Integrate Grid Service Hosting into Gateway – UPL portType
• Vsite Discovery• Consign Job• Retrieve Outcome
– Add Grid Service interface to Client– Broker Interfaces and Ontology
User certificatesecurity domain
User proxy certificatesecurity domain
Fujitsu Laboratories of Europe © 2003Fujitsu Laboratories of Europe © 2003
HPN Workshop via AG 23Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
UnicoreGateway
Unicore ClientNetwork
JobSupervisor
ResourceDatabase
UserDatabase
Organic Server
PrimePower
OGSA Service
ResourceBroker
User certificatesecurity domain
User proxy certificatesecurity domain
Future WorkFuture Work
Full OGSA compliant UNICORE Implementation
Fujitsu Laboratories of Europe © 2003Fujitsu Laboratories of Europe © 2003
HPN Workshop via AG 24Forschungszentrum Jülichin der Helmholtz-Gemeinschaft
Further InformationFurther Information
• http://www.unicore.org
• http://www.pallas.com
• http://www.eurogrid.org
• http://www.grid-interoperability.org
• http://www.gridstart.org
• http://www.fz-juelich.de