Jakość dostępu do danych w środowiskach obliczeniowych typu Grid na przykładzie PL-Grid
PL-Grid : National Grid Initiative in Poland Structure and Status
description
Transcript of PL-Grid : National Grid Initiative in Poland Structure and Status
![Page 1: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/1.jpg)
Polish InfrastructurePolish Infrastructurefor Supporting Computational Sciencefor Supporting Computational Science
in the European Research Spacein the European Research Space
EUROPEAN UNION
PL-Grid:PL-Grid:National Grid Initiative in PolandNational Grid Initiative in Poland
Structure and StatusStructure and Status
Jacek KitowskiJacek KitowskiInstitute of Computer Science AGH-USTInstitute of Computer Science AGH-UST
ACK CYFRONET AGHACK CYFRONET AGH
in collaboration within collaboration withPL-Grid RepresentativesPL-Grid Representatives
HP-CAST
Madrid, May 10-13, 2009
BigViewBigView
![Page 2: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/2.jpg)
PL-Grid Consortium Systematic approach establishing the Consortium and NGI Motivation Rationales Foundations International Collaboration
PL-Grid Project in a nutshell Status
PL-Grid Project Infrastructure Workpackages
Summary
OutlineOutline
![Page 3: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/3.jpg)
MotivationMotivation E-Science approach to research Integration activities ongoing in the world
3
![Page 4: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/4.jpg)
Need for e-Science ApproachNeed for e-Science Approach E-Science: collaborative research supported by advanced distributed computations
Multi-disciplinary, Multi-Site and Multi-National Building with and demanding advances in Computing/Computer Sciences
Goal: to enable better research in all disciplines System-level Science: beyond individual phenomena, components interact
and interrelate to generate, interpret
and analyse rich data resources• From experiments, observations
and simulations• Quality management, preservation
and reliable evidence to develop and explore models and simulations
• Computation and data at all scales• Trustworthy, economic, timely and relevant results
to enable dynamic distributed collaboration• Facilitating collaboration with information
and resource sharing• Security, trust, reliability, accountability, manageability and agility
M. Atkinson, e-Science (...), Grid2006 & 2-nd Int.Conf.e-Social Science 2006, National e-Science Centre UKI. Foster, System Level Science and System Level Models, Snowmass, August 1-2, 2007
![Page 5: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/5.jpg)
Thanks to CYFRONET activity
five High Performance Computing Polish Centres analysed according to: Participation in International and National Projects and Collaboration Need by Polish Scientific Communities Computational resources to date European/Worldwide integration Activities National Network Infrastructure ready (thanks to Pionier National Project)
Rationales behind PL-Grid ConsortiumRationales behind PL-Grid Consortium
GEANT2GEANT2
![Page 6: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/6.jpg)
~35 international projects FP5, FP6, FP7 on Grids (50% common) Grid Projects: Crossgrid, Gridstart, Unicore, EGEE I,II,III, K-WfGrid, Coregrid,
Virolab, Gredia, int.eu.grid, Eurogrid, Grip, Unigrids, Balticgrid I,II, Gridlab, Intelligrid, Ringrid, Brein, Beingrid, Qoscosgrid, Chemomentum, EUChinagrid, PRACE,…
~10 Polish projects (50% common) Pionier, Progress, SGIgrid, Clusterix, NDS, Platon, IT-SOA, ….
Close collaboration with Institute of Computer Science AGH-UST
Participation in Projects and CollaborationParticipation in Projects and Collaboration
![Page 7: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/7.jpg)
Need by Polish Scientific Communities Need by Polish Scientific Communities
Scientific Community /Representative % Polish publications
Warsaw / ICM 29,0 %Cracow / Cyfronet 16,4 %Wrocław / WCSS 11,1 %Poznań / PCSS 10,1 %Gdańsk / TASK 6,8 %
SUM 73,4 %
Poland 100 %
Distribution of Polish publications in period 01.2004 – 04.2008 according to
Science Citation Index Expanded + Social Science Citation Index + Arts & Humanities Citation Index
![Page 8: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/8.jpg)
Partners’ Resources to DatePartners’ Resources to Date
TOP500 – Nov.2008 Polish SitesTOP500 – Nov.2008 Polish Sites
![Page 9: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/9.jpg)
National and International Integration InitiativesNational and International Integration Initiatives
Ongoing European and Worldwide activities and consolidation EGEE EGI_DS, EGI DEISA PRACE ......
European e-
Infrastructure
2000 Testbeds 2010 Utility ServiceRoutine Usage
National
International
SGI Grid, Progress(Clusterix, National Data Store ...)
Chemomomentum, Virolab, CoreGrid, Gredia, int.eu.grid, Baltic Grid, GridLab,
Porta Optica, RINGRid, Phosphorus, QoSCoSGrid, Intelligrid, K-WfGrid, Unicore...
![Page 10: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/10.jpg)
Other Activities – Collaboration and DisseminationOther Activities – Collaboration and Dissemination EGI_DS – CYFRONET activity
Policy Board (deputy: M. Turala), NGI Observers -- „Polish Experts” (WP3, WP5) participation in Workshops, resulted in EGI_DS
draft deliverables, use-cases ... EGEE
From Cyfronet: chairing Resource Allocation Group in EGEE-III – trying to influence EGI• Group has a mandate to manage current process of resource allocation in EGEE and
propose better model for future• Operate CE ROC • More: https://twiki.cern.ch/twiki/bin/view/EGEE/RAG
E-IRG – CYFRONET activity Workshops Discussions on EGEE – EGI – NGI future
EU Unit F3 „Research Infrastructure” Experts in Program Committee, nominated by the Polish Ministry of Science and High Education (since 2007) from CYFRONET
ENPG seminar in Cracow (2008) Others
CGW 2007 and 2008 Cyfronet Users’ Annual Conference (2008 and 2009) Cyfronet Open Day (2007, 2008) ICCS 2008 Conference
![Page 11: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/11.jpg)
Other ActivitiesOther Activities
Computational Chemistry (VO) in EGEE Third CPU power consumer
• Nearly 3 million of jobs executed during 2007 Lead by Cyfronet since 2006
• Recently management of Computational Chemistry Cluster of Excellence In parallel current effort include:
• Grid ports of commercial and non commercial software packages with particular focus on their parallel version
• Development of “experiment centric” grid web portal for chemists
EGEE III / CE ROC
![Page 12: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/12.jpg)
EGEE-IIIEGEE-III / CE ROC / CE ROC• CE ROC – coordinator: ACK CYFRONET-AGH
• Consortium consists of institutes from 7 countries:Austria (2), Croatia (3), Czech (1), Hungary (5), Poland (4), Slovakia (1), Slovenia (1)
• Operate, maintain and support EGEE Grid Infrastructure in CE region
• Virtual Organizations supported in Central European region:HEP, computational chemistry, biomedicine, pharmacology, astrophysics, earth scienceand regional users (VOCE VO)
Poznan-PSNC
Cracow-CYFRONET
Warsaw-ICM
cores storage
Published by sites
![Page 13: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/13.jpg)
PL-Grid Foundations – SummaryPL-Grid Foundations – Summary
Motivation E-Science approach to research EGI initiative ongoing in collaboration with NGIs
Creation of Polish Grid (PL-Grid) Consortium: http://plgrid.pl Consortium Agreement signed in January 2007
PL-Grid Project (2009-2012) Application in Operational Programme Innovative Economy, Activity 2.3 (in Sept. 2008) Get funded March 2, 2009 (via European Structural Funds)
Consortium made up of five largest Polish supercomputing and networking centres (founders) ACK CYFRONET AGH (Cracow) – Coordinator
Polish Infrastructure for Supporting Computational Science in the European Research Space
Response to the needs of Polish scientists and ongoing Grid activities in Poland, other European countries and all over the world
![Page 14: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/14.jpg)
PL-Grid Consortium Founders PL-Grid Consortium Founders
• Academic Computer Center Cyfronet AGH (ACK CYFRONET AGH) Coordinator • Poznań Supercomputing and Networking Center (PCSS)• Wrocław Centre for Networking and Supercomputing (WCSS)• Academic Computer Center in Gdańsk (TASK)• Interdisciplinary Center for Math. and Computat. Modelling, • Warsaw University (ICM)
GEANT2GEANT2
![Page 15: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/15.jpg)
Grid infrastructure (Grid services)PL-Grid
App
licat
ion
App
licat
ion
App
licat
ion
App
licat
ion
Clusters High Performance Computers Data repositories
National Computer NetworkPIONIER
DomainGrid
Advanced Service Platforms
DomainGrid
DomainGrid
DomainGrid
Assumptions Polish Grid is going to have a common base infrastructure – similar to solutions adopted in
other countries. Specialized, domain Grid systems – including services and tools focused on specific types of
applications – will be built upon this infrastructure. These domain Grid systems can be further developed and maintained in the framework of
separate projects. Such an approach should enable efficient use of available financial resources. Creation of a Grid infrastructure fully compatible and interoperable with European and World
Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).
Plans for HPC and Scalability Computing enabled.
PL-Grid InfrastructurePL-Grid Infrastructure
Users
Nationalcomputernetwork
Grid Application
Programming Interface
Virtual organizations and
security systems
Basic Grid services
Gridservices
LCG/gLite(EGEE)
UNICORE(DEISA)
OtherGrids
systems
Gridresource
s
Distributedcomputationa
l resources
Grid portals, development tools
Distributed data
repositories
![Page 16: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/16.jpg)
The PL-Grid Project is split into several workpackages
Planned Realization of AimsPlanned Realization of Aims
PLANNING AND DEVELOPMENT
OF INFRASTRUCTURE
P2
Coordination StructureOperation Rules Dissemination
PROJECT MANAGEMENTP1
SECURITY CENTER
P6
Training
SUPPORT FOR VARIOUSDOMAIN GRIDS
P5P4 GRID SOFTWAREAND USERS
TOOLS DEVELOPMENTEGEE DEISA … .
OPERATIONS CENTERP3
Main Project Indicators: • Peak Perf.: 215 Tflops • Disk Storage: 2500 TB
![Page 17: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/17.jpg)
ChallengesChallenges Short term
To start – establishing PL-Grid VO using Partners’ local and EGEE resources To provide resources for covering operational costs To select computational/storage resources for PL-Grid infrastructure
Long term – continously To provide the necessary infrastructure (!!)
• Computer rooms, electrical power, many organizational issues Be prepared / work on approaching paradigms and integration development
• HPC – Scalable Computing• Clouds (internal-external, computing clouds, data clouds)• SOA paradigm, knowledge usage …
![Page 18: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/18.jpg)
Thank you
![Page 19: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/19.jpg)
Polish InfrastructurePolish Infrastructurefor Supporting Computational Sciencefor Supporting Computational Science
in the European Research Spacein the European Research Space
EUROPEAN UNION
PL-Grid:PL-Grid:National Grid Initiative in PolandNational Grid Initiative in Poland
Structure and StatusStructure and Status
Jacek KitowskiJacek KitowskiInstitute of Computer Science AGH-USTInstitute of Computer Science AGH-UST
ACK CYFRONET AGHACK CYFRONET AGH
in collaboration within collaboration withPL-Grid RepresentativesPL-Grid Representatives
HP-CAST
Madrid, May 10-13, 2009
Technical IssuesTechnical Issues
![Page 20: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/20.jpg)
Grid infrastructure (Grid services) PL-Grid
App
licat
ion
App
licat
ion
App
licat
ion
App
licat
ion
Clusters High Performance Computers Data repositories
National Computer Network PIONIER
DomainGrid
Advanced Service Platforms
DomainGrid
DomainGrid
DomainGrid
Assumptions Polish Grid is going to have a common base infrastructure – similar to solutions adopted in
other countries. Specialized, domain Grid systems – including services and tools focused on specific types of
applications – will be built upon this infrastructure. These domain Grid systems can be further developed and maintained in the framework of
separate projects. Such an approach should enable efficient use of available financial resources. Creation of a Grid infrastructure fully compatible and interoperable with European and World
Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).
Plans for HPC and Scalability Computing enabled.
PL-Grid InfrastructurePL-Grid Infrastructure
![Page 21: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/21.jpg)
Elements and FunctionalityElements and Functionality
PL-Grid software will comprise: user tools (portals, systems for
applications management and monitoring, result visualization and other purposes, compatible with the lower-layer software used in PL-Grid);
software libraries; virtual organization systems: certificates,
accounting, security, dynamic ; data management systems: metadata
catalogues, replica management, file transfer;
resource management systems: job management, applications, grid services and infrastructure monitoring, license management, local resource management, monitoring.
Users
Nationalcomputernetwork
Grid Application
Programming Interface
Virtual organizations andsecurity systems
Basic Grid services
Gridservices
LCG/gLite(EGEE)
UNICORE(DEISA)
OtherGrids
systems
Gridresources
Distributedcomputational
resources
Grid portals, development tools
Distributed data
repositories
Three Grid structures will be maintained: production, reseach, testing.
![Page 22: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/22.jpg)
The PL-Grid Project is split into several workpackages
Planned Realization of AimsPlanned Realization of Aims
PLANNING AND DEVELOPMENT
OF INFRASTRUCTURE
P2
Coordination StructureOperation Rules Dissemination
PROJECT MANAGEMENTP1
SECURITY CENTER
P6
Training
SUPPORT FOR VARIOUSDOMAIN GRIDS
P5P4 GRID SOFTWAREAND USERS
TOOLS DEVELOPMENTEGEE DEISA … .
OPERATIONS CENTERP3
Main Project Indicators: • Peak Perf.: 215 Tflops • Disk Storage: 2500 TB
![Page 23: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/23.jpg)
WP2: Planning of Infrastructure DevelopmentWP2: Planning of Infrastructure Development
Analysis of users’ requirement Analysis of worldwide activity in infrastructure development Expecting results:
• Current activity: organization of pilot PL-Grid VO using EGEE infrastructure• From analysis: clusters with x86 processors
![Page 24: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/24.jpg)
WP3: Operations Center’s tasksWP3: Operations Center’s tasks
EGIGroup
EGI Testing
Middleware
EGI ProductionMiddleware
Coordination of Operation Management and accounting EGI and DEISA collaboration
(HPC and Scalability Computing) Users’ requirements More spectacular (using
researchand test infrastructures) Computational Cloud Data Cloud Internal and External
Clouds
![Page 25: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/25.jpg)
WP4: Grid Software and Users Tools WP4: Grid Software and Users Tools
Workpackage on Development/Research• Analysis of users’ requirements• Software repository (different tools and components developed by Partners)• Reengineering of tools and applications (legacy codes, APIs, SOA applications)• Virtual Laboratory (component applications, workflows, applications monitoring) – from
EU FP6 Virolab Project• High-level virtual organizations using knowledge (semi-automatic creation using
contract, semantic description of resources, monitoring of SLA due to QoS), data access – FiVO from EU FP6 Gredia Project
• Tools for management, proactive monitoring and security (Bazaar, Permis)
![Page 26: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/26.jpg)
WP5: Support and TrainingWP5: Support and Training
Users’ support Making commercial software available for the users (license activity) Training, education
![Page 27: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/27.jpg)
WP6: SecurityWP6: Security
Users’ support • Authorization, authentication, ...• Certificates ...
![Page 28: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/28.jpg)
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688 [email protected]
• Gaussian VO created supported by several Partners (EGEE activity)
• Accepted by Vendor
• Registration: https://voms.cyf-kr.edu.pl:8443/vo/gaussian/vomrs
GAUSSIAN in Grid
• VO Manager: Mariusz Sterzel (CYFRONET), EGEE II Comput. Chemistry coordinator [email protected]
ACK: Tomasz Szepieniec
![Page 29: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/29.jpg)
Enabling Grids for E-sciencE
EGEE-II INFSO-RI-031688 [email protected]
Never Born Protein Folding
ComputingElement
Euchina Virtual Organization (EGEE)
StorageElement
2. Transfer
application
User InterfacePortal
AABTDDSAD
1.Submit
sequence
3. Store protein
4. Visualize
PDB1.32 3.23 3.442.77 4.33 5.661.32 3.23 3.44
ACK: Irena Roterman, Jagiellonian University, Tomasz Szepieniec
![Page 30: PL-Grid : National Grid Initiative in Poland Structure and Status](https://reader035.fdocuments.net/reader035/viewer/2022062315/56816047550346895dcf6ccd/html5/thumbnails/30.jpg)
ActivitiesActivities Short term
To start – establishing PL-Grid VO using Partners’ local and EGEE resources To provide resources for covering operational costs To select computational/storage resources for PL-Grid infrastructure
Long term – continously To provide the necessary infrastructure (!!)
• Computer rooms, electrical power, many organizational issues • Towards Cloud Computing (internal, external)• Towards HPCaaS and Scalable computing