Workshop on HPC in India Challenges of Garuda : The National Grid Computing Initiative of India...
-
Upload
jasper-eads -
Category
Documents
-
view
214 -
download
0
Transcript of Workshop on HPC in India Challenges of Garuda : The National Grid Computing Initiative of India...
Workshop on HPC in India
Challenges of Garuda : The National Grid Computing
Initiative of India
Subrata Chattopadhyay C-DAC Knowledge Park
Bangalore, India
ATIP 1st Workshop on HPC in India @ SC-09
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 2
Outline Introduction – C-DAC Garuda overview PoC Phase Foundation phase Current status Challenges Q & A
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 3
C-DAC - Overview Established in 1989 as a Resource Centre in
System Software for High Performance Computing (HPC)
Activities cover HPC & Grid computing, Real Time & Dependable Computing, Heritage Computing, E-Governance, Education and Training
Houses C-DAC’s TeraScale Supercomputing Facility (CTSF) as a National HPC Facility
Several turn key projects in system software, real time systems and E-Governance for the domestic and foreign markets
100-150 Technical Members
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 4
GARUDA - Overview Proof of Concept phase of project completed in
March 2008 Precursor to the National Grid Computing
Initiative Test Bed for grid technology/concepts and
applications leading to the plan for the main grid initiative
Major Deliverables on PoC Technologies, Architectures, Standards &
Research Initiatives Nation-wide high-speed communication fabric Aggregation of Grid Resources Deployment of Select applications of National
Importance Grid Strategic User Group
High-speed Networking Component in collaboration with ERNET
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 5
Garuda MPLS Network
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 6
Architecture
GARUDA Resources
•Compute, Data, Storage,
•Scientific Instruments,
•Software,..
Access Methods
• Access Portal for SOA
• Problem Solving Environments
Management, Monitoring & AccountingParyaveekshanam
•Web MDS
•GARUDA Information Service
•GARUDA AccountingSecurity Framework
• IGCA Certificates
• MyProxy
• VOMS
Resource Mgmt & Scheduling
• GridWay Meta-scheduler
• Resource Reservation
• Torque, Load Leveler
• Globus 4.x (WS Components)
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 7
Grid Infrastructure – Resources
PARAM Padma and Linux Clusters have been setup at Pune, Hyderabad & Chennai
Grid Labs have been setup at three locations
Fourteen of the partner institutions contributing resources including satellite terminals ( aggregating to more than 1500 CPUs and over 60TB storage)
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 8
GARUDA Partners Motivation
To Collaborate on Research and Engineering of Technologies, Architectures, Standards and Applications in Grid Computing
To Contribute to the aggregation of resources in the Grid
Current Participation 36 research & academic
institutions in the 17 cities
ERNET-HQ in Delhi 8 centres of C-DAC Total of 45 institutions
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 9
Virtual User Community Astrophysics High Energy Physics & Astronomy Grid Technology Disaster Management Earth Science Bioinformatics (Genome) Computational Fluid Dynamics Network Technology
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 10
Research Initiatives Objective
Create the foundation for next generation grids by addressing long term research issues in the strategic areas of
• Knowledge and data management • Programming models • Architectures • Grid Management and Monitoring• Problem solving environments • Tools and grid services
Research Initiatives in Semantic Grids Integrated Development Environment Network Simulation Mobile Agents Grid File Systems Resource Brokers
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 11
Collaborations Space Application Centre, Ahmedabad
Disaster Management Grid Middleware for Satellite Grids
Indian Institute of Science, Bangalore Climate Modeling
Madras Institute of Technology, Chennai Grid Middleware for Semantic Grid Services
Indian Institute of Technology, Mumbai Computational Fluid Dynamics Package on PARAM
Padma University of Pune
Quantum Chemistry, Materials Modeling, Bioinformatics
INDMOL Molecular Package developed EU-India Grid
BARC, C-DAC, ERNET, Saha Institute, TIFR, Univ. Pune Consortium GARR, ICTP, INFN (Italy) Cambridge University (UK)
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 12
GARUDA – POC - Overview Proof of Concept phase of project completed in
March 2008 Precursor to the National Grid Computing
Initiative Test Bed for grid technology/concepts and applications
leading to the plan for the main grid initiative Major Deliverables on PoC
Technologies, Architectures, Standards & Research Initiatives
Nation-wide high-speed communication fabric Aggregation of Grid Resources Deployment of Select applications of National
Importance Grid Strategic User Group
High-speed Networking Component in collaboration with ERNET
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 13
Highlights of GARUDA PoC Building core technical competence in grid
computing – development and deployment of Grid middleware, tools and services
Deployment of Cyberinfrastructure High speed Networking Resource aggregation (resources from various
institutes/organizations) Prototype Demonstration of Applications of
National importance Disaster Management (DMSAR) Bioinformatics
Building the Grid community Understanding research challenges in Grid
computing
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 14
Garuda–Foundation Phase:2008-09
Test interoperability of Garuda with other international grids
Monitor and operate the high speed bandwidth Garuda network and grid resources
Upgrade Garuda resources Demonstrate SOA framework that may lead to
participation from Industry Research on security and grid technologies Establish certificate authority for national grid
operation Enable selected applications on grid
environment
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 15
Achievements - Foundation Phase Explored viability of commercial applications / services
in areas of CAE and Molecular Docking on SOA GARUDA
Replaced commercial grid scheduler with OpenSource ‘Gridway’ meta scheduler
Exploring OpenSource solution for Data Grid through SRM as an alternate to SRB from Nirvana
Established the First CA in India to address security issues of grid – IGCA
Improved processing time from 6.1hrs to 54 mins for processing one set of Radar data (9GB) for the DMSAR application using the new resources of GARUDA (272 cores)
Winglet applications for processing 6000 winglets taking nearly 30 days to complete on ordinary machines were able to complete in 3 hours by harnessing the large computational power offered by GARUDA.
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 16
Objective ofOperational Phase of Garuda
To provide stable, robust, and efficient grid environment with guaranteed quality of service for various applications
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 17
Major components Architecture and Framework Middleware & Associated tools:
Development, deployment and operational support
Migration to interoperation with international grids
Resource aggregation & coordination with resources initiative
Operational pilot of applications & coordination with new application initiatives
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 18
Garuda Services
2010 -
2011 -
2012 -
2010 -
2011 -
2012 -
2010 -
2011 -
2012 -
2010 -
2011 -
2012 -
Resource Aggregation & AugmentationParticipation – Intl Commn, Stds ForumStabilize & Enhance – NKN, middleware, tools
Next Gen ServicesKnowledge DisseminationDeploy Operational Applications
-
-
-
-
-
-
Grid Enablement SupportMigration to NKNMiddleware & ToolsArchitecture & Framework
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 19
International Visibility
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 20
Eu-Indiagrid Project Goals To support the interconnection and inter-operability
of the prominent European Grid infrastructure (EGEE) with the Indian Grid infrastructure (Garuda National Grid Initiative, DAE-Grid) for the benefit of eScience applications;
To identify and aggregate research, scientific and industrial communities which may benefit from the use of Grid technology resulting in an eScience Network Community
To promote the use of advanced Grid technologies within the created Network Community relying on pilot applications in Biology and High Energy Physics Material Science, Earth and Atmospheric Sciences and specific outreach and dissemination activities
To disseminate European EGEE Grid technology achievements in India and leverage on Indian Grid experiences and skills.
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 21
ParticipantsEUROPE Italian National Institute of Nuclear Physics, INFN (project
coordinator), Metaware SpA, Italian Academic and Research Network (GARR) Cambridge University
INTERNATIONAL Abdus Salam International Centre for Theoretical Physics (ICTP)
INDIA Indian Education and Research Network (ERNET), University of Pune, SAHA Institute of Nuclear Physics, Kolkata, Centre for Development of Advanced Computing (C-DAC), Bhabha Atomic Research Centre, Mumbai TATA Institute for Fundamental Research (Mumbai)
National Centre for Biological Sciences (Bangalore)
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 22
Challenges! Heterogeneous Resources Policy across different administrative
domains Diverse requirements and priorities Funding and sustainability Killer Apps - Interoperation and standards
S Chattopadhyay ATIP 1st Workshop on HPC in India @ SC-09 23
Thank you!