"Cloud Computing for HPC"
-
Upload
eubrasilcloudforum- -
Category
Technology
-
view
70 -
download
0
Transcript of "Cloud Computing for HPC"
![Page 1: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/1.jpg)
1º WCN - Workshop on Cloud Networks
Collaborative Research in Cloud Computing: future and challenges
Cloud Computing for HPC Cloud Computing for HPC
Philippe O. A. NavauxPhilippe O. A. NavauxGPPD - Informatics Institute – UFRGSGPPD - Informatics Institute – UFRGS
Grupo de Processamento Paralelo e Distribuído
![Page 2: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/2.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 2
High performance applicationsHigh performance applications
Applications that require a lot of computing Fluid dynamics
Computational biology
Seismic models
Climatological models BRAMS and OLAM
HPC is traditionally obtained with the use of supercomputers, clusters and grids
![Page 3: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/3.jpg)
3
Fluids DinamicFluids Dinamic
Sterling e Brandt 2012
![Page 4: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/4.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 4
1997 2007DoE
Combustion EngineCombustion Engine
![Page 5: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/5.jpg)
5
Molecular DinamicMolecular Dinamic
Sterling e Brandt 2012
![Page 6: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/6.jpg)
6
HuricanesHuricanesDisasters PreventionDisasters Prevention
![Page 7: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/7.jpg)
Click to edit the outline text formatSecond Outline Level
Third Outline LevelFourth Outline Level
Fifth Outline LevelSixth Outline Level
Seventh Outline LevelAzure for Research Workshop
Click to edit the outline text format Second Outline Level
Third Outline LevelFourth Outline Level
Fifth Outline LevelSixth Outline Level
Seventh Outline LevelMay 25-26, 2015
“Numerous reports have documented the technical challenges and nonviability of simply scaling Numerous reports have documented the technical challenges and nonviability of simply scaling existing computer designs to reach exascale,” said Dongarra. “Drawing from these reports and existing computer designs to reach exascale,” said Dongarra. “Drawing from these reports and experience, our subcommittee has identified the top 10 computing technology advancements that experience, our subcommittee has identified the top 10 computing technology advancements that are critical to making a productive, economically viable exascale system.” (Dongarra 2014)are critical to making a productive, economically viable exascale system.” (Dongarra 2014)
Top 10 avancemments:Top 10 avancemments:• Energy efficiency,Energy efficiency,• Interconnect technology,Interconnect technology,• Memory Technology,Memory Technology,• Scalable System Software,Scalable System Software,• Programming systemsProgramming systems
Data managementData management
Exascale Algorithms,Exascale Algorithms,
Algorithms for discovery, design, Algorithms for discovery, design, and decision,and decision,
Resilience and correctness,Resilience and correctness,
Scientic productivity.Scientic productivity.
![Page 8: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/8.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 8
Supercomputers TitanSupercomputers Titan
![Page 9: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/9.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 9
Questions ?Questions ?
Can we process applications that need high performance processing on cloud?
E Science
Did the performance suitable?
![Page 10: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/10.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 10
Cluster vs CloudCluster vs Cloud
Traditional hardware Cluster Hardware cost
Hardware maintenance
People for maintenance
Cloud Computing Without acquisition and maintenance cost
Minimum staff
Pay per use
![Page 11: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/11.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 11
Challenges in applications migration to cloudChallenges in applications migration to cloud
Portability: Lack of application and code documentation, slow down migration. Scientific applications have a very specific problem domains. Migration requires a high level knowlegde of application . Applications developed without version control or comments creates a bottleneck in migration (legacy code)
Costs Porting application code to PaaS. Number of processing instances Acceptable time to solution
![Page 12: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/12.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 12
Challenges in applications migration to cloudChallenges in applications migration to cloud
Resources Network
Scaling, instance migration, resilience.
Storage Local and distributed file systems.
Processing Number of instances vs number of processors.
Applications CPU-bound, IO-bound, Network-bound.
![Page 13: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/13.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 13
Case: Weather ForecastingCase: Weather Forecasting
Numerical Weather Forecast
Used to accurately predict atmosphericbehavior for a future time period.Represents the state of the atmosphereat a xed time (temperature,wind components, etc) over a discretizeddomain.The results are denominated weatheror climate forecasts, a few days forweather, months for climate.
![Page 14: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/14.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 14
![Page 15: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/15.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 15
BRAMS Climate ModelBRAMS Climate Model
![Page 16: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/16.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 16
BRAMS ModelBRAMS Model
Brazilian Regional Atmospheric Modeling System
Mesoscale model based on
RAMS, developed by the Atmospheric
Science Department at the CSU,
adapted to Brasil Climate.
The main objective of BRAMS
is to provide a single model
to Brazilian Regional Weather
Centers. Each center execute
mesoscale forecasts in their geographical area.
![Page 17: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/17.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 17
BRAMS is a free BRAMS is a free softwaresoftware
![Page 18: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/18.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 18
BRAMS Ensemble ForecastBRAMS Ensemble Forecast
Ensemble Forecast
The impact of initial condition error
propagation in the forecast integration
time can be alleviated by
the ensemble forecast technique.
A set of independent integrations
are done, with small variations
in initial conditions among each
other, resulting in distinct forecasts.
![Page 19: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/19.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 19
BRAMS - ECMWF MethodBRAMS - ECMWF Method 12 executions per year (for 10 years)
![Page 20: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/20.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 20
BRAMS Result example: Rain PrecipitationBRAMS Result example: Rain Precipitation
![Page 21: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/21.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 21
BRAMS in Azure
Affinity groups.Wrapped in a Cloud ServiceProcessing VMs Created on demandFrontend is the only fixed instanceBilled by forecast runService is fully automatedData stored in Azure file service
![Page 22: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/22.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 22
Approach:Using Azure File Srvice:
Pros:•Legacy Compatibility•Using SMB•Highly scalable, replication•Easy to setup
Cons:•No file permissions•Throughput limited•(Preview)•Metadata is lost
![Page 23: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/23.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 23
Shared Usage
![Page 24: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/24.jpg)
Multiples instances of BRAMS in Cloud
72 hours
4 regions of South America
Sharing input data
![Page 25: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/25.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 25
Conclusions of HPC in Cloud: Climate caseConclusions of HPC in Cloud: Climate case
The main objective of the work was to migrate the Brazilian Regional
Atmospheric Modeling System (BRAMS) to a new platform that
uses the advantages provided by a cloud computing infrastructure
and evaluate the challenges of this procedure.
The goals of the work was:
Migrate Brams to Azure.
Measure the performance of the migrated version.
Analyse the cost of execution to achieve useful results.
Study the viability of usage of the implemented model by other weather centers.
![Page 26: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/26.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 26
Cloud and SupercomputingCloud and Supercomputing
Cloud Companies are installing Supercomputing
facilities to improve their processing power in
executing in cloud.
![Page 27: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/27.jpg)
1º WCN - Workshop on Cloud Networks July 06, 2016 27
Conclusion of HPC in Cloud and with Supercomputing Conclusion of HPC in Cloud and with Supercomputing facilitiesfacilities
- we can use HPC applications in cloud
-.the performance is not the same
- but cloud are changing including supercomputers features
- cloud hadd VM facilities
BUT
- interconnection is one of the challenges
- I/O is another challenge
![Page 28: "Cloud Computing for HPC"](https://reader033.fdocuments.net/reader033/viewer/2022052514/58ed3d051a28ab9b378b4647/html5/thumbnails/28.jpg)
1º WCN - Workshop on Cloud Networks
Thanks!Thanks!
Collaborative Research in Cloud Computing: future and challenges
Cloud Computing for HPC Cloud Computing for HPC
Philippe O. A. NavauxPhilippe O. A. NavauxGPPD - Informatics Institute – UFRGSGPPD - Informatics Institute – UFRGS
Grupo de Processamento Paralelo e Distribuído