Overview of High Performance Computing at KFUPM Khawar Saeed Khan ITC, KFUPM.

18
Overview of High Overview of High Performance Computing Performance Computing at KFUPM at KFUPM Khawar Saeed Khan Khawar Saeed Khan ITC, KFUPM ITC, KFUPM

Transcript of Overview of High Performance Computing at KFUPM Khawar Saeed Khan ITC, KFUPM.

Overview of High Overview of High Performance Computing Performance Computing

at KFUPM at KFUPM

Khawar Saeed KhanKhawar Saeed KhanITC, KFUPMITC, KFUPM

AgendaAgenda

►KFUPM HPC Cluster DetailsKFUPM HPC Cluster Details►Brief look at RHEL and Windows 2008 Brief look at RHEL and Windows 2008

HPC EnvironmentsHPC Environments►Dual Boot ConfigurationDual Boot Configuration► Job SchedulingJob Scheduling►Current and Soon to be available Current and Soon to be available

SoftwareSoftware►Expectations from usersExpectations from users

Why Cluster Computing andWhy Cluster Computing andSupercomputing?Supercomputing?

► Some Problems Larger Than Single Computer Can Some Problems Larger Than Single Computer Can ProcessProcess

► Memory Space (>> 4-8 GB)Memory Space (>> 4-8 GB)► Computation Cost Computation Cost ► More Iterations and Large Data setsMore Iterations and Large Data sets► Data Sources (Sensor processing)Data Sources (Sensor processing)► National PrideNational Pride► Technology Migrates to ConsumersTechnology Migrates to Consumers

How Fast Are How Fast Are Supercomputers?Supercomputers?

► The Top Machines Can Perform Tens of Trillions The Top Machines Can Perform Tens of Trillions Floating Point Operations per Second (TeraFLOPS)Floating Point Operations per Second (TeraFLOPS)

► They Can Store Trillions of Data Items in RAM!They Can Store Trillions of Data Items in RAM!► Example: 1 KM grid over USAExample: 1 KM grid over USA► 4000x2000x100 = 800 million grid points4000x2000x100 = 800 million grid points► If each point has 10 values, and each value takes If each point has 10 values, and each value takes

1010► ops to compute => 80 billion ops per iterationops to compute => 80 billion ops per iteration► If we want 1 hour timesteps for 10 years, 87600 If we want 1 hour timesteps for 10 years, 87600

itersiters► More than 7 Peta-ops total!More than 7 Peta-ops total!

Lies, Damn Lies, and Lies, Damn Lies, and StatisticsStatistics

► Manufacturers Claim Ideal PerformanceManufacturers Claim Ideal Performance► 2 FP Units @ 3 GHz => 6 GFLOPS2 FP Units @ 3 GHz => 6 GFLOPS► Dependences mean we won't get that much!Dependences mean we won't get that much!► How Do We Know Real PerformanceHow Do We Know Real Performance► Top500.org Uses High-Perf LINPACKTop500.org Uses High-Perf LINPACK► http://www.netlib.org/benchmark/hpl► Solves Dense Set of Linear EquationsSolves Dense Set of Linear Equations► Much Communications and ParallelismMuch Communications and Parallelism► Not Necessarily Reflective of Target AppsNot Necessarily Reflective of Target Apps

HPC in Academic InstitutionsHPC in Academic Institutions

►HPC cluster resources are no longer a HPC cluster resources are no longer a research topic but a core part of the research topic but a core part of the research infrastructure. research infrastructure.

► Researchers are using HPC clusters and are Researchers are using HPC clusters and are dependent on themdependent on them

► Increased competitiveness Increased competitiveness ► Faster time to researchFaster time to research► Prestige, to attract talent and grantsPrestige, to attract talent and grants► Cost-effective infrastructure spendingCost-effective infrastructure spending

Top Universities using HPC Top Universities using HPC ClustersClusters

► National Center for Supercomputing Applications at University National Center for Supercomputing Applications at University of Illinois Urbana Champagneof Illinois Urbana Champagne

► Texas Advanced Computing Center/University of Texas, Austin Texas Advanced Computing Center/University of Texas, Austin United StatesUnited States

► National Institute for Computational Sciences/University of Tennessee

United StatesUnited States► Information Technology Center, The University of Tokyo

JapanJapan► Stony Brook/BNL, New York Center for Computational Sciences

United StatesUnited States► GSIC Center, Tokyo Institute of Technology

JapanJapan► University of Southampton, UKUniversity of Southampton, UK► University of Cambridge, UKUniversity of Cambridge, UK► Oklahoma State University, USOklahoma State University, US

Top Research Institutes using Top Research Institutes using HPC ClustersHPC Clusters

► DOE/NNSA/LANLUnited StatesUnited States

► Oak Ridge National LaboratoryUnited StatesUnited States

► NASA/Ames Research Center/NASUnited StatesUnited States

► Argonne National LaboratoryUnited StatesUnited States

► NERSC/LBNLNERSC/LBNLUnited StatesUnited States

► NNSA/Sandia National LaboratoriesNNSA/Sandia National LaboratoriesUnited StatesUnited States

► Shanghai Supercomputer CenterShanghai Supercomputer CenterChinaChina

KFUPM HPC EnvironmentKFUPM HPC Environment

HPC @ KFUPMHPC @ KFUPM

► Planning & Survey started in early 2008Planning & Survey started in early 2008► Procured in October 2008Procured in October 2008► Cluster Installation and Testing during Nov-Dec-JanCluster Installation and Testing during Nov-Dec-Jan► Applications like Gaussian with Linda, DL-POLY, Applications like Gaussian with Linda, DL-POLY,

ANSYS tested on the cluster setupANSYS tested on the cluster setup► Test problems were provided by professors of Test problems were provided by professors of

Chemistry, Physics, Mechanical Engg. Departments.Chemistry, Physics, Mechanical Engg. Departments.► More applications on the cluster will be installed More applications on the cluster will be installed

shortly e.g., GAMESS-UK.shortly e.g., GAMESS-UK.

KFUPM Cluster HardwareKFUPM Cluster Hardware

► HPC IBM Cluster 1350 HPC IBM Cluster 1350 ► 128 nodes, 1024 Cores.128 nodes, 1024 Cores.

Master NodesMaster Nodes► 3x Xeon E5405 Quad-Core, 8 GB, 2x 500 GB HD (mirrored)3x Xeon E5405 Quad-Core, 8 GB, 2x 500 GB HD (mirrored)

Compute NodesCompute Nodes► 128 nodes(IBM 3550 rack mounted). Each node dual 128 nodes(IBM 3550 rack mounted). Each node dual

processor, Quad-Core Xeon E5405 (2 GHz). 8 GB RAM, 64TB processor, Quad-Core Xeon E5405 (2 GHz). 8 GB RAM, 64TB total local storage.total local storage.

► Interconnect 10GB Ethernet. Uplink 1000-Base-T Gigabit.Interconnect 10GB Ethernet. Uplink 1000-Base-T Gigabit.

Operating Systems for Compute nodes (Dual Boot)Operating Systems for Compute nodes (Dual Boot)► Windows HPC Server 2008 and Red Hat Linux 5.2.Windows HPC Server 2008 and Red Hat Linux 5.2.

Dual Boot clustersDual Boot clusters

► Choice of the right operating system for a HPC cluster can be Choice of the right operating system for a HPC cluster can be a very difficult decision. a very difficult decision.

► This choice will usually have a big impact on the Total Cost of This choice will usually have a big impact on the Total Cost of Ownership (TCO) of the cluster. Ownership (TCO) of the cluster.

► Parameters like multiple user needs, application environment Parameters like multiple user needs, application environment requirements and security policies add to the complex human requirements and security policies add to the complex human factors included in training, maintenance and support factors included in training, maintenance and support planning, all leading to associated risks on the final return on planning, all leading to associated risks on the final return on investment (ROI) of the whole HPC infrastructure. investment (ROI) of the whole HPC infrastructure.

► Dual Boot HPC clusters provide two environments (Linux and Dual Boot HPC clusters provide two environments (Linux and Windows in our case) for the price of one.Windows in our case) for the price of one.

Key takeaways:Key takeaways:

- Mixed clusters provide a low barrier to Mixed clusters provide a low barrier to leverage HPC related hardware, software, leverage HPC related hardware, software, storage and other infrastructure storage and other infrastructure investments better – investments better – “Optimize, flexibility “Optimize, flexibility of infrastructure”of infrastructure”

- Maximize the utilization of compute Maximize the utilization of compute infrastructure by expanding the pool of infrastructure by expanding the pool of users accessing the HPC cluster resources - users accessing the HPC cluster resources - “Ease of use and familiarity breeds usage”“Ease of use and familiarity breeds usage”

Possibilities with HPCPossibilities with HPC

►Computational fluid dynamicsComputational fluid dynamics►Simulation and ModelingSimulation and Modeling►Seismic tomographySeismic tomography►Nano SciencesNano Sciences►VizualizationVizualization►Weather ForecastingWeather Forecasting►Protein / Compound SynthesisProtein / Compound Synthesis

Available SoftwareAvailable Software

►Gaussian with LindaGaussian with Linda►ANSYSANSYS► FLUENTFLUENT► Distributed MATLAB,Distributed MATLAB,►MathematicaMathematica►DL_POLYDL_POLY►MPICHMPICH►Microsoft MPI SDKMicrosoft MPI SDK► The following software will also be made available in the near future.The following software will also be made available in the near future.► Eclipse, GAMESS-UK, GAMESS-US, VASP and NW-CHEMEclipse, GAMESS-UK, GAMESS-US, VASP and NW-CHEM

Initial Results of Beta TestingInitial Results of Beta Testing

►Few applications like Gaussian etc Few applications like Gaussian etc have been beta tested and have been beta tested and considerable speed up in computing considerable speed up in computing times has been reportedtimes has been reported

►MPI program run tested on the cluster MPI program run tested on the cluster reported considerable speed up as reported considerable speed up as compared to serial server runs. compared to serial server runs.

HPC @ KFUPMHPC @ KFUPM

► Several FirstsSeveral Firsts► Dual Boot ClusterDual Boot Cluster► Supports RedHat Linux 5.2 and Windows 2008 HPC Supports RedHat Linux 5.2 and Windows 2008 HPC

Server Server ► Capability to support variety of applicationsCapability to support variety of applications► Parallel Programming SupportParallel Programming Support► Advanced Job Scheduling optionsAdvanced Job Scheduling options

ExpectationsExpectations

►Own the systemOwn the system►Respect other’s jobsRespect other’s jobs►Assist ITC HPC team by searching and Assist ITC HPC team by searching and

sending complete installation, software sending complete installation, software procurement and licensing procurement and licensing requirementsrequirements

►Help other users by sharing your Help other users by sharing your experienceexperience Use Vbulletin at http://hpc.kfupm.edu.sa/ Use Vbulletin at http://hpc.kfupm.edu.sa/