Claude TADONKIMines ParisTech PSLHigh Performance Computing Challenges and Trends

High Performance Computing: Challenges and Trends Claude TADONKIThe need of competitive HPC systemsKey FactorsLarge-scale scientific & technical computing (numerical and non numerical) Large-scale data mining and statistics in experimental physicsImage and signal processingVideo and 3D animationsMolecular biology and structural genomicSorting and pattern matchingMeteorology, Atmospheric studies, Medical researchScientific and technical simulationHigh-standard industrial activities, services, and research investigationsGaming And more Massively parrallel computers & systemsDedicated architectures Specialized processors Processor frequency High level integrationMemory space, latency, and bandwidth High speed interconnection Advances in parallel algorithm synthesis and programming language featuresPowerful compilers Increasing Need of HPC (range of applications and computing power)

High Performance Computing: Challenges and Trends Claude TADONKILastest Top500

High Performance Computing: Challenges and Trends Claude TADONKILastest Top500 in Latin AmericaN1 in Latin America is in Brazil at SENAI-CIMATEC (Salvador de BAHIA)

High Performance Computing: Challenges and Trends Claude TADONKIAbsolute Performance

High Performance Computing: Challenges and Trends Claude TADONKIParallel computing easily justifies nowadays (processor frequency evolution)HPC interest covers a larger spectra of applications, hence a wider audiencePerformance expectations in some specific areas are beyond the capacity of standard computers Frequency has been multiplied by 10 since 1993 The number of transistors in Intel proc has been multiplied by 100 000 since 1971 Core voltage has been reduced by 10 (1,2V , the min is 0,7V)

However, this evolution is closed to its asymptotic threshold !!!Alternatives and trendsMulti-core processorsTLP-based parallel machinesGPU (assume a skillful use!)Hardcoded (embedded) solutionsReconfigurable HPCGrid/Meta ComputingSmallpox Research Grid (Oxford University/IBM/United Devices) Oustanding achievment in large-scale molecular analysis.Observations

High Performance Computing: Challenges and Trends Claude TADONKIhttp://www.fujitsu.com/global/news/pr/archives/month/2011/20110620-02.htmlThe K Computer RIKEN / Fujitsu ( JAPAN )Number one the 37th TOP5008.162 petaflops (Linpack) 93%68,544 energy-efficient CPUs672 computer racksComplete deployment in 201210 petaflops expected (2012)800 computer racks (2012)Worldwide shared use

High Performance Computing: Challenges and Trends Claude TADONKIhttps://asc.llnl.gov/computing_resources/sequoia/The Sequoia System LLNL / IBM ( USA )BlueGene technology500 teraFLOPS1.6 million of cores96 computer racks98,304 compute nodes1.6 petabytes of memory10 times faster than todaysMost powerful systemSequoia in 1 hour=6.7 billion people calculating 24h/24h during 320 years

High Performance Computing: Challenges and Trends Claude TADONKIAbout sustained performanceBefore we calculate, we need dataTime-to-CPU could be long % flopsMemory space decreases with levelShared use of memory slowdownSynchronization and data exchangeControl flow (if, while, for, case, )Even an optimal algorithm will run at a fraction of the peak performance !!!

High Performance Computing: Challenges and Trends Claude TADONKIImportant considerationsEnergy consumption and dissipationIntegration ( more powerful unit/system in a smaller chip/surface )Total cost of the system and maintenance issuesProgrammability (consider the case of the IBM CELL)Accessibility (remote and shared use among several entities) Software and tools (system, programming, monitoring, profiling, )Lifetime and evolution of the system (extensibility, devices change/upgrade, ) Computing nodes interconnection (topology and speed)vital on embedded systemssignificant heat and cause of failure

High Performance Computing: Challenges and Trends Claude TADONKIFundamental aspectsDesign of efficient parallel algorithms (modelling & scheduling)Complexity & Performance analysisAlgorithmic and programming paradigmsCode generation and transformations (automatic parallelization)Compilation techniques

High Performance Computing: Challenges and Trends Claude TADONKIProgramming modelsDistributed memory parallel programming (MPI)Shared memory parallel programming (OpenMP, Pthreads)Accelerator-based programming (GPU, FPGA, CELL, )Vector programming (SSE, VMX, SPU intrinsics, )Hybrid programming (MPI+OpenMP/Pthread, CPU+GPU, PPU+SPU, )

High Performance Computing: Challenges and Trends Claude TADONKIHorizonExscale (around 2018)Parallel CPU-GPUParallel GPUHigh Aggregation of Computing ResourcesStill fighting with Energy and Memory/Network overhards

High Performance Computing: Challenges and Trends Claude TADONKI

**************