quantum ESPRESSO: first-principles simulationsat the nanoscale (and towards the exascale)

Paolo GiannozziUniversita di Udine, Italy

First HPC workshop, Master in High Performance ComputingSISSA, Trieste, 2016/02/25

– Typeset by FoilTEX –

What is quantum ESPRESSO?

quantum ESPRESSO stands for Quantum opEn-Source Package for Researchin Electronic Structure, Simulation, and Optimization

quantum ESPRESSO is a distribution (an integrated suite) of software foratomistic calculations based on electronic structure, using density-functional theory,a plane-wave basis set, pseudopotentials.It is freely available under the terms of the GNU General Public License

Main goals of quantum ESPRESSO are

• innovation in theoretical methods and numerical algorithms

• efficiency on modern computer architectures

A great effort is also devoted to user friendliness and to the formation of a users’and developers’ community

quantum ESPRESSO exists since 2002, resulting from the merge of pre-existingpackages; some core components have been under development for ∼ 30 years

At the nanoscale: new materials

Most common atomicconfigurations in amorphousCdTeOx, x = 0.2; Phys. Rev.B 79, 014205 (2009).

At the nanoscale: new devices

Organic-inorganic semiconductor heterojunction, phtalocyanine over TiO2 anatasesurface; Chem. Mater. 21, 4555 (2009).

At the nanoscale: nanocatalysis

Cobalt-base catalyser for water splitting: J. Am. Chem. Soc. 135, 15353 (2013)

At the nanoscale: biological systems

Metal-β-amyloid interactions; Metallomics 4, 156 (2012).

Towards the exascale: massive parallelization

C@Ir(001)443  atoms2987  electrons

... still not forgettingsmaller machines! In thefigure, Nicola Marzari’ssmartphone runningquantum ESPRESSO

quantum ESPRESSO contributors

quantum ESPRESSO receives contributions from many individuals and partnerinstitutions in Europe and worldwide. Who “owns” quantum ESPRESSO?

quantum ESPRESSO Foundation

The quantum ESPRESSO Foundation: a non–profit (“limited by guarantee”)company, based in London, that

• coordinates and supports research, education, and outreach within thequantum ESPRESSO community

• owns the trademarks and protects the open-source character of quantumESPRESSO

• raises funds to foster the quantum ESPRESSO project and its development

quantum ESPRESSO Foundation Members

Current QEF members:

• Scuola Internazionale Superiore di Studi Avanzati (SISSA), Trieste

• Ecole Polytechnique Federale de Lausanne (EPFL)

• International Centre for Theoretical Physics (ICTP), Trieste

• Consiglio Nazionale delle Ricerche (IOM-CNR), Italy

• CINECA supercomputing center, Bologna

• University of North Texas

• Duke University

Development

The distribution is maintained as a single SVN (Subversion) tree. Slowly movingtowards a less centralized development model based on git.

• Web site: http://www.quantum-espresso.org

• Developers’ portal: http://www.qe-forge.org

Mailing list (public):

• pw forum@pwscf.org: for general discussions

More mailing lists (for developers only):

• qe developers@qe-forge.org: for technical discussions

• qe commits@qe-forge.org: receives commit messages

Developers’ portal: qe-forge

Currently 45 public projects, 570 registered users, 66 QE developers registeredQE-forge will undergo a significant overhaul in the near future

Users’ community: factoids

• About 1800 registered users for thepw forum mailing list

• An average of ∼ 10 messages a days onpw forum

• latest version (5.3.0) downloaded 6700times [*] since release date (Jan.9)

• 30 Schools or tutorials since 2002, attended by ∼ 1200 users

• 3 developers’ schools since 2013, latest in January 2015

• annual developers’ meeting since 2010, latest in January 2016

[*] this number may be inflated by bots, failed downloads, etc.

Schools and tutorial using quantum ESPRESSO

More: Penn State, June 2014; University of Tokyo, April 2014; Pune, July 2014;Cordoba (Argentina), September 2015

4400 citations as of today

Requirements on effective softwarefor quantum simulations at the nanoscale

• Challenging calculations stress the limits of available computer power: softwareshould be fast and efficient

• Diffusion of first-principle techniques among non-specialists requires softwarethat is easy to use and (reasonably) error-proof

• Introducing innovation requires new ideas to materialize into new algorithmsthrough codes: software should be easy to extend and to improve

• Complex problems require a mix of solutions coming from different approachesand methods: software should be interoperable with other software

• Finaly, scientific ethics requires that results should be reproducible andalgorithms susceptible of validation

Structure of the distribution

Technical characteristics (coding)

• 380000+ Fortran-95 lines, with various degrees of sophistication (i.e. use ofadvanced f95 features) – no “dusty decks” any longer

• use of standard library routines (lapack, blas, fftw) to achieve portability –Machine-optimized libraries can (should) (must!) be used if available

• C-style preprocessing options allow to keep a single source tree for (almost)architectures, from PC’s to BG’s (BlueGene)

• various parallelization levels via MPI calls or OpenMP directives, hidden intocalls to a few routines – almost unified serial and parallel versions; parallel codecan (usually) be written without knowing the details of how parallelism works.

• I/O based on an XML file plus binary files for large records (e.g. wavefunctions,charge density) – soon moving to schema-based, standard-compliant XML andportable (HDF5) binary files

Parallelization of quantum ESPRESSO

Several parallelization levels areimplemented; most of them require fastinterprocess communications.

Scalability of realistic calculations onup to tens of thousands cores, usingmixed MPI-OpenMP parallelization,has been demonstrated.

Careful optimization of nonscalableRAM and computations required!Scalability strongly depends upon thekind and size of system!

CP Scalability on BG/Q, 1532-atom porphyrin-functionalized carbon nanotube(data from paper appearing in next slide)

Summary of parallelization levels

Summary of parallelization levels (2)

group distributed quantities communications performances

image NEB images, very low linear CPU scaling,phonon modes fair to good load balancing;

does not distribute RAMpool k-points low almost linear CPU scaling,

fair to good load balancing;may distribute some RAM

bands Kohn-Sham orbitals high improves scalingplane- PW, G-vector coefficients, high good CPU scaling,wave R-space FFT arrays good load balancing,

distributes most RAMtask FFT on electron states high improves load balancinglinear- subspace hamiltonians very high improves scaling,algebra and constraints matrices distributes more RAMOpenMP FFT, libraries intra-node extends scaling on

multicore machines

Importance of collaboration with computing centers

DEISA EXTREME COMPUTING INITIATIVE

S. Corni, A. Calzolari, G. Cicero, C. Cavazzoni, A. Catellani and R. Di Felice

Density map of Oxygen in the hydration layers

(a-c) Löwdin charges (dq) for selected atoms as a

function of the positions, calculated with respect to the corresponding formal

atomic values

Ab-initio simulations of Protein-Surface Interactions mediated by WATer

quantum ESPRESSO on GPU’s

Perspectives and Outlook

• More packages for advanced methodologies

• Better-structured distribution, with interfaces to external codes and to pythonscripting

• Porting to new hybrid and accelerated architectures

• More parallelization everywhere, communication-reducing and latency-hidingalgorithms

quantum ESPRESSO is one of the community codes involved into the recentlyfunded MaX – Materials at the Exascale EU project (next talk)

Credits

• Thanks to all people whose slides and pictures I borrowed

• Thanks to all people who contributed to quantum ESPRESSO

• ...and thanks to you all