Virtual Test Facility: Materials Properties

ASCI Research ReviewJanuary 25, 1999

Scalability of QM Code

ASCI Research ReviewJanuary 25, 1999

QM Methodology (Jaguar)

•Psuedospectral Technology (with Columbia U.)Multigrids Dealiasing functions

•Replace N4 4-center Integrals with N3 potentials•Use Potentials to Form Euler-Lagrange Operator:

•CURRENT STATUS: •Single processor speed 9 times faster than best alternate methodology•Scales a factor of N2 better than best alternate methodology

17

Log (number basis functions)

Jaguar

Gaussian

CPU

Tim

e

Collaboration with Columbia U. and Schrödinger Inc.

QM Scalability: IBM SP2

QM Scalability: SGI (Blue Mountain)

QM Scalability: Comments• Clearly scaling needs work on Blue Mountain• Algorithm ill-suited to massive parallelizability

– Seriel diagonalization– Local data

• Two steps in Quantum Chemistry– Hamiltonian H formation– H diagonalization to produce density – Because H is a function of , this is a nonlinear problem

• Linearization and parallelization in Quantum Chemistry requires techniques to localize the density.– Modified Divide-and-Conquer technique– Solves the H-formation and H-diagonalization problems– Generalize to metallic systems

Improved DiagonalizationSeries of alkane chains, 276-552 basis functions, bandwidth ~80 basis functions

0

0.5

1

1.5

2

2.5

3

2.440909082 2.741939078

log(NBF)

log(t)

Band Diag: scales good (N2.3) but overhead too high

Normal Diag: scales poorly (N3.3) but generally efficient

Block Diag: scales best (<N2) but generalization problems

Fock Matrix

Divide and Conquer

F F F F F F F F F

F F F F F F F F

F F F F F F F F F

Cl Cl Cl Cl Cl Cl Cl Cl

ActiveZone

BufferZone

BufferZone

H

Hamiltonian:Divided into fragments andbuffer zones

nbf

Divide and Conquer Shortcomings

• GOOD:– Solves H-formation, H-diagonalization, and parallelization

simultaneously!

• BAD if: Correlation lengths > fragment size!– Metals, surfaces, conjugated systems

• Must hierarchically correct error in fragments– Pairwise recombination of fragments to yield larger fragments– Hierarchically combine larger fragments to yield still-larger fragments– Continue until converged– At each level, include additional H elements:

• Few, since fall off as 1/r3 (dipole potential)

Divide, Conquer, and Recombine

A

B

ifragment

aatoms

XCa

Ja

Nai

fieldexternal

BAA

VVVv

vvtH

ˆˆˆˆ

ˆˆˆˆ

increasesRasignore

BAAB

ab

vvtH

Eigenvalue Solving Going Up • Already have eigs of HA and HB. Make good guess at eigs of H(A+B)

• Can use fast (linear) diagonalization:• Krylov-space• Conjugate gradient

• Don’t have to do O(N3) diagonalization

fieldexternal

ABB vvtH ˆˆˆ

Petaflop Dreaming

• Tahir’s MD shock simulator with QM– 10,000,000 atoms on 1,000,000 processors 10 atoms/processor

• depends upon ability to divide-and-conquer• simulate real chemistry: full species, bonds breaking, diffusion...

– Shock wave travels 0.1 m in 100 ps• time step ~1 fs require 100,000 time steps• 1 time step takes 300 s• need 30,000,000 s = 10,000 hr = 1 year

– Greatly simplify model using FF for unshocked region• Factor of 100• 100 hr calculation!

10 nm

10 nm0.1 m

HMX

MP Software Integration Issues

ASCI Research ReviewJanuary 25, 1999

Intra-MP Software Integration Issues

• Developing PUMP (Parallel Unified Materials Properties Interface)– Python-based framework to allow QM, MD, and D programs to

transparently communicate.– Combine with simple OpenInventor-based graphics.– Combine with Thornley S-threads to allow load balancing on Intel

shared memory boxes.– Combine with MPI to allow parallel execution.

PUMP

MD DQM

Properties

Visualization

Blue Mountain ASCI Red Blue Pacific

CALTECHComputing

EnvironmentMSC & CACR

MP-Applications Integration Issues

• Materials Property Database Under Construction• Need General Ways of Exchanging Complex Data

– FF, EOS with HE– Reaction Mechanisms with HE– FF, EOS with SD/CT

• Include in PUMP ability to write different archive formats– CVS archiving capabilities– Interface with Matlab/Python mathematical ability to derive data– XML-based web pages/publication of data

Extending Nitramine Reaction Pathways

ASCI Research ReviewJanuary 25, 1999

Additions to HE Reaction Kinetics

• GRI Nitromethane Mechanism– Right physics for small (C2NO2) species, but no HMX, RDX, TATB

• Add in Yetter (Princeton) RDX Decomposition Pathways– Comb. Sci. Tech., 1997, 124, pp. 25-82

• Determine analogous HMX Pathways• Compute themochemical properties for all new species• Final mechanism:

– 66 species– 414 reactions

RDX Decomposition Steps

N N

N

NO2

NO2O2N

N N

N

NO2O2N

N N

N

NO2O2N

H2C N

H2C N NO22

N N

N

NO2O2NN N

N

NO2O2N

HMX Decomposition Steps

N

N

N

N

NO2

O2N

O2N H2C N

H2C N NO23

N

N

N

N

NO2

O2N

O2NN

N

N

N

NO2

O2N

O2N

N

N

N

N

NO2

NO2

O2N

O2NN

N

N

N

NO2

O2N

O2N

New Species Required in Mechanism

N N

N

NO2

NO2O2N

N N

N

NO2O2N

N

N

N

N

NO2

NO2

O2N

O2N

N

N

N

N

NO2

O2N

O2N

RDX

RDXR

RDXRO

HMX

HMXR

HMXRO

N

N

N

N

NO2

O2N

O2N

N N

N

NO2O2N

Fit NASA Parameters to QM Calculations

• Obtain thermochemistry from QM– Get QM structure at B3LYP/6-31G** level– Compute/scale frequencies– Obtain Cp, S, H from 300 - 6300 K

• Fit to NASA standard form for thermochemical data:

TaTaTaTaTaa

RTH

aTaTaTaTaTaRS

TaTaTaTaaR

Cp

645342321

7453423

21

45

34

2321

5432

432ln

Heat Capacity Fit

Entropy Fit

Enthalpy Fit

Testing the Mechanism

• CV Calculations– T = 1500 K– P = 1-100000 atm

• Species Profiles• Induction Times

RDX/HMX Induction Times vs. Pressure

RDX Combustion, P = 1000 atm

HMX Combustion, P = 1000 atm

Next HE Steps...

• TATB and PETN Decomposition Steps

• F-containing species important in binder

– Same fraction of F and Cl as binder– Explore reactions of intermediates

NH2

NO2

NH2

NO2

H2N

O2NO

O2N

O

NO2

O

O2N

OO2N

F

ClF

F