7/29/2019 Computational Astrophysics

1/20

William DorlandDepartment of Physics

University Honors Program

University of Maryland

Computational Astrophysics

7/29/2019 Computational Astrophysics

2/20

In collaboration with

Ramani Duraiswami, Nail Gumerov,

Kate Despain, Derek Juba,Yuancheng Luo,Amitabh Varshney,

George Stantchev, Bill Burns,

Frank Herrmann, John Silberholz,

Matias Bellone, Manuel Tiglio

7/29/2019 Computational Astrophysics

3/20

An intensive summer programfor graduate studentsand postdoctoral fellows

July 13-24, 2009

ComputationalAstrophysics

7/29/2019 Computational Astrophysics

4/20

Overview Maryland astrophysics effort: what, why

Results of integration: Middleware library to acceleratedevelopment in Fortran 9x: Flagon

Maryland astromap

7/29/2019 Computational Astrophysics

5/20

In collaboration with

Ramani Duraiswami, Nail Gumerov,

Kate Despain, Derek Juba,Yuancheng Luo, Amitabh Varshney,

George Stantchev, Bill Burns,

Frank Herrmann, John Silberholz,

Matias Bellone, Manuel Tiglio

7/29/2019 Computational Astrophysics

6/20

Binary Black Holes with GPUs

Gravitational wave astronomy requires sourcecharacterization -- large-scale problem in numericalrelativity

Project: determine final relative orientations of spinaxes for black holes of different masses as a function ofinitial conditions -- 7-D ensemble, expensive

7/29/2019 Computational Astrophysics

7/20

Binary Black Holes with GPUs

Gravitational wave astronomy requires sourcecharacterization -- large-scale problem in numericalrelativity

Project: determine final relative orientations of spinaxes for black holes of different masses as a function ofinitial conditions -- 7-D ensemble, expensive

7/29/2019 Computational Astrophysics

8/20

Binary Black Holes with GPUs

Development on local Teslas [now production-ready]

60x speed-up achieved on MPI+CUDA-enabled MonteCarlo sampling of 7-D space (systems of ODEs, post-

Newtonian approx) Next step: 0.5M hours on NCSA Lincoln cluster

1536 cores, 384 Teslas

Production cluster (in addition to new localresource)

7/29/2019 Computational Astrophysics

9/20

For more info today:

Frank Herrmann, John Silberholz,

Matias Bellone, Manuel Tiglio

7/29/2019 Computational Astrophysics

10/20

Milky Way in X-Rays from Chandra

7/29/2019 Computational Astrophysics

11/20

Pulsar in Crab Nebula: Chandra, Hubble

7/29/2019 Computational Astrophysics

12/20

Everything you can see in these pictures is associated

with super-hot, turbulent plasma (ionized gas).

To understand the details, one needs to understandhow astrophysical objects turn gravitational energy

into light.

This requires high-performance computing, now beingaccelerated with NVidia GPUs.

7/29/2019 Computational Astrophysics

13/20

Key Questions Addressed

How stable is the hardware + software platform?

Which major algorithms can successfully exploit GPUhardware?

What is the development and maintenance path forporting large-scale applications to this platform? Is therea net benefit? Or is the programming model toocumbersome?

Which algorithms can exploit clusters of GPUs?

Aiming for a $1-5M purchase in 2009-10 timeframe.CPU? GPU? Which vendor?

7/29/2019 Computational Astrophysics

14/20

Refuting the Moores law argument

Argument ~ Wait for processor performance to solve

complex problems, without algorithm improvement

Is this true?

Yes, for algorithms with linear asymptotic complexity

No!! For algorithms with different aymptotic complexity

Most scientific algorithms ~ or

For a million variables, we would need about 16generations of Moores law before an algorithm wascomparable with an O(N) algorithm.

Implies need for sophisticated algorithms, but are theyprogrammable on a GPU?

N2

N3

3

N2

7/29/2019 Computational Astrophysics

15/20

Turbulence theory is guided by

computation/simulation Properties of plasma

turbulence are important in

laboratory experiments (such

as fusion research), in spacephysics (solar wind), and in

astrophysics (ISM, accretion

flow luminosity)

Most computations from

Maryland carried out atnational supercomputing

facilities, on hundreds to

thousands of processors.

Fusion turbulence

7/29/2019 Computational Astrophysics

16/20

Turbulence theory is guided by

computation/simulation Properties of plasma

turbulence are important in

laboratory experiments (such

as fusion research), in spacephysics (solar wind), and in

astrophysics (ISM, accretion

flow luminosity)

Most computations from

Maryland carried out atnational supercomputing

facilities, on hundreds to

thousands of processors.

Accretion flow luminosity[Goldston, Quataert, Igumenshchev, 2005]

7/29/2019 Computational Astrophysics

17/20

Turbulence theory is guided by

computation/simulation Properties of plasma

turbulence are important in

laboratory experiments (such

as fusion research), in spacephysics (solar wind), and in

astrophysics (ISM, accretion

flow luminosity)

Most computations from

Maryland carried out atnational supercomputing

facilities, on hundreds to

thousands of processors. GPU Goal

7/29/2019 Computational Astrophysics

18/20

Conversion of legacy Fortran 9x codeimportant to scientific community

Maintainability, portability

Developed Flagon, an F9xwrapper to ease transition to

CUDA

Flagon available at Sourceforgeas Open Source project

7/29/2019 Computational Astrophysics

19/20

GPU Performance Achievable

by Non-Expert

Plasma turbulence

code (Orzag-Tang

problem) ported from

existing Fortran 95

code in one day

achieves 25x speedup

GPU non-expert (me)

Expert (Despain) then

upped to 32x

7/29/2019 Computational Astrophysics

20/20

Starmap

Upgrading current clusters, testing Teslas and looking at OpenCL

Major questions remaining to be answered:

1. Multi-GPU computing necessary for astro apps we care about

Current and continuing focus

2. Is development platform rich enough for GPU non-experts?