Gengbin Zheng Parallel Programming Laboratory University of Illinois at Urbana-Champaign.
Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei...
-
Upload
dwayne-mcdaniel -
Category
Documents
-
view
219 -
download
1
Transcript of Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei...
![Page 1: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/1.jpg)
Overcoming Scaling Challenges in Bio-molecular Simulations
Abhinav BhateléSameer KumarChao MeiJames C. PhillipsGengbin ZhengLaxmikant V. Kalé
![Page 2: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/2.jpg)
2
Outline
• NAMD: An Introduction• Scaling Challenges
– Conflicting Adaptive Runtime Techniques– PME Computation– Memory Requirements
• Performance Results• Comparison with other MD codes• Future Work and Summary
![Page 3: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/3.jpg)
3
What is NAMD ?
• A parallel molecular dynamics application• Simulate the life of a bio-molecule• How is the simulation performed ?
– Simulation window broken down into a large number of time steps (typically 1 fs each)
– Forces on every atom calculated every time step
– Velocities and positions updated and atoms migrated to their new positions
![Page 4: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/4.jpg)
4
How is NAMD parallelized ?
Patches
Computes
HYBRIDDECOMPOSITION
![Page 5: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/5.jpg)
5
BondedComputes
Non- bondedComputes
Patch Integration
Patch Integration
Reductions
MulticastPoint to Point
Point to Point
PME
![Page 6: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/6.jpg)
6
What makes NAMD efficient ?
• Charm++ runtime support– Asynchronous message-driven model– Adaptive overlap of communication and
computation
![Page 7: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/7.jpg)
7
BondedComputes
Non- bondedComputes
Patch Integration
Patch Integration
Reductions
MulticastPoint to Point
Point to Point
PME
Non-bonded Work
Bonded Work Integration
PME
Communication
![Page 8: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/8.jpg)
8
What makes NAMD efficient ?
• Charm++ runtime support– Asynchronous message-driven model– Adaptive overlap of communication and
computation
• Load balancing support– Difficult problem: balancing heterogeneous
computation– Measurement-based load balancing
![Page 9: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/9.jpg)
9
What makes NAMD highly scalable ?
Patch
Compute
• Hybrid decomposition scheme• Variants of this hybrid scheme used by Blue
Matter and Desmond
![Page 10: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/10.jpg)
10
Scaling Challenges
• Scaling a few thousand atom simulations to tens of thousands of processors– Interaction of adaptive runtime techniques– Optimizing the PME implementation
• Running multi-million atom simulations on machines with limited memory– Memory Optimizations
![Page 11: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/11.jpg)
11
Conflicting Adaptive Runtime Techniques
• Patches multicast data to computes• At load balancing step, computes re-assigned to
processors• Tree re-built after computes
have migrated
![Page 12: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/12.jpg)
12
![Page 13: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/13.jpg)
13
![Page 14: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/14.jpg)
14
• Solution– Persistent spanning trees– Centralized spanning tree creation
• Unifying the two techniques
![Page 15: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/15.jpg)
15
PME Calculation
• Particle Mesh Ewald (PME) method used for long range interactions– 1D decomposition of the FFT grid
• PME is a small portion of the total computation– Better than the 2D decomposition for small
number of processors
• On larger partitions– Use a 2D decomposition– More parallelism and better overlap
![Page 16: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/16.jpg)
16
Automatic Runtime Decisions
• Use of 1D or 2D algorithm for PME • Use of spanning trees for multicast• Splitting of patches for fine-grained
parallelism• Depend on:
– Characteristics of the machine– No. of processors– No. of atoms in the simulation
![Page 17: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/17.jpg)
17
Reducing the memory footprint
• Exploit the fact that building blocks for a bio-molecule have common structures
• Store information about a particular kind of atom only once
![Page 18: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/18.jpg)
18
O
H
H
O
H
H
14333
14332
14334
14496
14495
14497
O
H
H
O
H
H
-1
0
+1
-1
0
+1
![Page 19: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/19.jpg)
19
Reducing the memory footprint
• Exploit the fact that building blocks for a bio-molecule have common structures
• Store information about a particular kind of atom only once
• Static atom information increases only with the addition of unique proteins in the simulation
• Allows simulation of 2.8 M Ribosome on Blue Gene/L
![Page 20: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/20.jpg)
20
Memory Reduction
0.01
0.1
1
10
100
1000IA
PP
DH
FR
Ly
so
zym
e
Ap
oA
1
F1
-A
TP
as
e
ST
MV
Ba
rD
om
ain
Rib
os
om
e
Benchmark
Mem
ory
Usa
ge
(MB
)
OriginalNew
< 0.5 MB
![Page 21: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/21.jpg)
21
NAMD on Blue Gene/L
1 million atom simulation on
64K processors(LLNL BG/L)
![Page 22: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/22.jpg)
22
NAMD on Cray XT3/XT4
5570 atom simulation on
512 processors at 1.2 ms/step
![Page 23: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/23.jpg)
23
Comparison with Blue Matter
• Blue Matter developed specifically for Blue Gene/L
Time for ApoA1 (ms/step)
• NAMD running on 4K cores of XT3 is comparable to BM running on 32K cores of BG/L
![Page 24: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/24.jpg)
25
Comparison with Desmond
• Desmond is a proprietary MD program
Time (ms/step) for Desmond on 2.4 GHz Opterons and NAMD on 2.6 GHz Xeons
• Uses single precision and exploits SSE instructions
• Low-level infiniband primitives tuned for MD
![Page 25: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/25.jpg)
28
Future Work
• Reducing communication overhead with increasing fine-grained parallelism
• Running NAMD on Blue Waters– Improved distributed load balancers– Parallel Input/Output
![Page 26: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/26.jpg)
29
Summary
• NAMD is a highly scalable and portable MD program– Runs on a variety of architectures– Available free of cost on machines at most
supercomputing centers– Supports a range of sizes of molecular systems
• Uses adaptive runtime techniques for high scalability
• Automatic selection of algorithms at runtime best suited for the scenario
• With new optimizations, NAMD is ready for the next generation of parallel machines
![Page 27: Overcoming Scaling Challenges in Bio-molecular Simulations Abhinav Bhatelé Sameer Kumar Chao Mei James C. Phillips Gengbin Zheng Laxmikant V. Kalé.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f1b5503460f94c30754/html5/thumbnails/27.jpg)
Questions ?