FluidSimParallel fluid particle simulation and visualization

Paul Bienkowski2bienkow@informatik.uni-hamburg.de

by

UNIVERSITÄT HAMBURGPRAKTIKUM „PARALLELE PROGRAMMIERUNG”

September 24, 2014

Contents

2/25

1 – Introduction1.1 Motivation1.2 Particle Model1.3 Force Model

2 – The Simulator2.1 Parallelization2.2 Synchronization2.3 Data output

3 – The Visualizer3.1 Basic Technology3.2 Data Transfer

4 – Results4.1 Did it work?4.2 Live Demonstration4.3 Performance4.4 Difficulties4.5 Problems

5 – Conclusion

Introduction 3/25

Introduction

Motivation

Introduction 4/25

Motivation

Introduction 4/25

Motivation

Introduction 4/25

Motivation

Introduction 4/25

IdeaSimulate 2D-particles that repel each other

Motivation

Introduction 4/25

IdeaSimulate 2D-particles that repel each other

GoalSimulate an airplane wing andmeasure lift

Motivation

Introduction 4/25

IdeaSimulate 2D-particles that repel each other

GoalSimulate an airplane wing andmeasure lift

TechnologyC++11, MPI, OpenMP, SFML

ParticleModel

Introduction 5/25

Each particle has 3 basic properties:

1. position2. velocity3. force

Only position and velocity need to be stored across iterations, forceis recomputed every iteration.

ParticleModel

Introduction 5/25

Each particle has 3 basic properties:

1. position2. velocity3. force

Only position and velocity need to be stored across iterations, forceis recomputed every iteration.

ParticleModel

Introduction 6/25

pi pj

Any two particles repel each other:

forcei :=∑j

force(∣∣pi − pj

∣∣) · norm(pi − pj)

The force on a particle affects its velocity:

velocityi := velocityi + forcei · dt

The velocity of a particle affects its position:

positioni := positioni + velocityi · dt

ForceModel

Introduction 7/25

force(x) =

{F ·

(1− x−T

D

)P for 0 ≤ x ≤ D+ T0 otherwise

wherex is the distance between the particles.F is the force strength factorD is the influence distanceT is the distance threshold (particle radius)P is the force power

ForceModel

Introduction 7/25

force(x) =

{F ·

(1− x−T

D

)P for 0 ≤ x ≤ D+ T0 otherwise

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

distance

force

P=0.2P=1P=2P=3P=20 Changing the

Force Power (P)F = 1D = 1T = 0

ForceModel

Introduction 7/25

force(x) =

{F ·

(1− x−T

D

)P for 0 ≤ x ≤ D+ T0 otherwise

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

distance

force

T=0.0T=0.1T=0.2T=0.3 Changing the

DistanceThreshold (T)

F = 1D = 1

P = 10

ForceModel

Introduction 7/25

force(x) =

{F ·

(1− x−T

D

)P for 0 ≤ x ≤ D+ T0 otherwise

I found these values work for the wing simulation:

D = 0.001T = 0.06P = 1F = 20

Meshes

Introduction 8/25

Amesh is a simple polygon, each segment (line) is checked forcollision with every particle.

Simple linear algebra calculations are made for reflecting particlesoffmesh segments.

Mesh

Particle

This creates a force on the mesh.

Meshes

Introduction 8/25

Amesh is a simple polygon, each segment (line) is checked forcollision with every particle.

Simple linear algebra calculations are made for reflecting particlesoffmesh segments.

Mesh

Particle

This creates a force on the mesh.

Meshes

Introduction 8/25

Amesh is a simple polygon, each segment (line) is checked forcollision with every particle.

Simple linear algebra calculations are made for reflecting particlesoffmesh segments.

Mesh

Particle

This creates a force on the mesh.

The Simulator 9/25

TheSimulator

Parallelization

The Simulator 10/25

each particle has to be updated (simple for loop)→ threading trivial with OpenMPparticles can be distributed across multiple processes→ Domain Grid

P0 P1 P2 P3

P4 P5 P6 P7

P8 P9 P10 P11

Parallelization

The Simulator 10/25

each particle has to be updated (simple for loop)→ threading trivial with OpenMPparticles can be distributed across multiple processes→ Domain Grid

P0 P1 P2 P3

P4 P5 P6 P7

P8 P9 P10 P11

Synchronization

The Simulator 11/25

P0 P1 P2 P3

P4 P5 P6 P7

P8 P9 P10 P11

(a) Checkerboard

P0 P1 P2 P3

P4 P5 P6 P7

P8 P9 P10 P11

(b) StripesModes of Domain coloring

Mode Sending Receiving Directions1. Checkerboard black white N, E, S, W2. Checkerboard white black N, E, S, W3. Stripes black white NE, SE, SW, NW4. Stripes white black NE, SE, SW, NW

Data output

The Simulator 12/25

0000000 0020 0000 0001 0000 0004 0000 2710 00000000010 1387 0000 0000 0000 1389 0000 0000 00000000020 8a72 d187 c4bb 3fa1 73d1 4e75 d95f 3fbe0000030 d3f4 2c9a cf42 3fbc b890 9e75 5a61 3fc1...00013a7 9aa6 af8e c353 3fc7 1df2 1539 3ec1 3fc300013b7 c065 7825 f853 3fb3 717a c31f 1f55 3fc3...

Header length: 32 bytesIteration number: 1Number of processes: 4Particle count: 10000Particle count by process: 4999 / 0 / 5001 / 0Particle positions of P1 (x1, y1, x2, y2, ...)Particle velocities of P1

The Visualizer 13/25

TheVisualizer

Basic Technology

The Visualizer 14/25

SFML for window, input, rendering (OpenGL inside)Load Iterations into memoryPlay/Pause/LiveDifferent display modes (coloring of particles)

Data Transfer

The Visualizer 15/25

How is data transfered from Simulator to Visualizer?

Socket/Network/MPI?→ too complicatedSSHFS ,status file contains metadata

number of iterationsgrid size

visualizer reads status in regular intervals

Data Transfer

The Visualizer 15/25

How is data transfered from Simulator to Visualizer?Socket/Network/MPI?

→ too complicatedSSHFS ,status file contains metadata

number of iterationsgrid size

visualizer reads status in regular intervals

Data Transfer

The Visualizer 15/25

How is data transfered from Simulator to Visualizer?Socket/Network/MPI?→ too complicated

SSHFS ,status file contains metadata

number of iterationsgrid size

visualizer reads status in regular intervals

Data Transfer

The Visualizer 15/25

How is data transfered from Simulator to Visualizer?Socket/Network/MPI?→ too complicatedSSHFS ,

status file contains metadatanumber of iterationsgrid size

visualizer reads status in regular intervals

Data Transfer

The Visualizer 15/25

How is data transfered from Simulator to Visualizer?Socket/Network/MPI?→ too complicatedSSHFS ,status file contains metadata

number of iterationsgrid size

visualizer reads status in regular intervals

Results 16/25

Results

Did it work?

Results 17/25

Did it work?

Results 17/25

Live Demonstration

Results 18/25

Performance

Results 19/25

All measurements were made with IO disabled, no particledata was recorded.

Performance

Results 19/25

All measurements were made with IO disabled, no particledata was recorded.

Performance

Results 19/25

All measurements were made with IO disabled, no particledata was recorded.

Difficulties

Results 20/25

MPI file input/output was hard to get working,miscalculated seek offsets etc.

lots of segmentation faults and uninitialized values ,Cannot insert: Count 0xdeafbeed >= Size 0xdeafbeed.fluidsim: Quickset.hpp:14: Assertion ‘count < size’ failed.Aborted.

Segmentation fault.(gdb) frame 3(gdb) print buf$1 = 0xdeafbeeddeafbeed;

2D collisions are not that trivialthe model (uplift) did not work out until I implementedsurface damping

Difficulties

Results 20/25

MPI file input/output was hard to get working,miscalculated seek offsets etc.lots of segmentation faults and uninitialized values ,Cannot insert: Count 0xdeafbeed >= Size 0xdeafbeed.fluidsim: Quickset.hpp:14: Assertion ‘count < size’ failed.Aborted.

Segmentation fault.(gdb) frame 3(gdb) print buf$1 = 0xdeafbeeddeafbeed;

2D collisions are not that trivialthe model (uplift) did not work out until I implementedsurface damping

Difficulties

Results 20/25

MPI file input/output was hard to get working,miscalculated seek offsets etc.lots of segmentation faults and uninitialized values ,Cannot insert: Count 0xdeafbeed >= Size 0xdeafbeed.fluidsim: Quickset.hpp:14: Assertion ‘count < size’ failed.Aborted.

Segmentation fault.(gdb) frame 3(gdb) print buf$1 = 0xdeafbeeddeafbeed;

2D collisions are not that trivial

the model (uplift) did not work out until I implementedsurface damping

Difficulties

Results 20/25

MPI file input/output was hard to get working,miscalculated seek offsets etc.lots of segmentation faults and uninitialized values ,Cannot insert: Count 0xdeafbeed >= Size 0xdeafbeed.fluidsim: Quickset.hpp:14: Assertion ‘count < size’ failed.Aborted.

Segmentation fault.(gdb) frame 3(gdb) print buf$1 = 0xdeafbeeddeafbeed;

2D collisions are not that trivialthe model (uplift) did not work out until I implementedsurface damping

Problems

Results 21/25

software not optimized for RAM usage→ can’t run more than 6 processes locally /

memcpy is slow16 send/receive operations on a 12-node cluster,probably room for improvement,

however this methodscales to every cluster size

Problems

Results 21/25

software not optimized for RAM usage→ can’t run more than 6 processes locally /memcpy is slow

16 send/receive operations on a 12-node cluster,probably room for improvement,

however this methodscales to every cluster size

Problems

Results 21/25

software not optimized for RAM usage→ can’t run more than 6 processes locally /memcpy is slow16 send/receive operations on a 12-node cluster,probably room for improvement,

however this methodscales to every cluster size

Problems

Results 21/25

software not optimized for RAM usage→ can’t run more than 6 processes locally /memcpy is slow16 send/receive operations on a 12-node cluster,probably room for improvement,

however this methodscales to every cluster size

Problems

Results 21/25

software not optimized for RAM usage→ can’t run more than 6 processes locally /memcpy is slow16 send/receive operations on a 12-node cluster,probably room for improvement, however this methodscales to every cluster size

Conclusion 22/25

Conclusion

Conclusion

Conclusion 23/25

Yes, it works!Even in real-time!Even thoughO(n2 + nm)with n ∈ O(10000)

It looks kind of fancy...I learned a lot.

Furtherworks

Conclusion 24/25

Load Balancer, based on number of particles inrows/columns

SIMD“Ball” particle model (elastic collision of circle shapes)different World Scenarios / presets (gravity, water, ...)animation export

Furtherworks

Conclusion 24/25

Load Balancer, based on number of particles inrows/columnsSIMD

“Ball” particle model (elastic collision of circle shapes)different World Scenarios / presets (gravity, water, ...)animation export

Furtherworks

Conclusion 24/25

Load Balancer, based on number of particles inrows/columnsSIMD“Ball” particle model (elastic collision of circle shapes)

different World Scenarios / presets (gravity, water, ...)animation export

Furtherworks

Conclusion 24/25

Load Balancer, based on number of particles inrows/columnsSIMD“Ball” particle model (elastic collision of circle shapes)different World Scenarios / presets (gravity, water, ...)

animation export

Furtherworks

Conclusion 24/25

Load Balancer, based on number of particles inrows/columnsSIMD“Ball” particle model (elastic collision of circle shapes)different World Scenarios / presets (gravity, water, ...)animation export

Conclusion 25/25

Thank you for your attention!

Questions?