Parallel Computing in the Mainstream

Sanjeev Kumar

Intel Corporation

PPoPP’08 Panel: Where Will All the Threads Come From?

Enabing Parallel Computing in the Mainstream 2Sanjeev Kumar

Our Work

RMS Applications

Architectural Enhancements

Language & CompilerImprovements

1

2

Can we use chips with a large number of cores? Are there applications with large computational requirements? Do these applications scale? What does it take? How can the parallelization process be simplified?

Enabing Parallel Computing in the Mainstream 3Sanjeev Kumar

RMS Applications

A taxonomy of a large class of emerging applications

Enabing Parallel Computing in the Mainstream 4Sanjeev Kumar Image Courtesy: Siggraph 2004 paper by Dreamworks

Direct Illumination only

Adding Indirect Illumination

Ray Tracing Ray Tracing Based Rendering Rendering

Movie Movie Physics Game Physics Physics

Financial Analytics

Enabing Parallel Computing in the Mainstream 5Sanjeev Kumar

VisionVision

Video/Data MiningVideo/Data Mining

Image ProcessingImage Processing

Virtual Worlds (Second Life)

Enabing Parallel Computing in the Mainstream 6Sanjeev Kumar

Public Benchmarks PARSEC Benchmark Suite

Joint effort between Princeton and Intel Available at http://parsec.cs.princeton.edu/ Includes some RMS applications

Tutorial on PARSEC at ISCA’08

Enabing Parallel Computing in the Mainstream 7Sanjeev Kumar

What have we learned? Are there applications with large computational requirements?

Do these applications scale?

What does it take to scale these applications?

How can the parallelization process be simplified?

Enabing Parallel Computing in the Mainstream 8Sanjeev Kumar

Large Computation Requirements?G

FL

OP

S

Ray

T

raci

ng

Flu

id D

ynam

ics

Fo

reg

rou

nd

Est

imat

ion

Ass

et L

iab

ilit

y M

anag

emen

t

Yes

Enabing Parallel Computing in the Mainstream 9Sanjeev Kumar

Do these applications scale?

0 8 16 24 32 40 48 56 64

Raytracing

Fluid Dynamics

Face Simulation

Cloth Simulation

Rigid Body Simulation

Body Tracker

Foreground Estimation

Text Indexing

Portfolio Management

Yes

Enabing Parallel Computing in the Mainstream 10Sanjeev Kumar

What does it take to scale applications? Lots of programming effort

Large number of cases: this suffices Plus algorithmic changes

Usually involves one time performance hit Plus architectural enhancements

Support for fine-grained concurrency [ ISCA’07 ] Support for atomic vector operations [ ISCA’08 ] Increased bandwidth, bigger caches (Stacked DRAM) Support for efficient data movement …

Enabing Parallel Computing in the Mainstream 11Sanjeev Kumar

How can parallelization be simplified? No silver bullet in sight

Problem harder than Scientific Computing Every approach has its limitations

Better languages. Maybe even domain specific Parallelized & optimized libraries Use compilers to automate optimizations

Big performance gains with “Close to metal” programming

Tools for correctness checking Tools for understanding performance

Enabing Parallel Computing in the Mainstream 12Sanjeev Kumar

FIMIPDE NLP

Level Set

Computer Vision

Physical Simulation

(Financial) Analytics Data Mining

ParticleFiltering

SVMClassification

SVMTraining

IPM(LP, QP)

Fast MarchingMethod

K-Means

TextIndexerMonte Carlo

BodyTracking

FaceDetection CFD

Face,Cloth

RigidBody

PortfolioMgmt

OptionPricing

Cluster/Classify

TextIndex

Basic matrix primitives(dense/sparse, structured/unstructured)

Basic Iterative Solver(Jacobi, GS, SOR)

Direct Solver(Cholesky)

Krylov Iterative Solvers(PCG)

Rendering

Global Illumination

Collisiondetection

LCP

MediaSynth

Machinelearning

Filter/transform

Basic geometry primitives (partitioning structures, primitive tests)

Non-ConvexMethod

Enabing Parallel Computing in the Mainstream 13Sanjeev Kumar

Conclusion A large number of applications have

High computational requirements Lots of parallelism

Challenge: Simplify parallelization