Shared Memory Considerations

INTEL CONFIDENTIAL

Shared Memory ConsiderationsIntroduction to Parallel Programming – Part 4

Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.

2

Review & Objectives

Previously:Described method to identify opportunities for parallelism in

code segments and applications

At the end of this part you should be able to: Describe the shared-memory model of parallel

programmingDemonstrate how to implement domain and

task decompositions using threadsDecide whether a variable in a multithreaded

program should be shared or private


Cooperating Parallel Processes

Parallel computing multiple processes working together to speed the solution of a task

Working together process cooperationKinds of cooperation

Sharing information (communication)Keeping out of each other’s way

(synchronization)

3


The Shared-Memory Model

4

Shared Memory

Core

PrivateMemory

Core

PrivateMemory

Core

PrivateMemory


Evaluating Parallel Models

How do processes share information?How do processes synchronize?In shared-memory model, both accomplished

through shared variablesCommunication: bufferSynchronization: semaphore

5


Methodology

Study problem, sequential program, or code segmentLook for opportunities for parallelismUse threads to express parallelism

6


What Is a Process?

A program in some state of executionCodeDataLogical address space

Information about a process includesProcess stateProgram counterValues of core’s registersMemory management information

7


What Is a Thread?

“A unit of control within a process” — Carver and Tai

Main thread executes program’s “main” functionMain thread may create other threads to execute

other functionsThreads have own program counter, copy of core

registers, and stack of activation recordsThreads share process’s data, code, address space,

and other resourcesThreads have lower overhead than processes

8


9

Utility of Threads

Threads are flexible enough to implementDomain decompositionTask decompositionPipelining


Domain Decomposition Using Threads

10

SharedMemory

Thread 0 Thread 2

Thread 1

f ( )

f ( )

f ( )


Task Decomposition Using Threads

11

SharedMemory

Thread 0 Thread 1

e ( )

g ( )h ( )

f ( )


Pipelining Using Threads

12

Thread 0 Thread 2Thread 1

Shared MemoryInput Output

e ( ) f ( ) g ( )Dataset 2

Data sets5, 6, ...

Dataset 4 Data

set 3 Dataset 1


Shared versus Private Variables

13

SharedVariables

PrivateVariables

PrivateVariables

Thread

Thread


Domain Decomposition

Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);

14



Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);Thread 0:for (i = 0; i < 500; i++) a[i] = foo(i);Thread 1:for (i = 500; i < 1000; i++) a[i] = foo(i);

15



Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);Thread 0:for (i = 0; i < 500; i++) a[i] = foo(i);Thread 1:for (i = 500; i < 1000; i++) a[i] = foo(i);

16

SharedPrivate


Task Decomposition

int e;

main () { int x[10], j, k, m; j = f(x, k); m = g(x, k); ...}

int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}

int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}

17


Task Decomposition

int e;

main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}



18

Thread 0

Thread 1


Task Decomposition

Volatile int e;




19

Thread 0

Thread 1

Static variable: Shared


Task Decomposition

Volatile int e;




20

Thread 0

Thread 1

Heap variable: Shared


Task Decomposition

Volatile int e;




21

Thread 0

Thread 1

Function’s local variables: Private


Shared and Private Variables

Shared variablesStatic variablesHeap variablesContents of run-time stack at time of call

Private variablesLoop index variablesRun-time stack of functions invoked by thread

22


The Shared-Memory Model (Reprise)

23

Shared Memory

Core

PrivateMemory

Core

PrivateMemory

Core

PrivateMemory


The Threads Model

24

Shared Variables

Thread

PrivateVariables

Thread

PrivateVariables

Thread

PrivateVariables


References

Jim Beveridge and Robert Wiener, Multithreading Applications in Win32®, Addison-Wesley (1997).

David R. Butenhof, Programming with POSIX® Threads, Addison-Wesley, (1997).

Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).

Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).

25

Shared Memory Considerations

Documents

Transcript of Shared Memory Considerations