CSE 160 – Lecture 5
Introduction to PVM
PVM
• How PVM is structured
• Sending and receiving messages
• Getting started with the programming assignment
PVM Programming Model• Dynamic collection of serial and parallel computers
appear as single distributed memory Virtual Machine.
• Tasks can be dynamically spawned and killed by any other task. • Any PVM task can send a message to any other.• No limit to size or number of messages.
• Model supports fault tolerance, resource control, process control, heterogeneous networks and hosts.
host (one per IP address)pvmd - one PVM daemon per host
pvmd
pvmd
pvmd
How PVM is Designed
libpvm - task linked to PVM library
pvmds fully connected using UDP
task task task
Unix Domain Socketsinner host messages
OS network interface
task task task
Shared Memory
shared memory multiprocessor
P0 P1 P2
task task task
distributed memory MPP
task task tasktask task task
internal interconnect
tcpdirect connect
Step 1 – Enrolling in PVM
• PVM is a user-level library and tasks must be enrolled to be addressed.
myid = pvm_mytid();printf(“ -> %x”, myid);-> 0x40001
• First call to any PVM function will enroll your program PVM.
• Above construct is the standard practice
Task ID = TID
• PVM TIDs are very strange looking numbers.– Give host information– Node information– Type of message (unicast or multicast)– System bit for messages destined to PVMDs (as
opposed to regular tasks).
• PVM uses tid to identify pvmd, tasks, groups
• Fits into 32-bit integer
• S bit addresses pvmd, G bit forms mcast address
• Local part defined by each pvmd - eg. for PGON
Task ID (tid)
18 bits12 bitsS G host ID local part
12 bitsS G host ID process node ID
11 bits7 bits
4096 hosts 2048 nodeseach with
Popular Distributed Programming Schemes
• Master/SlaveMaster task starts all slave tasks
and coordinates their work and I/O
• SPMDSame program executes on different pieces
of the problem
• FunctionalSeveral programs are written. Each performs
a different function in the application
Sending and receiving messages
• Send a messageerr = pvm_send( dest, tag);Dest = TID of destination;Tag = user-defined distinguishing integer;
• Receive a messageErr = pvm_recv( src, tag);Src = TID of senderTag = distinguished integer
Receive Wildcards
• A receiver may wildcard the sending tid or the matching tag. –1 is the wildcard.
Any sender, specific tag
pvm_recv(-1, tag);
Specific sender, any message
pvm_recv(src, -1);
Any sender, any message
pvm_recv(-1, -1);
PVM Examples in Package
Supplied examples include:hello hello_other PVM equivalent to hello worldmaster slave Master/slave examplespmd SPMD examplegexample Group and collective ops.timing timing_slave Timing example - comm perfhitc hitc_slave Dynamic load balanceinheritb Communication contextimbi gmbi Persistent messages templatemhf_server mhf_tickle Message handlers
Examples illustrate PVM usage and serve as templatesto build your own applications.
Examples include Makefile.aimk and both C and Fortran versions.
Starting PVM
pvm [-nhostname] [hostfile]
PVM console. Starts PVM or just attaches to running PVM. Recommended way to start and stop PVM 3.
xpvm [hostfile]
PVM Graphical Interface includes console functions, and more. Starts PVM or attaches to running PVM.
pvmd [-nhostname] [hostfile]direct startup, seldom used except for debugging. Option -n useful if host has multiple network cards eg. ATM, ethernet
Three ways to start PVM
Virtual Machine Construction via the PVM Console
• When you run pvm from the console– If no existing VM for this user, start a new one
– If an existing VM, enroll in PVM and wait for commands.
• Remember! The collection of communicating PVMDs define a VM.
• The pvm console is “just another pvm program”. – Anything that can done from the console, can be done
from your program.
VM Construction Cont’d
• VMs with one physical machine are OK, but not very interesting .– Other machines must be added from the first machine you
started (eg. From whip)whip% pvmpvm> add fraphost frap added
• A hostfile can be used to list all the hosts you want in VM.whip% pvm hostfile(start pvm on whip and add all hosts to VM listed in hostfile)
Some Common Startup Problems
• PVM needs to be to remote shell to a host to add a pvmd and extend the machine– Make sure you rsh to a machine without
needing a password
• Old PVMD was running, but didn’t clean up properly (Pvmd already running? Error)– Look for /tmp/pvmd.uid and remove it
PVM Console
add hostname
alias
conf
delete hostname
halt
help [command]
kill tid
ps -a
quit
reset
sig
spawn
trace
version
pvm> Available commands and their meaning
Add host(s) to virtual machine (can list several)
define/list command alias
list hosts in the virtual machine
delete hosts from virtual machine
shut down PVM and tasks
print information about commands and options
kill a task
list all running tasks
exit console - leave PVM and tasks running
kill all tasks and reset PVM
send signal to a task
spawn tasks (many options)
set/display trace events
print PVM version
Creating a Hostfile
lo= login name for this host
so=pw ask for passwd this host
dx= location of pvmd
ep= path to user’s executables
wd= working directory to spawn tasks
sp= relative speed rating
bx= location of desired debugger
so=ms manual pvm startup this host
Simplest form is just a list of hostnames
Same as initial host
Same as initial host
$PVM_ROOT/lib/pvmd
~/pvm3/bin/$PVM_ARCH
$HOME
1000 (not used by PVM)
$PVM_ROOT/lib/debugger
use to bridge firewalls
Hostfile options Default values
Example: # Beowulf hostfile (comment lines start with #)* ep=/stuff/myapp wd=/stuff/myapphost1 host2 lo=tom so=pw
Starting tasks
• Use spawn in the pvm consolepvm> spawn -> /bin/date pvm decides where to start task
pvm> spawn -> -whip /bin/datestart on host whip
pvm> spawn -> –4 /bin/hostnamestart 4 copies
• Use the pvm_spawn() command to start tasks from your own program
PVM_ROOT
• PVM_ROOT – Where PVM looks for various files
• PVM_ARCH – architecture code for the machine– eg. LINUX, SUN4SOL2, SGI5
• If you do a “spawn -> gexample”, where does it look for the executable?– $PVM_ROOT/bin/$PVM_ARCH– $HOME/bin/$PVM_ARCHBoth of these paths are relative to the remote machine
Capturing Output
• When tasks are spawned, there output goes to /tmp/pvml.uid on the local machine?– Output shows up in many different files
• PVM answer: pvm_catchout(stdout)– Capture standard output/error from spawned (children)
tasks and copy to your standard output.
• The ‘->’ in the console spawn command tells PVM to capture output of all tasks and display
• Remote output is buffered. Data will show up when buffer is full or when remote program ends.
First Part of Homework Assignment
• Write a launcher, that uses pvm_spawn, captures output, and responds to errors by killing tasks.
• Spawning on a particular hostchar *remotehost;
int remtid;
numt = pvm_spawn(task, task_argv, PvmTaskHost, remotehost, 1, &remtid);
• If spawn fails, numt != number of tasks (1);– Errors are sent back in remtid (array)
Pseudo code - LauncherRead nprocs and hostfileRead program name, and argsRead numhosts from hostfile into a hosts arraymytid = pvm_mytid(); /* enroll */pvm_catchout(stdout);for (i = 0; i < nprocs; i++){
numt = pvm_spawn(prog, progargv, PvmTaskHost, &hosts[numhosts % i], &tids[i]);if (numt != 1)
print spawning error code;for (j = 0; j < i; j ++)
pvm_kill(tids[j]);pvm_exit();exit();
}pvm_exit();exit();
Messages
• Messages serve TWO purposes– Exchange data
– Synchronize processes
• PVM uses the concept of message buffers– Buffers may have heterogenous and typed information
in them.
– Message contents are up to the user• Unpacking/packing order are defined by your program. Types
of packed data not included in message itself.
Example Packed Message buffer
nitems Ao A1 A2 Anitems
message length
floatsint
• By default, data is packed in a universal format – XDR (External Data Representation), so that the data can be read by any architecture.
Initializing A Message Buffer
• pvm_initsend(PvmDataDefault)– Initialize the current send buffer
– The buffer itself is hidden.• This causes no end to confusion !
• Suppose you want to send a message formatted as follows:– First entry: integer K, number of strings to follow
– Next K entries: string data
Example – Packing, sending
#define TEST 101pvm_initsend(PvmDataDefault);pvm_pkint(&K, 1, 1);/* pack one integer, note & */
for (i = 0; i < K; i ++)pvm_pkstr(&strings[I]);
pvm_send(dest, TEST);Sends the packed message to dest with tag TEST;
Receiving the Message and Unpacking
#define TEST 101
pvm_recv(-1, TEST);
recv’s the packed message from anybody with tag TEST; Set the current receive buffer for unpacking.
pvm_upkint(&K, 1, 1);
/* find out how many strings in msg */
for (i = 0; i < K; i ++)
pvm_upkstr(&strings[I]);
TIDs revisited
• TIDs have a strange assignment that makes it hard to write SPMD-style programs.
• PVM uses Groups to ease the mapping of TIDs to a 0 – N (for N+1, members).– Each member of a group is called an instance– Algorithms often are based on 0 – N membership
• PVM uses user-defined group names– look at example where N is given as a command arg
• PVM send/recv use TIDs as dest/src NOT instance numbers
Groups
#define GNAME “myGroup”Read nprocs from command linemytid = pvm_mytid();myinst = pvm_joingroup(GNAME);pvm_freezegroup(GNAME, nprocs);pvm_initsend();pvm _pkxxx();pvm_send(pvm_gettid(GNAME, oinst), TAG);
pvm_recv(pvm_gettid(GNAME, ainst), TAG);pvm_upkxxx();pvm_lvgroup(GNAME);pvm_exit();
Ring Gather - Overview
3
2
1
0
send(2, tag)
1
send(0, tag)
3
send(1, tag)
2 time step
4
Print Results
Ring Gather – Pseudo Codeint ndata = 0;myinst = pvm_joingroup(GNAME);if (myinst != nprocs – 1)
pvm_recv(pvm_gettid(GNAME, myinst+1), TAG);pvm_upkint(&ndata,1,1);for (i = 1; i <= ndata; I++)
pvm_upkxxx(&Data[i][0], length, 1); fill data[0][] with local information
if (myinst != 0)ndata ++;pvm_pkint(&ndata,1, 1);for (i = 0; i <= ndata; i++)
pvm_pkxxx(&data[i][0], length, 1); pvm_send(pvm_gettid(GNAME, myinst – 1), TAG);
else /* instance 0 */print out data entries 0 to ndata
Tree Gather - Overview
3
4
5
6
1
2
0
time step
send(1,tag)
send(1,tag)
send(2,tag)
send(2,tag)1
send(0,tag)
send(0,tag)
2
Print Results
3
Tree Gather – Pseudo Codeint ndata = 0; int index = 1; myinst = pvm_joingroup(GNAME);if (2*myinst + 1 < nprocs) /* “left” subtree */
pvm_recv(pvm_gettid(GNAME, 2*myinst+1), TAG);pvm_upkint(&ndata,1,1);for (i = index; i <= ndata; i++)
pvm_upkxxx(&Data[i][0], length, 1);index = ndata + 1;
if (2*myinst + 2 < nprocs) /* “right” subtree */pvm_recv(pvm_gettid(GNAME, 2*myinst+2), TAG);pvm_upkint(&tmpdata,1,1);ndata += tmpdata;for (i = index; i <= ndata; i++)
pvm_upkxxx(&Data[i][0], length, 1);
fill data[0][] with local information
Tree Gather Pseudo Code Part 2if (myinst != 0)
ndata ++;
pvm_pkint(&ndata,1, 1);
for (i = 0; i <= ndata; i++)
pvm_pkxxx(&data[i][0], length, 1);
dst = (myinst – 1)/2;
pvm_send(pvm_gettid(GNAME, dst), TAG);
else /* instance 0 */
print out data entries 0 to ndata
Next Time
• Start on parallel algorithms and parallel programming paradigms
Top Related