1

CSCI 633: Advanced Operating Systems

Dept. of Computer ScienceCSU San Marcos

Fall 2003

Kayhan Erciyes

2

The Plan: Applied Stuff

Introduction to distributed systems– Overview, definitions, characteristics, issues,

challenges

A practical developer’s overview of networking– Characteristics of IP, TCP, UDP– Writing networked applications: UDP vs. TCP vs.

higher level approaches

Then on to more theoretical aspects of distributed computing

3

The Plan: Theoretical Stuff

Theoretical Foundations– Fundamental Limitations– Causality– Logical clocks (logical, vector, matrix clocks)– Global states

Algorithms for distributed mutual exclusion Distributed Shared Memory (DSM) Topics in fault tolerance and reliability

4

Distributed Systems: Intro

Distributed System: – Autonomous Computers + Network– Communication via message-passing– No shared memory– No global clock– Range:

• Two PC’s connected by $25 worth of networking hardware

• Beowulf clusters: racks (or stacks) of PCs connected by high-speed networking

• Millions of computers, connected by diverse networking technologies ranging from modems to gigabit connections (the Internet)

mobilecomputing

distributedcomputing

5

Network Operating Systems

Network operating systems extend sequential operating systems to provide:– Resource sharing (files, devices, …)

– Interoperability (email, remote command execution, remote login…)

User is generally aware of machine boundaries (do “this” on “that” machine)

6

DOS vs. NOS

(Virtual) Transparency: The ability to see what you want to see, and not see what you consider to be of no interest

An exaggeration:

DOS’s allow you to “slide” toward a “one large machine” view of the network of computers

dOSDistributeNetworkOScytransparen

lim

7

Unix/”Distributed Unix”

Unix: pervasive when cheap network technologies became available (1970’s), so a logical choice for building “distributed systems”

Extensions to Unix which provided interprocess communication were a principal building block of early distributed systems

Unix sockets API

8

Distributed Unix, Cont.

Problems with distributed Unix, though: – Monolithic kernels– Scattered process information– Progress migration, checkpointing difficult

These problems stem from taking a tool in wide use and “molding” it to fit a new need

Why not “kill” Unix and use a modern, “this is what I do well” distributed OS?

Commercial pressures. Too much code already in use.

9

Desirable Characteristics

These are the “selling points” for distributed systems, answers to the question “What can they provide for me?”

Scalability– Want to be able to pile on more hardware as

needed to tackle bigger problems without rewriting applications

– E.g., in Parallel Virtual Machine (PVM), add more processors to share workload of smaller pieces of a large computation

10

Desirable: Fault Tolerance

Fault Tolerance– Higher availability and more resilience to faults

than uniprocessor/shared memory multiprocessor solutions

– Redundancy is the key—it’s present in all fault tolerance schemes, e.g.,

• replicated servers (e.g., for file or database storage)

• snapshots of application state for recovery

11

Desirable: Transparency

(Virtual) Transparency– Want distributed system to appear to be one big,

seamless machine, however...– ‘A distributed system is a system on which I

cannot get any work done because a machine I’ve never heard of is down.’

-- L. Lamport– Fault tolerance/transparency must be considered

together– Don’t want “transparent” components failing and

preventing work from getting done

12

Desirable: Concurrency

Concurrency– Distributed systems can bring a lot of hardware to

bear on difficult or time-consuming applications– MIMD situations (e.g. file server, compute server,

web server machines)• (Multiple Instruction, Multiple Data—means distributed

software components are different)

– SIMD situations (e.g. parallel rendering applications for computer graphics)

• (Single Instruction, Single Data—means lots of instances of a software component that performs a specific operation)

13

Desirable: Resource Sharing

Resource Sharing– Resource: display, printer, disk, CD-ROM,

applications– “One Cadillac instead of 12 Yugos”– Distributed systems allow resources to be shared

freely (or not), regardless of their location in the system

– Can drastically reduce cost, improve utilization of resources, reduce administration nightmare

– Security issues must be considered; security issues arise in distributed systems which don’t exist in isolated systems

14

Desirable: “Openness”

“Openness”– Heterogeneous hardware and software can be used to

build systems and solve complicated problems– Published protocols and interfaces make putting

together the diverse pieces possible• Which protocols are spoken?• What data formats are used?• Where are you?

– Example: WWW. Diverse machines “speak” a standard protocol: HTTP. “Open” extensions include CGI (Common Gateway Interface)

– Example: Universal Plug and Play (UPnP), Service Location Protocol (SLP) for building highly dynamic client/server systems

15

Advantages in Brief

The potential for building large, scalable, fault-tolerant “computers” with huge resources from commodity machines

Commodity “supercomputers” In many circumstances, individual machines

can still be used for traditional tasks– E.g., no reason individual users couldn’t read mail

on one node of the Beowulf cluster…

Web-based supercomputing

16

A Few of the Challenges

No shared memory =>– an unfamiliar programming model– application state is spread around– existing algorithms may be inappropriate– object-based distributed computing helps

No perfectly synchronized time source =>– difficult to order events– difficult to say “do something NOW” to the entire

system We’re stuck with the speed of light (?) More complicated failure modes than single

machines! Much easier for things to be “half broken”

17“Failure” has Many Meanings

Halting failure: component simply stops Fail-stop: halting failures with ability to detect

failures Omission failure: failure to send/recv message Network failure: network link breaks Network partition: network fragments into two or

more disjoint subnetworks Timing failure: action early/late; clock fails, etc. Byzantine failure: arbitrary “malicious” behavior

– This one models random, worst-case behavior

19Topic Switch: Networking Basics

Network programmingGoal: be able to implement

networked/distributed software rather than just talk about it– solve real problems– design client/server protocols– evaluate proposed solutions experimentally

20

Network Protocols

Protocol: Set of rules and data formats which make communication possible

A “language” for communicationProtocols are typically constructed

using layers, with more abstract services provided by higher-level layers

Bottom layer(s) are the actual network hardware

21Networking Performance Parameters Latency - time to transfer “empty” message Bandwidth or data transfer rate - how many

bits/sec can be transferred (how thick the “pipe” is)

message_transfer_time = latency + msg_length / data_transfer_rate

Consider: a modem connection vs. a van of magnetic tapes traveling an interstate highway

QoS: Quality of Service (bandwidth/latency guarantees for particular connections)

22

OSI Protocol Stack

OSI - Open Systems Interconnect Application - application interfaces (httpd, ftp) Presentation - network representation for data Session - connections, encryption Transport - message packets Network - network-specific packets, routing Data Link - transmission of packets between

“directly” connected machines + error issues Physical - hardware (“I can touch it”)

23Communication Through Layers

Presentation

Session

Transport

Network

Data Link

Physical

Application

Presentation

Session

Transport

Network

Data Link

Physical

Application

24

TCP/IP Protocol Stack

ISO stack is good as a model for understanding networks Layers in “real” network stacks aren’t so differentiated TCP/IP stack has won primarily because of the free

implementation shipped in early versions of BSD Unix Addresses above IP are (port, address) combinations

Physical

IP

TCPUDP

ApplicationApplication

Transport

Network

25

Transport Protocols

UDP (User Datagram Protocol)– Connectionless– Fast setup– Easy one-to-many communication– Datagram-oriented (fixed size chunks of data)– Packet reordering– Packet loss (no flow control, bad packets dropped)– Packet duplication– (Absolute) maximum datagram length: 64K– Usable maximum is more complicated– 8K is generally safe for modern systems

26

Transport Protocols, Cont. TCP (Transmission Control Protocol)

– Connection-oriented– Byte stream-oriented– Slower setup– Consumes file handles: one per connection– Flow control, automatic retransmission

• No packet reordering (delivery is FIFO)• No packet loss• No duplication

– Theoretically “no” limit on size of objects that can be dumped into a TCP stream

– In practice, limits exist

27Unix Sockets: TCP and UDP from a Programming Perspective First the standard Unix system calls for C, then from a

Java perspective Unix C Server:

– int socket(PF_INET | PF_UNIX,SOCK_STREAM |

SOCK_DGRAM,…)

– int bind(socket, localaddr …)– int listen(socket, queuelength)– int accept(socket, remoteaddr)– select( … ) allows a set of sockets to be

checked to determine if input is available– Allows service of multiple clients without

multithreading

28

Unix Sockets, Cont.

Unix C Client:– int socket(PF_INET | PF_UNIX,

SOCK_STREAM | SOCK_DGRAM,…)

– int connect(socket, remoteaddr) Unlike the server, the client typically doesn’t care

which port; the system selects one Then data is transmitted and received (for both

client and server) with:– write(socket, message, len, …)– read(socket, buffer, len, …)

29

void ServeEchoClients(int port) { int i, found; int alive; // client still around after read? int sock; // socket for listening int newconn; // socket for new client int highest; // highest handle in use; needed for select() int ready; // number of ready sockets (from select() call) int connected[100]; // handle only 100 simultaneous clients. fd_set socks; // sockets ready for reading, for select() call struct sockaddr_in server_address; // structure for bind() call int reuse=1; // avoid port in use problems  // initialize sockets stuff sock = socket(AF_INET, SOCK_STREAM, 0); if (sock < 0) { Shutdown("echo_server: socket() call failed. Can't continue."); } setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)); memset((char *) &server_address, 0, sizeof(server_address)); server_address.sin_family = AF_INET; server_address.sin_addr.s_addr = htonl(INADDR_ANY); server_address.sin_port = htons(port); if (bind(sock, (struct sockaddr *)&server_address,

sizeof(server_address)) < 0 ) { Shutdown("echo_server: bind() call failed. Can't continue."); }

SERVER

30 listen(sock, 15); // 15 is queue length for incoming connections highest = sock; memset((char *) &connected, 0, sizeof(connected));  printf("echo_server: Listening...\n");  while (1) { FD_ZERO(&socks); // initialize set of sockets to monitor FD_SET(sock,&socks); // always care about listening socket // also care about sockets for connected clients for (i=0; i < 100; i++) { if (connected[i] != 0) {

FD_SET(connected[i],&socks);if (connected[i] > highest) { highest = connected[i];}

} } ready = select(highest+1, &socks, NULL, NULL, NULL); if (ready < 0) { Shutdown("echo_server: select() call failed. Can't continue."); } 

31

// see who's knocking at our (socket) door...  if (FD_ISSET(sock,&socks)) { // new client newconn = accept(sock, NULL, NULL); if (newconn < 0) {

printf("** FAILED TO CONNECT TO NEW CLIENT **\n"); } else {

// find a home for new client socket found=0;for (i=0; i < 100 && ! found; i++) { if (connected[i] == 0) { printf("echo_server: Connected to new client.\n"); connected[i] = newconn; found=1; }}if (! found) { printf("echo_server: OVERLOADED.\n"); close(newconn);}

} } 

32

// check connected clients, deal with one line for each ready client for (i=0; i < 100; i++) { if (FD_ISSET(connected[i],&socks)) {

alive = ReadAndEcho(connected[i]);if (! alive) { close(connected[i]); connected[i] = 0; // client hung up}

} } }}   

33

int ReadAndEcho(int handle) { char c=-1; int count=1; int ret=1;  printf("echo_server: Reading, hoping for \\n...\n"); count = read(handle, &c, 1); // read one char while (c != '\n' && count > 0) { count = write(handle, &c, 1); // echo it if (count) { putchar(c); count=read(handle, &c, 1); // read one char } }  if (count == 0) { printf("echo_server: Client hung up.\n"); ret=0; } else { // echo final \n count = write(handle, &c, 1); putchar('\n'); }  printf("echo_server: Returning to listening state.\n"); return ret;

34void EchoClient(char *ip, int port) {  struct sockaddr_in them; // address of server int sock; // socket for communication w/ server int err; int len; char buf[512]; char c; int count; struct hostent *remip; // will use this one... unsigned long remip2; // or this one as the binary remote addr bzero((char *)&them, sizeof(them)); them.sin_family = AF_INET; them.sin_port = htons(port); // hton*() convert integer byte order

// try inet_addr() call first; some unixes freak if we provide a // dotted numeric IP address to gethostbyname() remip2=inet_addr(ip); if (remip2 <= 0) { remip=gethostbyname(ip); if (remip == NULL) { herror(NULL); Shutdown("Couldn't initialize connection parameters."); } }

CLIENT

35 if (remip2 <= 0) { memcpy(&(them.sin_addr.s_addr), remip->h_addr, remip->h_length); } else { them.sin_addr.s_addr = remip2; } if ((sock=socket(AF_INET, SOCK_STREAM, 0)) < 0) { printf("echo_client: socket() failed with error %d.\n", sock); Shutdown("Can't continue."); } if ((err = connect(sock, (struct sockaddr*)&them,

sizeof(struct sockaddr_in)))) { printf("echo_client: connect() failed with error %d.\n", err); Shutdown("Can't continue."); }

36 printf("echo_client: \".\" on a line by itself disconnects.\n"); gets(buf); while (buf[0] != '.') { len=strlen(buf); buf[len++]='\n'; // add newline buf[len]=0; write(sock, buf, strlen(buf)); // transmit  // get response one char at a time printf("echo_client: Service response:\""); count = read(sock, &c, 1); while (c != '\n' && count > 0) { putchar(c); count=read(sock, &c, 1); // read one char } printf("\"\n"); if (count == 0) { printf("echo_client: Server hung up. How rude!\n"); buf[0]='.'; } gets(buf); } close(sock);} // end of EchoClient

37

Java IPC

TCP and UDP socket protocols have separate interfaces in Java

More abstract than standard Unix interface, but interoperable and almost as powerful

Far more portable Simple Java TCP Client Simple Java TCP Server Simple Java UDP datagram send/receive

38

Simple TCP Client in Java

try { s=new Socket(servhostname, port); out=new DataOutputStream(s.getOutputStream()); in=new DataInputStream(s.getInputStream());}catch (Exception e) { /* error */}// do standard I/O operations on ‘in’ and ‘out’

39

Simple TCP Server in Java

try { servsock=new ServerSocket(listenport);}catch (Exception e) { /* error */ }...while ( … ) { try { cl=servsock.accept() out=new DataOutputStream(cl.getOutputStream()); in=new DataInputStream(cl.getInputStream()); // do standard I/O operations on ‘in’ and ‘out’ … cl.close(); } catch (Exception e) { /* error for this client connection */ }}

40

Limitations

Simple client/server are single threadedAffects server most, since it can only

service only client at a timeOther clients are blocked while server is

busy“Bad” client can tie up server foreverJava does NOT support select() for

sockets

41Simple UDP Client/Server in Java

byte[] buf = new byte[MAXDGRAMSIZE]; sock=new DatagramSocket(myport);while (...) { try { // incoming datagram DatagramPacket ingram = new DatagramPacket(buf, buf.length); sock.receive(ingram); … // outgoing datagram DatagramPacket outgram = new DatagramPacket(buf, buf.length,

theiraddr, theirport); sock.send(outgram); } catch (UnknownHostException uhe) { … } catch (IOException ioe) { }}

42Multithreading for the TCP Serverpublic class MTServer { public static void main(String[] args) { int port=2345; final int MaxClients = 35; ... ServerSocket serversocket = null; try { serversocket = new ServerSocket(port,

MaxClients); while (true) { Socket sock = serversocket.accept(); ServerThread thr = new ServerThread(sock ...); thr.start(); } } catch (Exception e) { /* server must die */ } }

43Multithreading, Cont.

public class ServerThread extends Thread { private Socket sock; private DataInputStream in=null; private DataOutputStream out=null; public ServerThread(Socket sock, ... ) { this.sock = sock; ... try { in=new DataInputStream(this.sock. getInputStream()); out=new DataOutputStream(this.sock. getOutputStream()); } catch (Exception e) { /* oops */ }}

44Multithreading, Cont.

public void run() { boolean bye=false; try {

while (!bye) {String command = in.readUTF();if (command.equals(“BYE”) {

bye = true; …}else if (…) {…

}}catch (Exception e) {

/* death for this client connection */}finally {

/* cleanup for this client */ out.close(); in.close();

}}

45

Robust TCP Want to detect broken connections, avoid client

and/or server “hangs” Timeouts (simple) Unix “keepalive” timers (SO_KEEPALIVE)

– Evil!– Default is generally several hours!– Timeout period generally global, hard to configure

“Connection exercises” Heartbeats

ALIVE!

ALIVE!

ALIVE!Computer

Computer

Computer

Computer

46Java, TCP, and Robust Client/Server

The following read can hang indefinitely if the server dies

try { s = input.readUTF(); // ...}catch (Exception e) { System.out.println(”Broken connection”);

}

47TCP: Maintaining Control with Timeoutstry { gotstr = false; while (! gotstr) {

try { sock.setSoTimeout(90000); // 90s timeout s = input.readUTF(); gotstr=true;

} catch (InterruptedIOException ie) { // at least I’m not stuck!

// can do other processing here }

}}catch (Exception e) { System.out.println(”Broken connection”);}

48

Server Side “Probing”

// Always detects broken connectiontry { gotstr = false; while (! gotsr) { try {

sock.setSoTimeout(90000); s = input.readUTF(); gotstr=true; } catch (InterruptedIOException ie) { // timeout...tempt fate by writing System.out.println(”Connection check”);

// client must ignore output.writeUTF("$!$"); System.out.println("Connection OK"); }

} }catch (Exception e) { System.out.println(”Broken connection”);}

49

Client Side “Probing”

// Clients wants to to read a string...// Assumes that a response is expected w/in 30s

boolean significant=false;while (! significant) { socket.setSoTimeout(30000); // 30s timeout try {

s = input.readUTF(); significant = (! s.equals("$!$"));

} catch (InterruptedIOException ie) {

// assume connection is broken throw new IOException("Server is down?");

}}

50

Want more?

See my “www.cs.uno.edu/~golden/teach.html” examples

For a good FAQ on sockets programming:

http://www.developerweb.net/sock-faq/W. Stevens (deceased) books are

classics for network programming (search for his name)

51

Higher-level Communication

MOM (Message-oriented Middleware)Message-passing libraries

– PVM– MPI

Spaces– Linda

Object-based approaches– RMI– CORBA

52Message Oriented MiddlewareWeakens link between the client and

server…Client sees an asynchronous interfaceRequest is sent independent of replyReply must be dequeued from a reply

queue laterClient and server do not need to be

running at the same time.

53

MOM: Guts

MOM system implements a queue between clients and servers

Each sends to other by enqueuing messages on one or more queues

Queues can have names, for “subject” of the queue

Simple API

54

MOM: Guts, Cont.

client

MO

M A

PI

MOMqueue

server

request

MOM APIreply

55

Other MOM Issues?

Administrative overhead– Management of the queues– Replication– Load Balancing

Handling runaway applications that flood queue with requests or fail to collect responses

Cleanup after crashesPerformance

56PVM/MPI: Message Passing Libraries

Libraries which provide– dynamic process creation– message passing primitives– synchronization primitives

Automatic assignment of new processes to machines (currently round robin for PVM?)

Primitive marshalling (“packing”) facilities, for basic C/Fortran types only

User-installable resource managers for load balancing, checkpointing (NO!!!!!!), etc.

57

PVM Schematic pvmd

pvmd

pvmd

“master” pvmd

Slave pvmd’s arestarted via rsh…

58Simple PVM C Example, First Process

#include <stdio.h>#include "pvm3.h"int main(){ int cc, tid; char buf[100]; printf("i'm t%x\n", pvm_mytid()); cc = pvm_spawn("hello_other", (char**)0, 0, "", 1, &tid); if (cc == 1) { cc = pvm_recv(-1, -1); pvm_bufinfo(cc, (int*)0, (int*)0, &tid); pvm_upkstr(buf); printf("from t%x: %s\n", tid, buf); { } else { printf("can't start hello_other\n"); } pvm_exit(); exit(0);}

59Simple PVM C Example, Second Process

#include "pvm3.h"int main(){ int ptid; char buf[100]; ptid = pvm_parent(); strcpy(buf, "hello, world from "); gethostname(buf + strlen(buf), 64); pvm_initsend(PvmDataDefault); pvm_pkstr(buf); pvm_send(ptid, 1); pvm_exit(); exit(0);}

60

Linda: Tuple Spaces

Linda is a small “coordination” language for distributed systems development with a few simple operations

Extends a traditional language like C or FORTRAN or Java

Easy for programmers since it isn’t necessary to learn an entire language from scratch

Appropriate for “bag of tasks” problems– e.g., many rendering algorithms in computer

graphics

61

Linda Operations

out(t) puts a tuple t into the “bag” in(t) to get a tuple t from the bag rd(t) to read (w/o removing) a tuple t eval(t) to create a process to evaluate the

tuple t Predicate primitives (newer):

– inp, rdp test for presence, behave like blocking versions if they return true

62

Simple Linda Example

int main(int argc, char *argv[]) {int nworker, j, hello(); nworker=atoi (argv[1]); for (j=0; j < nworker; j++)

eval ("worker", hello(j)); for(j=0; j < nworker; j++)

in("done" /* , could read other values here */); printf(“Got responses from all slaves.\n”);}

int hello(int i) { printf("Slave %d reporting.\n",i); out("done" /* , could return other values here */); return(0); }

63

Java is a Natural for Linda-ness

Instead of tuple spaces, object spaces…bags of objects

Operations insert and remove arbitrary objects from the space

Retrieve by “name” or by class Fairly easy to implement because of object

serialization facility Threaded implementations make powerful

extensions like transactions fairly easy JavaSpaces…

64

Java: RMI

RMI: Remote Method Invocation Java’s OO facility provides a superset of RPC

(Remote Procedure Call) functionality RMI provides distributed objects for Java

– Objects can reside on different machines, and other objects can invoke their methods

When searching for objects on a remote host:– rmi://host:port/name

Port defaults to 1099 if omitted

65

RMI Compilation/Deployment

Write interface for server Write server implementation Compile service interface, implementation Run rmic on server implementation

– Generates server_stub, server_skel

Client needs only server interface Server needs server_stub rmiregistry runs on server machine Server provides location of server_stub to rmiregistry (Client will automatically download server_stub upon

a lookup, if necessary)

66

RMI Schematic

Client Server_stub Server_skel

Server_Interface

Serverimplements

RMIC

67

RMI-in-action Schematic

Client Server_stub

Server_skel

public interface server

a(..)Server

rmiregistry

bind(this)

O = (server_type)lookup()

O.a(..)

68

Simple RMI Server Interface

// Meaning of life server interface.

import java.lang.*;

import java.io.*;

import java.rmi.*;

public interface RMI_MOLServerInterface extends Remote {

// reveal the meaning of life

public String reveal() throws java.rmi.RemoteException;

}

69

Simple RMI Client// Meaning of life client

import java.lang.*;import java.io.*;import java.rmi.*;public class RMI_MOLClient { public static void main (String args[]) throws Exception { if (args.length != 1) { throw new RuntimeException("Usage: java RMIMOLClient <host>");

} System.setSecurityManager(new RMISecurityManager());

RMI_MOLServerInterface mol = null; try {

mol = (RMI_MOLServerInterface)Naming.lookup("rmi://"+ args[0] + "/MOL");

}catch (java.rmi.NotBoundException e1) { System.out.println("No MOL service object bound on that host.");}catch (java.rmi.ConnectException e2) { System.out.println("Either the RMI registry or the MOL service is dead on that host.");}if (mol != null) { System.out.println(mol.reveal());}

}}

70Simple RMI Server Implementation

// Meaning of life server implementation.

import java.lang.*;import java.io.*;import java.rmi.*;import java.rmi.server.UnicastRemoteObject;

public class RMI_MOLServer extends UnicastRemoteObject implements RMI_MOLServerInterface { public RMI_MOLServer(String mol) throws RemoteException { System.setSecurityManager(new RMISecurityManager());

this.mol = mol; } public String reveal() throws java.rmi.RemoteException { return mol; } public static void main (String args[]) throws Exception { if (args.length != 1) { throw new RuntimeException("Usage: java RMI_MOLServer <string>");

} RMI_MOLServer us = new RMI_MOLServer(args[0]); Naming.rebind("MOL", us); }

// shhhhhhhh! private String mol;}

71

Compile…

javac RMI_MOLServerInterface.java javac RMI_MOLServer.java javac RMI_MOLClient.java rmic RMI_MOLServer

– generates RMI_MOLServer_stub.class and RMI_MOLServer_skel.class

Run rmiregistry on server end To run client and server…

java -Djava.security.policy="policy.all“ -Djava.rmi.server.codebase=

"file://c:/rmi/mol/" RMI_MOLServer "Life ain’t no box of chocolates.”

java -Djava.security.policy="policy.all" RMI_MOLClient localhost

72Un/reliable Communication Reliable Communication

– Virtual circuit: one path between sender & receiver. All packets sent through the path.

– Data received in the same order as it is sent.– TCP (Transmission Control Protocol) provides reliable

communication.

Unreliable communication– Datagrams: Different packets are sent through different

paths.– Data might be lost or out of sequence.– UDP (User datagram Protocol) provides unreliable

communication.

73RPC Design

Structure– Caller: local call + stub– Callee: stub + actual procedure

Binding– Where to execute? Name/address of the server that offers

a service– Name server with inputs from service specifications of a

task.

Parameter & results– Packing: Convert to remote machine format– Unpacking: Convert to local machine format

74RPC Execution

Local call

Return

Querybindingserver

Paramspacking

Wait

Unpackresult

Unpack

Localcall

Packresults

Executeprocedure

Return

ReceiveQuery

ReturnServer Address

RegisterServices

BindingServer

Caller Callee

Stub StubLocal Proc. Remote Proc.

75RPC Semantics

At least once– A RPC results in zero or more invocation.– Partial call, i.e., unsuccessful call: zero, partial, one or

more executions.

Exactly once– Only one call maximum– Unsuccessful? : zero, partial, or one execution

At most once– Zero or one. No partial executions.

76RPC Implementation

Sending/receiving parameters:– Use reliable communication? : – Use datagrams/unreliable?– Implies the choice of semantics: how many times a RPC

may be invoked.

77RPC Execution

Local call

Return

Querybindingserver

Paramspacking

Wait

Unpackresult

Unpack

Localcall

Packresults

Executeprocedure

Return

ReceiveQuery

ReturnServer Address

RegisterServices

BindingServer

Caller Callee

Stub StubLocal Proc. Remote Proc.

78Sun RPC Specification

struct square_in { /* input argument */long arg1;

}struct square_out { /* output argument */

long res2;}program SQUARE_PROG { version SQUARE_VERS { square_out SQUAREPROC(square_in) = 1; /* procedure no. = 1 */ } = 1; /* version number */} = 0x3123 0000; /* program number */

Server Side:

square.x: /* file name */

rpcgen –C square.x /* -C: generate C prototypes in square.h */

Compilation Procedure:

79Sun RPC: Server Side

#include “unpipc.h” /* local headers */#include “square.h” /* generated by RPCgen */

square_out * squareproc_1_svc (square_in *inp, struct svc_reg *rqstp) { static square_out out;

out.res1 = inp->arg1 * inp->arg1; return(&out);}

server.c:

cc –c server.c –o server.occ –c square_svc.c –o square_svc.o /* contains “main” */cc –o server server.o square_svc.o square.xdr.o libunpipc.a -lnsl

Compilation Procedures:

Notes:libunpipc.a: library used in Stevens book; lnsl: Solaris system libraryIncluding RPC and XDR runtime functions

80Client Side

#include “unpipc.h” /* local headers */#include “square.h” /* generated by rpcgen */

main(int argc, char **argv) { CLIENT *cl; /* defined in rpc.h */ square_in in; square_out *outp;

if (argc != 3) err_quit(“usage: client <hostname> <integer_value>”); cl = Clnt_create(argv[1], SQUARE_PROG, SQUARE_VERS, “tcp”); in.arg1 = atol(argv[2]); if ((outp = squareproc_1(&in, cl) == NULL)

err_quit(“%s”, clnt_sperror(cl, argv[1])); printf(“result: %d\n”, outp->res1); exit(0);}

client.c :

81Client Compilation

cc –c client.c –o client.occ –c square_clnt.c –o square_clnt.occ –c square_xdr.c –o square_xdr.occ –o client client.o square_client.o square_xdr.o libunpipc.a -lnsl

Compilation:

Execution:

client bsdi 11 -> result: 121client 209.76.87.90 22 -> result: 484client nosuchhost 11 -> nosuchhost:RPC:Unknownhost….Client localhost 11 -> localhost: RPC: Program not registered

Notes:Rpcgen: generates square_xdr.c -> XDR data conversions

square_clnt.c -> client stub

82RPC Client-Server

square.x

RPC Specification File

square.h

client.c square_clnt.c square_xdr.c square_svc.c server.c

cc cc

client server

rpcgen

Runtimelibrary

#include

83RPC Implementation

Sending/receiving parameters:– Use reliable communication? : – Use datagrams/unreliable?– Implies the choice of semantics: how many times a RPC

may be invoked.

Reference Book:Unix Network Programming by Richard Stevens, Prentice-Hall.