Data Link Protocols

Reliable FIFO communication

using less reliable channels

By Ken Schmidt

Protocols discussed• Stennings Protocol

– Sent with unbounded integer tags– Tolerates loss– Duplication– reordering

• Alternating Bit Protocol– Sent with bounded tags– Tolerates loss– Duplication– Does NOT tolerate reordering

• Bounded Tag Protocol– That tolerates reordering

• Tolerating Crashes

Message Levels

Two levels• User

– High-level– Alphabet M– SEND, RECEIVE, etc upper case

• Underlying channel – Low-level– Alphabet M’– send, receive, etc lower case

How are these protocols used?OSI Model

Application Layer

Presentation Layer

Session Layer

Transport Layer

Network Layer

Data Link Layer

Physical Layer

TCP-speed up, slow down

IP addr: 192.168.1.1

MAC addr:

00:60:08:C4:99:AA Cat 5 cable

FTP

SEND

send

RECEIVE

receive

Data Link Protocol

Universal Reliable FIFO Channel

RECEIVE1-2P1 P2

SEND1-2

C2,1

C1,2send 1,2

send 2,1

receive1,2

receive2,1

Unreliable FIFO Channel

(The Internet)

Loss Definitions

• SLL: Strong loss limitation – fair to a particular message (infinitely many receive events all caused by the same send event

• WLL: Weak loss limitation – fair to messages as a group ( infinitely many sends cause receives of infinitely many messages)

Three types of unreliability in the channel

• Loss of a message

• Duplication of messages

• Reordering of messages

Stenning’s Protocol• Tolerates all three types of channel unreliability

• P1 puts high-level message from U1 in buffer1, tags with unbounded successive integers starting with 1

• Repeatedly sends first message in buffer1, along with it’s tag, to P2

• P2 accepts the first message tagged with 1 that it receives

• P2 accepts subsequent messages each one of which has a tag that is one greater than the previously accepted one

• P2 places messages in buffer2 for reliable, in order, delivery to U2

• P2acks a high-level message repeatedly by sending its tag back to P1

• When P1 receives an ack for its current tag, it moves to the next high-level message 694

Lemma 22.2: Let be any execution of Stenning

1.|P1 & |P2 are fair

2.|A1,2 & |A2,1 satisfy liveness

Then |ext(F) fairtraces(F)

•Show that any high level message that is submitted to P1 eventually gets to P2

•By contradiction: suppose that a message is submitted but not delivered, k is the tag, so the tag2 remains <= k-1 forever

•The message reached the front of buffer1 (to be sent), if k=1, then it is the 1st message, if k>=1 then previous message was received by P2 and delivered to U2

•Tag2 eventually reaches and stays at k-1

•By fairness of P2, P2 keeps sending k-1 messages forever and by weak loss limitation (WLL), a copy of k-1 is received by P1, the message with tag = k-1 gets removed from buffer1 and m reaches the front

•Once at the front, it stays there forever because we supposed that it never reached P2, so P1 keeps sending forever (m,k)

•WLL implies that eventually a copy of (m,k) is received and accepted by P2, a contradiction

This is the main correctness result for Stenning protocol

Alternating Bit Protocol• Similar to Stenning but with tags bounded to 0,1 (Stenning

integer tags replaced with the low order bit)• Can loose or duplicate messages,can NOT reorder them• P1 places high level message in buffer1 tag alternating 0-1• P1 repeatedly sends the 1st message in its buffer & tag to P2

• P2 accepts first message tagged with 1 it receives and each subsequent message if its tag is different from the tag of the previously accepted message

• P2 places message in buffer2 and delivers it to U2

• P2 acks the message repeatedly by sending its tag back to P1

• When P1 receives and ack for its current tag, it begins processing the next message

• Satisfies lossy FIFO channel of 14.1.1 462 (limited loss, finite duplication, no reordering)

• Premise of proofs for ABP is that ABP maps exactly to Stenning for all buffers and tags except that ABP tags are Stenning tags mod 2

• And where ABP specifies a lossy FIFO channel and Stenning uses a lossy reordering FIFO channel to obtain reliable FIFO delivery

Similarity between Stenning and ABP for proofs

Bounded Tag Protocols Tolerating Reordering

• What goes wrong with ABP• Channels that reorder low level

messages can get fooled into accepting an old high level message that happens to arrive tagged with the same bit as the one currently expected

• Can cause duplicate delivery of a high level message, violating requirements for reliable communication – ABP does not work with channels that can reorder low level messages

P1 P2

(m,1)

(m,1)

(m’,0)

accept m

accept m

accept m’

Impossibility of a protocol that both Reorders and Duplicates

• Theorem 22.8: There is no bounded tag protocol that guarantees reliable FIFO delivery using channels that can reorder and duplicate messages

• By contradiction, suppose there is such an implementation, construct an execution with incorrect behavior

• Run the system until it is not possible for Process P1 to send any additional low-level messages with new values (a finite extension of 1)

• Let 2 be a fair consistent extension of 1 with one additional SEND event for a total of n+1 SEND events

• By correctness, all messages submitted by U1 in 2 must eventually get delivered to U2, so that there are n+1 RECEIVE events in 2

• Let 3 be a prefix to 2 up to an including the last RECEIVE event• Construct a finitely consistent execution 4 where

4 is an extension of 1

4 is indistinguishable from 1 to P1

4 is indistinguishable from 3 to P2

• Construct 4 by preventing all events involving P1 immediately after 1 while allowing all events of P2 to proceed exactly as in 3

• The additional events of P2 might include receive events, send events, and internal events, as well as the required RECEIVE event

• Show that P2 can be permitted to receive the same low level message after 1 as it dos in 3 even though P1 does not send any additional low level messages after 1

• This is possible because all low level messages sent by P1 in 3 after 1 contain values that P1 has already sent in 1

• Any low level message received by P2 after 1 could be a duplication of some low level message sent in 1

• In 4 there are exactly n SEND events and n+1 RECEIVE events• Extend 4 to a fair consistent execution without introducing any new SEND

events, the resulting execution has more RECEIVE events than SEND events contradicting correctness.

Bounded Tag Protocol Tolerating Loss and Reordering

Layer1

Layer1

Layer2

Layered structure of the Probe protocol 707

I1,2

I2,1

Probe algorithm

• Layer 1 uses the given channels to implement intermediate channels I1,2 and I2,1 that do not reorder messages, but can lose or duplicate them (satisfies the cause function)

• Cause function is required not to reorder messages, but to satisfy the WLL, infinite duplication is allowed

• Layer 2 uses the resulting FIFO channels to implement reliable FIFO communication

Layer1

Layer1

Layer2

SEND1-2 RECEIVE1-2

send 1,2

receive2,1

receive1,2

send 2,1

I1,2

I2,1

•Layer 2, ABP can be used

•I1,2 and I2,1 are not only lossy FIFO channels, but also allow infinite duplication

• In Each Layer 1 implementation:• Process P1 sends a low level message to P2 only in response to

an explicit probe message from P2

• The low level message P1 sends always contains the value of the most recent high level message that it has received form U1 which it keeps track of in latest e.g.:P1 does not remember all messages submitted by U1, just the most recent one (I channel is permitted to lose some high level messages)

• To ensure that P1 only sends messages in response to probes, P1 keeps a variable unanswered which it increments when it receives a probe and decrements whenever it sends a low level message

• Process P2 continually sends probes to P1, keeping track in pending the # of probes it has ever sent

• P2 counts in count(m) # of copies of each high level message received since the last time it delivered a high level message to U2

• Whenever it delivers a msg to U2, P2 sets old to pending• When count(m) exceeds old, P2 can output m

P1 P2

pending ++ (= 1)probe

unanswered ++ (= 1) pending ++ (= 2)

probe

unanswered ++ (= 2)SEND(m)latest = munanswered - - (= 1) note: init old=0send (m = latest)

count ++ (= 1)

now, count > old so: RECEIVE reset count =0

old = pending = 2unanswered - - (= 0) send (m = latest)

count ++ (= 1)since no duplication, if count<oldreceived, must have been sent no RECEIVE

Layer 1(p710) can loose and reorder, cannot duplicateLayer 2(ABP) can loose and duplicate, cannot reorder

unanswered = probes received - # of sends

pending = total no of probes ever sentcount = # msg received since last RECEIVEold = # of receives before a RECEIVE can occur

Now for the next SEND, the # of sends must exceed old before one is RECEIVEdFor instance, the next SEND would require 3 probes and 3 sends before a RECEIVE could occur, since there are no channel duplicated messages, at least one of these 3 must have been sent by P1 as a new message since the last RECEIVE. & Layer 2 no reordering

Lemma 22.9: Layer 1 of the probe protocol, with the given nonduplicating channels implements the intermediate channel I1,2

• I1,2 does not reorder messages: when P2 performs any RECEIVE after the first one, it checks that count(m)>old where m is the high level msg being delivered

• Old variable plus the fact that P1 only sends msg in response to probe and channels do not duplicate msg, imply at most old low level msg in transit from P1 to P2 at the point of the preceding RECEIVE event

• At least one of the msg containing m must have been sent by P1 since the preceding RECEIVE event

• Implies that m must have been the value of latest1 at some point after the preceding RECEIVE event, implying no reordering occurs

• I1,2 guarantees the WLL condition, if there are infinitely many SEND events, then infinitely many of them must have corresponding RECEIVE events

• Suppose infinitely many SEND events, P2 keeps sending probes• P1 keeps responding to probe messages, liveness for the channel and the

finiteness of the high level message alphabet M imply that P2 performs infinitely many RECEIVE events

• Any msg that is delivered to U2 after the first one must have the value of latest at some point after the previous RECEIVE event, implying that RECEIVE events correspond to infinitely many different SEND events

Complete probe needs SLL• Theorem 22.10: The probe protocol, using any nonduplicating SLL

channels implements the reliable FIFO channel F

• For every fair execution , |ext(F) fairtraces(F)

Complexity• Can require more and more low level messages to deliver later and

later high level messages

• Layer 1, once k low level messages have been lost, it requires at least k +1 low level messages to deliver each subsequent high level message even if no further losses occur

Nonexistence of Efficient Protocols Tolerating Loss and Reordering

• Definition of k-extension: if is a complete execution (# of SEND and RECEIVE events are equal)– there is an extension ’ after which contains exactly one SEND and one

RECEIVE– no old messages are received in ’– # of receive events in ’ after <= k (k-message bounded)

• Theorem 22.11: There is no message bounded protocol that implements a reliable FIFO communication using channels that can lose and reorder messages– By contradiction, assume there is such a channel that is k bounded– Similar to Theorem 22.8, produce a finitely consistent execution 1

1 is an extension of 1 is indistinguishable from in P1

1 is indistinguishable from ’ in P2

– Do this by preventing all events involving P1 immediately after while allowing all events of P2 to proceed exactly as in ’

– Because messages already in transit after are received at P2

– Contradiction: there are more RECEIVE events than SEND events

Tolerating Crashes1. Crash with later recovery where process just resumes where it left off,

no different from a process that pauses

2. Crash where information in the state is lost (from volatile memory) and the process later recovers

• Run a recovery protocol to restore info from stable memory to restore the volatile memory (recover step)

• Input action CRASH and output RECOVER, if CRASH occurs, enables RECOVER and disables everything else until RECOVER completes

RECEIVE1-2P1 P2

SEND1-2

C2,1

C1,2send 1,2

send 2,1

receive1,2

receive2,1

CR

ASH

1

CR

ASH

2RE

CO

VE

R 1

RECOVER 2

A Simple Impossibility Result

• RECOVER sets the entire state back to some arbitrary start state, all state info is lost

• Easy to see that is impossible to solve the reliable FIFO communication problem even with reliable FIFO channels

– After a crash of P2, the protocol is unable to tell whether or not a high level message has just been delivered to U2

A Harder Impossibility Result

• Weaken the problem statement by requiring less at the external interface: no duplication but allows reordering, and only require that a message be delivered if its SEND event has no following RECOVER event (infinitely many CRASH and RECOVER events, no messages are required to be delivered at all, if finitely many events, any message sent after the last RECOVER must be delivered)

• Do not permit duplication or reordering, all channels can do is lose messages

• Theorem 22.14: There is no protocol using lossy low level channels that guarantees communication with no duplication and with no losses after all crashes and recoveries have ceased

• It is possible, using crashes, to create a situation in which both processes have the same states that they had at the end of , but in which one of the channels has in transit the entire sequence of low level messages sent along that channel in

• Notation ī is the opposite process to i

• Claim 22.15 Let be any crash free finitely consistent execution, either contains no steps or the last step in is a step of Pi, then there is a finitely consistent execution of ’ where

– The state of Pi is the state (, i)

– The state of Pī is the state (, ī)

– The sequence out(, i) is in transit from Pi to Pī

– Execution ’ may contain CRASH and RECOVER events, but there are no unmatched CRASH events, i.e.: each CRASH has a following corresponding RECOVER

• Use claim 22.15 to complete Theorem 22.14– Let be any crash free finitely consistent execution containing

exactly one SEND event and its corresponding RECEIVE event and assume without loss of generality that ends with the RECEIVE event

– Construct 1 whose final process states are the same as those in but that has a SEND as its last external event (no following SEND, RECEIVE, CRASH, or RECOVER events and no unmatched CRASH events) 721

– Claim 22.15 yields a finitely consistent execution ’ that ends with the process state equal to state (, 1) and state (, 2) and with out(, 2) in transit from P2 to P1 and no unmatched CRASH events

– Construct 1 by extending ’ by crashing and recovering P1 then running it on its own just as in (with the input sequence in(, 1) is in transit in the incoming channel)

– This allows P1 to reach state(, 1) again (is a SEND step but no other user interface step in the portion of 1 after ’, yielding the claim properties for 1)

– Contradiction: let 2 be an extension of 1 to a fair consistent execution that contains no further SEND, CRASH, or RECOVER events and in which every low level message received after 1 is send after 1 (all old low level messages are lost)

– By correctness, there is at least one RECEIVE event in the suffix, to correspond with the last SEND in 1 (the portion of 2 after 1 could also be attached after yielding a fair consistent execution because the two processes are in the same states after and 1 and because all old low level messages are lost

– This violates correctness, since already has an equal number of SEND and RECEIVE events (one of each) and the suffix contains at least one more RECEIVE but no more SENDs)

A Practical Protocol

• It is important in practice to have reliable FIFO message delivery which will work in spite of crashes

• FivePacketHandshake Protocol is used in setting up network connections, used in TCP, ISO TP-4, NCP and other transport protocols (packet = low level message)

• P1 maintains a buffer of high level message submitted by U1 and works to get the message to P2 one at a time

• For each high level message that P1 tries to send, there is an initial two way exchange of low level packets

• P1first sends a new UID v to P2 in a (“needuid”,v) packet

• P2 pairs this UID v with another new UID u that it generates and sends the pair (u, v) back to P1 in an (“accept”, u, v)

• P1 can now recognize that this packet is recent because it contains P1’s latest UID v

• P1 then chooses u as the UID for the high level message it is trying to send

• P1 sends the latest high level message m to P2 paired with the new UID u in a (“send”, m, u) packet

• P2 can recognize that the packet is recent because it contains P2’s latest UID u

• After accepting a message, P2 acks the packet with a (“ack”, u)

• P1 sends a (“cleanup”, u) to P2 to tell P2 to discard the current UID

P1 P2

“needuid”, v

“accept”, u, v

“send”, m, u

“ack”, u

“cleanup”, u

• Theorem 22.16: FivePacketHandshake guarantees message delivery using channels that lose, reorder and duplicate messages with SLL and finite restrictions on duplications

• Safety: protocol does not reorder or duplicate messages is easy to see that the use of UIDs u and v prevent this from happening

• Liveness: There is a situation where P1 has status = needuid while P2 has status = accept but UID v = value v that is different from P1s current value of UID v (any current needuid packet from P1 is ignored by P2

• Show that v is eventually dislodged

– Fairness for P2 implies that P2 sends infinitely many (“accept”, u, v) packets

– Channel liveness implies that infinitely many of these arrive at P1 which cause a (“cleanup, u) to be sent

– Channel liveness implies that eventually one of these (“cleanup, u) packets must arrive at P2 dislodging v

– After v is dislodged, it is possible for P2 to acquire another value of UID v that is also not current value at P1, which can happen if P1 receives an old needuid packet.

– This old value v is dislodged as above, which can happen only finitely many times (because channels only allow finite duplication), so eventually a current needuid packet arrives at P2

• Eventual quiescence– In case there are only finitely many SEND, CRASH, and RECOVER

events, eventually both processes reach and remain in states that are the same as their initial states, except for the used sets

– No memory needs to be reserved for the use of this protocol when it is not actively processing messages from U1 to U2

– Allows the same pair of processes in a network to simulate the FivePacketHandshake protocol in parallel for a very large number of distinct pairs (U1, U2) of users

• Finite UID sets– The number of available UIDs is very large but not infinite

– UIDs can be chosen to be successively increasing integers mod n for some very large n

– Finite version works correctly (and can be proven via simulation relation) provided that UIDs wrap around and old values of u have already been discarded

– This is possible because of known limits on the message delivery time, local processing time and rate of submission of high level messages, or of an explicit policy of discarding old packets