Consistency and Replication

Chapter 6

Object Replication (1)

Organization of a distributed remote object shared by two different clients.

Object Replication (2)

a) A remote object capable of handling concurrent invocations on its own.b) A remote object for which an object adapter is required to handle concurrent

invocations

Object Replication (3)

a) A distributed system for replication-aware distributed objects.b) A distributed system responsible for replica management

Data-Centric Consistency Models

The general organization of a logical data store, physically distributed and replicated across multiple processes.

Strict Consistency

Behavior of two processes, operating on the same data item.• A strictly consistent store.• A store that is not strictly consistent.

Idea: All writes are instantaneously propagated to all processes!

time

Sequential Consistency (1)

a) A sequentially consistent data store. P3 and P4 see same sequences

b) A data store that is not sequentially consistent.P3 and P4 see different sequences

Idea: Results appear as if operations of different processes were executed sequentially, and operations within a process appear in the order of its program.

Sequential Consistency (2)

Four valid execution sequences for the processes. The vertical axis is time.

x = 1;

print (y, z);

y = 1;

print (x, z);

z = 1;

print (x, y);

Prints: 001011

Signature: 001011

(a)

x = 1;

y = 1;

print (x,z);

print(y, z);

z = 1;

print (x, y);

Prints: 101011

Signature: 101011

(b)

y = 1;

z = 1;

print (x, y);

print (x, z);

x = 1;

print (y, z);

Prints: 010111

Signature:

110101

(c)

y = 1;

x = 1;

z = 1;

print (x, z);

print (y, z);

print (x, y);

Prints: 111111

Signature:

111111

(d)

Process P1 Process P2 Process P3

x = 1;

print ( y, z);

y = 1;

print (x, z);

z = 1;

print (x, y);

Causal Consistency (1)

This sequence is allowed with a causally-consistent store, but not with sequentially or strictly consistent store.

Idea: If writes are causally related, they appear in the same order, otherwise there is no restriction for their order.

concurrent

Causal Consistency (2)

a) A violation of a causally-consistent store.b) A correct sequence of events in a causally-consistent store.

FIFO Consistency

A valid sequence of events of FIFO consistency

Idea: Writes done by a single process are seen by all other processes

in the order in which they were issued, but writes from different

processes may be seen in a different order by different processes.

Weak Consistency

a) A valid sequence of events for weak consistency.b) An invalid sequence for weak consistency.

Idea: Use of an explicit synchronization operation S. When called,

everything is made consistent: local changes are propagated

to other processes and remote changes are propagated to local

process.

Release Consistency

A valid event sequence for release consistency.

Idea: Why only one synchronization operation, better two:

Acquire and Release. This enhances performance relatively to

Weak Consistency:

1. Acquire: All data are brought from remote sites.

2. Release: All local changes are made visible to others.

Entry Consistency

A valid event sequence for entry consistency.

Idea: Even Release Consistency is improvable:

1. Delay propagation after a release until a remote process

acquires the data

Lazy Release Consistency

2. Do not do that for all data, rather used ones.

Associate locks with (individual) shared data items.

Summary of Consistency Models

a) Consistency models not using synchronization operations.

b) Models with synchronization operations.

Consistency Description

Strict Absolute time ordering of all shared accesses matters.

SequentialAll processes see all shared accesses in the same order. Accesses are not ordered in time

Causal All processes see causally-related shared accesses in the same order.

FIFOAll processes see writes from each other in the order they were used. Writes from different processes may not always be seen in that order

(a)

Consistency Description

Weak Shared data can be counted on to be consistent only after a synchronization is done

Release Shared data are made consistent when a critical region is exited

Entry Shared data pertaining to a critical region are made consistent when a critical region is entered.

(b)

Eventual Consistency

The principle of a mobile user accessing different replicas of a distributed database.

Eventual consistency: few updates that propagate gradually to all replicas.

High degree of inconsistency is tolerated

E.g. WWW

Problem: mobile users (see figure); solution: client-centric consistency models.

W(x)a

R(x)b ???Previously written a

has not yet beenpropagated to this

replica

Monotonic Reads

The read operations performed by a single process P at two different local copies of the same data store.

a) A monotonic-read consistent data store: WS(x1;x2) means that version x1 has been propagated to L2.

b) A data store that does not provide monotonic reads: WS(x2) does not include version x1!

Idea: If a client reads some value, a subsequent read returns that value or a newer one (but never an older one).

Monotonic Writes

The write operations performed by a single process P at two different local copies of the same data store

a) A monotonic-write consistent data store: previous value x1 has been propagated.

b) A data store that does not provide monotonic-write consistency: value x1 has not been propagated.

Idea: If a client writes some value A, a subsequent write will operate on the most recently written value A (or a newer one).

{client writes x1}x1 propagated

{client writes x2}

Read Your Writes

a) A data store that provides read-your-writes consistency.b) A data store that does not.Idea: If a client writes some value A, a subsequent read will return the most

recently written value A (or a newer one).

{client writes x1}x1 propagated

{client reads x2}

Writes Follow Reads

a) A writes-follow-reads consistent data storeb) A data store that does not provide writes-follow-reads consistencyIdea: If a client reads some value A, a subsequent write will operate on the most

recently read value A (or a newer one).

{client reads x1}x1 propagated

{client writes x2 usingversion x1}

Replica Placement

The logical organization of different kinds of copies of a data store into three concentric rings.

Permanent replicas: such as replicas in a cluster or mirrors in WAN.Server-initiated replication: Server keeps track of clients that frequently use data, and it

replicates/migrates the data closer to clients, if needed. push cashesClient-initiated replication: Client uses a cache for most frequently used data.

Server-Initiated Replicas

Counting access requests from different clients.In the figure: Server Q owns a copy of file F and counts the accesses originating from

clients C1 and C2 on behalf of server P. If necessary, Q decides to migrate/replicate the file to/in the new location P.

Pull versus Push Protocols for Update Propagation

A comparison between push-based and pull-based protocols in the case of multiple client, single server systems.

Update propagation: Push-based: server initiates the update propagation to other replicas (e.g. in permanent/server-initiated replication). Use of “short” leases (time period in which an update is guaranteed) may help server reduce its state and be more efficient. Pull-based: Client requests a new update from server (rather with client-initiated replication).

Issue Push-based Pull-based

State of server List of client replicas and caches (stateful) None (stateless)

Messages sentUpdate (if only invalidation is sent, later, a fetch update is initiated by client)

Poll and update

Response time at client

Immediate (or fetch-update time) Fetch-update time

Remote-Write Protocols (1)

Primary-based remote-write protocol with a fixed server to which all read and write operations are forwarded.

– In this consistency protocol, there is only one copy (i.e. original) of an item x (no replication).

– Writes and reads go through this remote copy (well-suited for sequentially consistency)

Remote-Write Protocols (2)

The principle of primary-backup protocol.– In this consistency protocol, there are multiple copies of an item x (replication).– Writes go through a primary copy and are forwarded to backups, reads may work

locally (also well-suited for sequential consistency, since primary serializes updates).

Local-Write Protocols (1)

Primary-based local-write protocol in which a single copy is migrated between processes.– In this consistency protocol, there is only one copy of an item x (no

replication).– Writes and reads are performed after migrating the copy to the local server.

Local-Write Protocols (2)

Primary-backup protocol in which the primary migrates to the process wanting to perform an update.– In this consistency protocol, there are multiple copies of an item x (replication)

– Writes are performed after migrating primary to local server; updates are forwarded to other backup locations. Reads may proceed locally.

Active Replication (1)

The problem of replicated invocations.Active replication: Each replica has its own process for carrying updates.

Problems: 1) Updates need to be made in same order ( timestamps).

2) Invocations cause a problem (see figure).

e.g. withdraw frommy account $300instead of $100 !

Active Replication (2)

a) Forwarding an invocation request from a replicated object.b) Returning a reply to a replicated object.

Quorum-Based Protocols

Examples of the voting algorithm:

a) A correct choice of read and write set: Any three will include one from write quorum (latest version).

b) A correct choice, known as ROWA (read one, write all): Read from any replica will lead to latest version, but writes involves all replicas.

N: number of replicas (here 12)

NR : number of replicas in read quorum

NW : number of replicas in write quorum

NR + NW > N (1)

NW > N/2 (2)

(a) (b)

Orca

A simplified stack object in Orca, with internal data and two operations.

OBJECT IMPLEMENTATION stack; top: integer; # variable indicating the top stack: ARRAY[integer 0..N-1] OF integer # storage for the stack

OPERATION push (item: integer) # function returning nothing BEGIN GUARD top < N DO stack [top] := item; # push item onto the stack top := top + 1; # increment the stack pointer OD; END;

OPERATION pop():integer; # function returning an integer BEGIN GUARD top > 0 DO # suspend if the stack is empty top := top – 1; # decrement the stack pointer RETURN stack [top]; # return the top item OD; END;BEGIN top := 0; # initializationEND;

Management of Shared Objects in Orca

Four cases of a process P performing an operation on an object O in Orca.