Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University...

21
Sib: Alternatives of stateful replication in applicati on servers Huaigu WU Mcgill University February, 2003
  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    221
  • download

    0

Transcript of Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University...

Page 1: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Sib:Alternatives of stateful replication

in application servers

Huaigu WU

Mcgill University

February, 2003

Page 2: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Purpose of Sib

• An alternative approach to implement replication in application server– Abstract level (derived from the J2EE specific

ation)– Focus on Stateful Session Bean– Will extend to Stateless Session Bean and En

tity Bean

Page 3: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Outline

• Sib VS. JBoss implementation– Simple Calls– Calling Hierarchies– Direct Database Acce

ss

• Compare– Normal process– Site crash and failove

r

Application Server

Client

SFB 1

1 2

Application Server

Client

SFB 1

1 4

Data

2 3

Application Server

Client

SFB 1

1 4

SFB 22

3

Page 4: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Assumption

• We don’t consider partition.• Abstract component architecture

– Component in this talk consists of client, Stateful Session Bean (SFB) and third-party database

• Third-party database will be used without modification

• Deterministic component for basic analysis• One primary, one (or more) backup• So far, we assume every component is single thr

eaded

Page 5: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Normal Processing in Simple Calls (JBoss)

• Only primary receives and processes request

• Replicate primary’s up-to-date state to backups at the end of every request before returning its reply

• Backup updates state after every request

Backup Pri mary

Client

Cluster

SFB 1 SFB 1

1 43 6

2 5

Page 6: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Normal Processing in Simple Calls (Sib)

• At the beginning, states of all replicas are the same

• request is broadcast to both primary and backup

• Primary processes requests• Backup receives requests• Occasionally (periodically),

Primary’s state is replicated to backup– Replication Point

(operational quiescence )• Independent for

component• Deterministic component:

it could be any time point

Pri mary

Client

Cluster

SFB 1 SFB 1

Backup

4'

1

2

3

4

0

Page 7: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Comparison• Network overhead for one component

– Message types:• Request (req), Reply (rep), Replicated state information (s)

– Message size: s >> req • JBoss analysis: s = 2-3 KByte

– Message number (N is number of sites)• JBoss: 1 request: 1 req + 1 rep + N*s• Sib: 1 requset: N req + 1 rep + q * N * s (q is the frequency of repli

cation, q<=1)• Compare: JBoss > Sib

• Backup’s CPU overhead– JBoss: receive and install state– Sib: receive requests– Assume install state > install request, so JBoss > Sib

Page 8: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Site Crash and Failover for Simple Calls (JBoss)

• Characters– When primary fails, client resends

request to the new primary• Issues

– Resend must be repeatable • Content

– Duplicated requests must be identified by SFB

Cluster

Client

SFB 1

Primary

1

SFB 1

Backup

Resend 1

3

2

Cluster

Client

SFB 1

Primary

1

SFB 1

Backup

Resend 1

3

2

Cluster

Client

SFB 1

Primary

1

SFB 1

Backup2

Resend 1

3

Identify duplicated request 1(Does JBoss do it ?)

Page 9: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Site Crash and Failover for Simple Calls (Sib)

• Characters– When primary fails, the new

primary automatically re-executes all requests starting from last replication point

– Client does not need to resend request

• Issues– Duplicated replies must be

identified by client– Replies must be deterministic

Cluster

Client

SFB 1

Primary

SFB 1

Backup

2

2

Identify duplicated reply 2(testable result)

Execute request 1

1

Cluster

Client

SFB 1

Primary

SFB 1

Backup

2 Execute request 1

1

Page 10: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Normal Processing in Calling Hierarchies

JBoss Sib

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

1 6

2 4

3

5

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

4

3

12

0

0

3'

Page 11: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Site Crash and Failover for Calling Hierarchies (JBoss)

• Assumption– Node fails (all beans in the node fail)

• Characters– Similar to simple calls for one

component

• Issues– Duplicated internal

requests must be identified by SFB

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

1

2

Resend 1

6

3

45

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

1

2

Resend 1

2

6

5

3

4

Duplicatedinternal

request 2

4

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

1

2

Resend 1

2 4

3

6

5

Page 12: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Site Crash and Failover for Calling Hierarchies (Sib)

• Assumption:– Node fails (all beans in the node fail)

• Character:– Every backup SFB re-executes

requests starting from its own last replication point

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

4

3 7 7632

8

3'

Duplicatedreply 8

2 6

0

0

Discardduplicated

internalrequests

Identifyduplicated

internalrequests

Duplicatedreply 4

1 5

• Issues– Duplicated internal requests

must be identified by SFB

– Duplicated internal replies must be identified by SFB

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

4

3 7 7

8

Duplicatedreply 81 5

2 6

0

0 4'

Re-executerequest 2,6

Re-executerequest 5

3 6

Duplicatedinternalrequest

Duplicatedinternalreplies

BackupPrimary

Client

Cluster

SFB 1 SFB 1

SFB 2 SFB 2

4

3 7 7

8

Duplicatedreply 81 5

2 6

0

0 4'

Re-executerequest 6

Re-executerequest 5

6

Duplicatedinternalrequest

3'

Page 13: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Direct Database Access (JBoss)

• Normal process

Cluster

Client

SFB 1

Primary

1

SFB 1

Backup

Data

Write and com

mit

Ack

Resend 1

3

2

Possible duplicateddatabase accessW

rite

and

com

mit

• Failover– NO “Exactly-once” Semantic

Cluster

Client

SFB 1

Primary

1

SFB 1

Backup

Data

Write and com

mit

2

3

Ack

Resource Manager(Resource Adapter)

Page 14: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Direct Database Access (Sib)

• Normal process– Data bind to specified

Resource Manager

– All requests to some data is forwarded to its specified RM

– Focus on write operation

Data

BackupPrimary

Client

Cluster

SFB 1 SFB 1

RM RM

2

Ack

1

0

0

SQ

L

Page 15: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Direct Database Access (Sib)• Failover suggestion 1

– Characters• Write bind to commit• Converse Undo, start from

the last write.• If original write is

successful, undo rollback it.• If original write is failed,

undo has no effect.– Issues

• Tables must have primary key to be able to generate undo

• In case primary committed, other access between commit and undo might be possible

– Conclusion• Simple during normal

process• Complex during recovery

Data

BackupPrimary

Client

Cluster

SFB 1 SFB1

RM RM

10

0

Write &

Com

mit U

ndo

toro

llbac

k(r

epea

tabl

e)A

ckW

rite

& C

omm

itA

ck

AckSQL

2

Re-executerequest 1

Duplicated internalSQL request

Re-executeSQL request

SQ

L

Generateundo

Page 16: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Direct Database Access (Sib)

• Failover Suggestion 2– Characters

• Insert Additional mark before every commit

• If mark exists, redo is not required

• If mark does not exist, redo

– Issues• additional insert

overhead In normal process

• RM must maintain global transaction ids

– Conclusion• Complex during normal

process• Simple and correct

during recovery Data

BackupPrimary

Client

Cluster

SFB 1 SFB1

RM RM

1

0

0

Write A

ckR

edo,

if no

mar

k

AckSQL

2

Re-executerequest 1

Duplicated internalSQL request

SQ

L

Insert

Mark

comm

it

Che

ck M

ark

MarkExist?

Page 17: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Summary• Sib’s Advantage

– Better performance• State information (including response information) is much

bigger than request• Infrequent replication points enough since EJB is short-living

so that failure is unlikely

– More powerful for “Exactly-once” database access

• Sib’s Disadvantage– More complicated recovery– Duplicated external responses

• Client needs to remove duplicated responses

Page 18: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Current Work• Compare schemas for complex transactions

– A transaction crosses several requests– Entity beans replication in complex transactions

• Multi-Thread: Typical Non-deterministic environment– Multi-Thread Client / Single-Thread SFB

• JBoss: – Repeatable request might be problematic (e.g. resend in different orde

r)

• Sib– Avoid resend requests– Total order o requests in all replicas

– Multi-Thread Client / Multi-Thread SFB• Both JBoss and Sib might be problematic

– Sib: special concurrency control and synchronous mechanism

• Component Failure

Page 19: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Problematic Example under Multi-Thread Environment

Cluster

Client

Primary

1

Backup

Resend 2

2

2

X+3X-3

Resend 1

S2

X=0

X=3

X=-3

X=0

S2 S1

X-3

Page 20: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Reference• [1] W.Zhao, L.E.Moser and P.M.Melliar-smith, “Unification of Replication and Transaction P

rocessing in the Three-Tier Architectures”, 22nd International Conference on Distributed Computing Systems (ICDCS'02), Vienna, Austria, July 02 - 05, 2002.

• [2] Marcia Pasin, Michel Riveill and Taisy Silva Weber, “High-Available Enterprise JavaBeans Using Group Communication System Support”, Proceedings of the Fourth European Research Seminar on Advances in Distributed Systems - ERSADS 2001, pages 161-166, University Residential Center of University of Bologna, Berliner (Frolic), Italy, 14-18 May 2001.

• [3] Roger Barga, David Lomet, Gerhard Weikum, “Recovery Guarantees for General Multi-Tier Aplications”, Proceedings of the 18th International Conference on Data Engineering (ICDE'02)

• [4] P.Narasimhan, L.E.Moser and P.M.Melliar-smith, “State Synchronization and Recovery for Strongly Consistent Replicated CORBA Objects”, International Conference on Dependable Systems and Networks, Göteborg, Sweden (July 2001).

• [5] N.Narasimhan, L.E.Moser and P.M.Melliar-smith, “Transparent Consistent Replication of Java RMI Object”, DOA 2000: 17-26

• [6] Svend Frølund, Rachid Guerraoui, “e-Transactions: End-to-End Reliability for Three-Tier Architectures”, TSE 28(4): 378-395 (2002)

• [7] Svend Frølund, Rachid Guerraoui, “Implementing e-Transactions with Asynchronous Replication”, IEEE Transactions on Parallel and Distributed Systems 12(2): 133-146 (2001)

• [8] Object Management Group, “Fault Tolerant CORBA Specification”. December 1999, OMG Technical Committee Document (orbos/00-04-04)

Page 21: Sib: Alternatives of stateful replication in application servers Huaigu WU Mcgill University February, 2003.

Reference• [9] IONA Technologies PLC, “White Paper Orbix E2A Application Load Balancing and

Fault Tolerance”, online documentation, http://www.iona.com/forms/wprequest.htm, April, 2002.

• [10] BEA WebLogic Server™, “Programming WebLogic Enterprise JavaBeans”, Copyright © 2000 BEA Systems, Inc. All rights reserved.

• [11] Gerhard Weikum, Gpttfried Vossen, “Transactional Information Systems: Theory. Algorithms, and the Practice of Concurrency Control and Recover”, Morgan Kaufmann Publishers, January 2002.

• [12] Ed Roman, Scott Ambler and Tyler Jewell, “Mastering Enterprise JavaBeans second edition”, Wiley Computer Publishing, John Wiley & Sons, Inc., ISBN: 0-471-41711-4, December 2001.

• [13] Sun Microsystems Inc., “Java™ 2 Platform Enterprise Edition Specification, v1.3”, online documentation, http://java.sun.com/j2ee/, Oct. 20, 2000.

• [14] Sun Microsystems Inc., “Enterprise JavaBeansTM Specification, Version 2.0”, online documentation, http://java.sun.com/j2ee/, Oct. 23, 2000.

• [15] R. Jimenez-Peris, M. Patino-Martinez, “Deterministic Scheduling and Online Recovery for Replicated Multithreaded Transactional Servers”, WDMS 2002, June 26, 2002.

• [16] Roman Vitenberg, Idit Keidar, Gregory V.Chockler and Danny Dolev, “Group Communication Specifications: A Comprehensive Study”, In ACM Computing Surveys 33(4), pages 1-43, December 2001.

• [17] Sacha Labourey, Bill Burke, “JBoss Clustering”, The JBoss Group, November 2002

• [18] Andreas Schaefer, “JBoss 3.0 Quick Start Guide”, The JBoss Group, July 2002