1

Phoenix Project:Phoenix Project:Making Applications Persistent Making Applications Persistent

David Lomet, Roger Barga

Microsoft Research, Redmond

Gerhard Weikum

University of Saarlands

Mark Tuttle

Compaq Cambridge Research Lab

Interns: Sanjay Agrawal, Thomas Baby,

Sirish Chandrasekaran, Stylianos Paparizos

2

Ex: An E-Commerce ServerEx: An E-Commerce Server

TP App Server usestransactions to accessthe DBMSDBMS

Web Server

TP App Server…

IP Sprayer

Web Server

TP App Server

Internet

3

ProblemProblem Half+ of TP system is outside the transactionHalf+ of TP system is outside the transaction

Much of it is distributed computingMuch of it is distributed computing Some is TP-specific (e.g. persistent queues, workflow)Some is TP-specific (e.g. persistent queues, workflow)

CurrentCurrent “Fact of Life”: “Fact of Life”: after a crashafter a crash DatabasesDatabases recover – to last committed transaction recover – to last committed transaction But But applicationsapplications “disappear” “disappear”

They crash themselves and cannot recoverThey crash themselves and cannot recover Or the DBMS crashes, and they cannot continue executionOr the DBMS crashes, and they cannot continue execution

Availability requires programmer actionsAvailability requires programmer actions ““Stateless” applicationsStateless” applications Explicit storing of state in queues or databaseExplicit storing of state in queues or database

4

Network

Traditional TP ApproachTraditional TP ApproachStateless ComponentsStateless Components

DCOM

Client Machine

ComponentComponent

Server Process

Request Q

Each method call is a transaction1.Read state from a transactional

queue 2.Invoke the method3.Write state to the transactional queue

Each method call is a transaction1.Read state from a transactional

queue 2.Invoke the method3.Write state to the transactional queue

Cmpnt1Cmpnt1 Reply Q

DCOM

Cmpnt1Cmpnt1

ComponentComponent

Server Process

5

Stateful Applications NowStateful Applications Now

Local Servicesand Data

ComponentBased

App Code

Network

Application programmers tend to write stateful applications that retain state across method calls

Application programmers tend to write stateful applications that retain state across method calls

DCOM

ComponentComponent

ComponentComponent

ComponentComponent

DCOM

Client Machine

Server Machine

Server MachineServer Machine

Server MachineServer Machine

The problem with stateful apps is the risk of losing state as a result of a failure

6

Transparent PersistenceTransparent Persistence

Programmer simply writes his programProgrammer simply writes his program System ensures persistence– System ensures persistence– HOW?HOW?

Virtual ComponentsVirtual Components Virtual component mapped to physicalVirtual component mapped to physical

Mapping can be changedMapping can be changed Extract and restore state of virtual componentExtract and restore state of virtual component

Into physical componentInto physical component Under system discretion for Under system discretion for scalabilityscalability As As recovery techniquerecovery technique after system crash after system crash

Efficiency byEfficiency by Replaying to rebuild stateReplaying to rebuild state Not by simply frequently saving entire component stateNot by simply frequently saving entire component state Rationale: modest length of execution histories Rationale: modest length of execution histories

7

Important Benefits forImportant Benefits for Application programmerApplication programmer

Program does not have to deal with crashesProgram does not have to deal with crashes Greatly simplifying programGreatly simplifying program

Natural stateful applicationsNatural stateful applications More focus on correctness/functionalityMore focus on correctness/functionality

Operational application Operational application No manual restore of system of application stateNo manual restore of system of application state

Hence shorter outagesHence shorter outages Users/customersUsers/customers

Less lost work, fewer frustrationsLess lost work, fewer frustrations

BusinessBusiness Increased availability means Increased availability means

Less lost revenue (e.g. Ebay lost $1M/hr in revenue)Less lost revenue (e.g. Ebay lost $1M/hr in revenue) Happier customersHappier customers

8

Outline of TalkOutline of Talk BackgroundBackground

Application Availability ProblemApplication Availability Problem Phoenix goal: transparent application persistencePhoenix goal: transparent application persistence

Persistent Stateful ApplicationsPersistent Stateful Applications Fundamentals of multi-component protocolsFundamentals of multi-component protocols

Three forms of componentsThree forms of components Three interaction contractsThree interaction contracts

Two sub-projectsTwo sub-projects Phoenix/COM for persistent componentsPhoenix/COM for persistent components Phoenix/ODBC for transactional componentsPhoenix/ODBC for transactional components

External components need no system supportExternal components need no system support

9

Persistent Stateful Persistent Stateful ApplicationsApplications

For general multi-tier applications

including web/app servers

and database backends

10

Component TypesComponent Types Persistent(Persistent(PCOMPCOM)): : e.g.e.g. web/application serversweb/application servers

State survives system failuresState survives system failures Phoenix/COMPhoenix/COM component acts as PCOM component acts as PCOM

Via logging interactions with other system componentsVia logging interactions with other system components

Transactional(Transactional(TCOMTCOM)): : e.g.e.g. SQL DB sessionsSQL DB sessions Transactions can abort- aborted state may “die”Transactions can abort- aborted state may “die”

But But committedcommitted states, messages survive system failures states, messages survive system failures Phoenix/ODBCPhoenix/ODBC session acts as TCOM session acts as TCOM

Via Via server reply loggingserver reply logging

External(External(XCOMXCOM)): : e.g.e.g. users, autonomous sitesusers, autonomous sites,,etc.etc. Not under our control (Not under our control (need do nothingneed do nothing)) Prompt logging by PCOM limits exposurePrompt logging by PCOM limits exposure

Only failure during interaction is “unmasked”Only failure during interaction is “unmasked”

11

Interaction ContractsInteraction Contracts Committed interaction contractCommitted interaction contract((CICCIC) )

PPCOMCOMPCOMPCOM

Guarantees that interaction persists across failuresGuarantees that interaction persists across failures Transaction’l interaction contractTransaction’l interaction contract ( (TICTIC) )

PPCOMCOMTCOMTCOM Permits transactional component to abortPermits transactional component to abort But final commit and its result msg is persistent But final commit and its result msg is persistent

External interaction contractExternal interaction contract ( (XICXIC) ) PPCOMCOMXCOMXCOM Permits interaction with external world, which does not Permits interaction with external world, which does not

play by our rules play by our rules Immediate forced logging by PCOM to limit failure Immediate forced logging by PCOM to limit failure

windowwindow

12

System SchematicSystem Schematic Replay needs captured at component interactionsReplay needs captured at component interactions

Actions depend on component types involvedActions depend on component types involved Interaction contracts provide persistence guaranteesInteraction contracts provide persistence guarantees

Forms of Components and Interaction ContractsForms of Components and Interaction Contracts Persistent [PCOM]-Persistent [PCOM]- pervasive within system pervasive within system External [XCOM]-External [XCOM]- at system edges (can initiate work) at system edges (can initiate work) Transactional [TCOM]-Transactional [TCOM]- at system edges (receives work) at system edges (receives work)

CICXIC

TICClient

“SQLServer”

AppServer

Web Server

13

Phoenix/COM: Phoenix/COM:

Persistent Components– Persistent Components–

for multi-tier applicationsfor multi-tier applications

TransparentlyTransparently

14

Phoenix/COM ApproachPhoenix/COM Approach

Persistence as a runtime servicePersistence as a runtime service Component Component registeredregistered as persistent [I.e. a PCOM] as persistent [I.e. a PCOM]

Similar to COM+ transaction property registration Similar to COM+ transaction property registration State persistence is transparent and recovery automaticState persistence is transparent and recovery automatic

Phoenix/COM runtime provides Phoenix/COM runtime provides interaction contractsinteraction contracts Programmers write application logic Programmers write application logic onlyonly..

Via application replayVia application replay Piece-wise deterministic assumptionPiece-wise deterministic assumption

Between non-deterministic events, program execution is Between non-deterministic events, program execution is repeatablerepeatable

Non-determinism removed by logging, e.g.Non-determinism removed by logging, e.g. User interactions, Procedure calls, Reading sys clock, etc. User interactions, Procedure calls, Reading sys clock, etc.

Efficiency by minimizing log forces & logged dataEfficiency by minimizing log forces & logged data Via efficient interaction protocols & “logical” log operationsVia efficient interaction protocols & “logical” log operations

15

Recovery InfrastructureRecovery Infrastructure

Phoenix/COM Phoenix/COM

LogLog interactions, checkpoint state IF IF REQUIREDREQUIRED to fulfill CIC requirements

Log

Com Object App 1

Com ObjectApp 2

CIC

App see COM interface

CIC: keeps App’s recoverable

16

Phoenix/COM ElementsPhoenix/COM Elements Virtual ComponentsVirtual Components

Isolate application from physical component failureIsolate application from physical component failure Permit another component to “continue” executionPermit another component to “continue” execution

Normal operation: interception and loggingNormal operation: interception and logging Log activation, call, etc. to capture object stateLog activation, call, etc. to capture object state

Recovery: detect failure & recoverRecovery: detect failure & recover Install virtual object state in new physical objectInstall virtual object state in new physical object Error handler reconnects to recovered virtual objectError handler reconnects to recovered virtual object

Implementation on Implementation on contextcontext infrastructure infrastructure Feature of COM+, .NET CLR, COMAppsFeature of COM+, .NET CLR, COMApps

17

Committed Interaction ContractCommitted Interaction ContractPhoenix/Com ResponsibilityPhoenix/Com Responsibility

Sender guarantees:Sender guarantees: S1: Persistent state at least up to sendS1: Persistent state at least up to send S2: Persistent messageS2: Persistent message

S2A: Repeated transmission retry S2A: Repeated transmission retry S2B: Transmission on requestS2B: Transmission on request

Receiver guarantees:Receiver guarantees: Duplicate message eliminationDuplicate message elimination Persistent state with msg receive: S2A releasedPersistent state with msg receive: S2A released

Subsequently, receiver must request messageSubsequently, receiver must request message Message content stable: S2B releasedMessage content stable: S2B released

Sender can now discard message contentsSender can now discard message contents

18

Infrastructure:Infrastructure: ContextContext Boundary definitionBoundary definition around components around components

Cross-context boundary events are raisedCross-context boundary events are raised Permitting cross context call interceptionPermitting cross context call interception

Every component lives in a contextEvery component lives in a context Subset of an apartmentSubset of an apartment

Context

Component

Context Properties

Context Properties

Contexts specify interception Contexts specify interception

19

Infrastructure:Infrastructure: PolicyPolicy HandlerHandler for context boundary events for context boundary events

Cancel or repeat the callCancel or repeat the call Examine the call state (IID, method, Examine the call state (IID, method,

parameters, etc.) parameters, etc.) Take action as requiredTake action as required

Example: Synchronization PolicyExample: Synchronization Policy onEnter:onEnter: acquire mutex acquire mutex onLeave:onLeave: release mutex release mutex

Policies enable runtime extension… as Policies enable runtime extension… as

interception handlersinterception handlers

20

Context CContext C11 Context CContext C22

ClientSide

Policies

ServerSide Policies

Call,Return

Enter,Leave

Pass Buffer

Pass Buffer

ProxyProxy StubStub

A B

Infrastructure:Infrastructure: SwitcherSwitcher

SW

ITC

HER

The The Interception MechanismInterception Mechanism

21

Interception mechanismInterception mechanism for method for method executionexecution

Detects errors during processingDetects errors during processing

Resolves reference to physical OIDResolves reference to physical OID

Calls a registered error handlerIf R_OK it retries method

call/response

API to rebind “virtual” object To new physical OIDEnhanced switcher detects errors, provides Enhanced switcher detects errors, provides

automatic recovery, and automatic recovery, and virtualizes componentsvirtualizes components……

Role for SwitcherRole for Switcher

22

Infrastructure SummaryInfrastructure SummaryHow Phoenix/COM leverages it…How Phoenix/COM leverages it… ContextContext

Registers interception boundaryRegisters interception boundary

PoliciesPolicies Log Component CreationLog Component Creation Log Method Call/ResponseLog Method Call/Response

SwitcherSwitcher Detect Errors from Component FailureDetect Errors from Component Failure Remap Virtual Component ReferencesRemap Virtual Component References Retry Method Call/ResponseRetry Method Call/Response

25

Phoenix/ODBCPhoenix/ODBC

Transactional components for Database servers

26

Phoenix/ODBC ObjectivesPhoenix/ODBC Objectives Persistent Database SessionsPersistent Database Sessions

Mask network and SQL Server failuresMask network and SQL Server failures Leverage database recovery mechanisms Leverage database recovery mechanisms Recovery is Recovery is transparent to clienttransparent to client Originally:stand-alone session persistenceOriginally:stand-alone session persistence

No protection from client crashesNo protection from client crashes Realize Transactional ComponentsRealize Transactional Components

Within Phoenix/COM Within Phoenix/COM Phoenix/ODBC wrapped DB becomes TCOMPhoenix/ODBC wrapped DB becomes TCOM Client becomes a PCOMClient becomes a PCOM

Component registered as transactional (TCOM)Component registered as transactional (TCOM)

27

ODBC Database SessionODBC Database Session Allocate ODBC handlesAllocate ODBC handles Connect to SQL ServerConnect to SQL Server Set session attributesSet session attributes Send SQL statements to SQL ServerSend SQL statements to SQL Server Fetch records from result setFetch records from result set

Client

SQLServer

Interconnect

SQL

28

Phoenix/ODBC ElementsPhoenix/ODBC Elements Phoenix-enabled ODBC connectionPhoenix-enabled ODBC connection

Virtual session & connection from client to serverVirtual session & connection from client to server

Intercept and Intercept and “wrap”“wrap” Log actions that affect ODBC session contextLog actions that affect ODBC session context

Connection, logon, database and userid, set optionsConnection, logon, database and userid, set options Make temporary tables/cursors on server persistentMake temporary tables/cursors on server persistent Rewrite SQL statement(s) to Rewrite SQL statement(s) to persist SQL resultspersist SQL results Receive server responses and error messagesReceive server responses and error messages Fetch results from persistent result tablesFetch results from persistent result tables

Recover sessionRecover session Remapping virtual ODBC session to new real sessionRemapping virtual ODBC session to new real session

29

Phoenix/ODBC InfrastructurePhoenix/ODBC Infrastructure Currently implemented in ODBC Driver ManagerCurrently implemented in ODBC Driver Manager

Which wraps native ODBC driversWhich wraps native ODBC drivers NoNo changes required for ODBC driver(s) changes required for ODBC driver(s) NoNo changes required for database systems changes required for database systems NoNo special client programming special client programming

SQLServer

Oracle

Informix

Client

SQL SRVR 3.0

Oracle 3.0

SQL SRVR 2.5

Informix 3.0

Phoenix ODBCDriver

Manager TCOMTCOM

30

Transaction Interaction ContractTransaction Interaction Contract Phoenix/ODBC ResponsibilityPhoenix/ODBC Responsibility

Before PCOM issued “Commit”Before PCOM issued “Commit” TCOM can abort, forgetting everythingTCOM can abort, forgetting everything PCOM can forget some/all state/messagesPCOM can forget some/all state/messages

At PCOM issued “Commit”At PCOM issued “Commit” PCOM sender promises are CIC promisesPCOM sender promises are CIC promises CommittingCommitting TCOM receiver guarantees: TCOM receiver guarantees:

Duplicate message eliminationDuplicate message elimination Persistent state with message receive/commit:Persistent state with message receive/commit:

S2A, “S2B” released when TCOM commits & sends replyS2A, “S2B” released when TCOM commits & sends reply PCOM can discard msg contents when reply receivedPCOM can discard msg contents when reply received

Persistent reply messagePersistent reply message A persistent result set committed by transactionA persistent result set committed by transaction

31

Default Result Set (stable reply)Default Result Set (stable reply)

ClientSQL

ServerODBCSELECT … WHERE FALSECREATE TABLE phtableSELECT … INTO phtableSELECT * FROM phtable

Determine format of result setDetermine format of result set

Create persistent table on SQL ServerCreate persistent table on SQL Server

Rewrite SQL stmt to load persistent tableRewrite SQL stmt to load persistent table

Open access to persistent tableOpen access to persistent table

Select cust_name,cust_address from customers

where acct_balance > max_allowed_balance

32

Database Session RecoveryDatabase Session Recovery Intercept errors and detect Intercept errors and detect possiblepossible system failure system failure Ping database server (interval and timeout)Ping database server (interval and timeout) Reconnect to database serverReconnect to database server

Determines if server really failedDetermines if server really failed Verifies persistent structures were recoveredVerifies persistent structures were recovered

Reinstall session context from client logReinstall session context from client log Advance through result set(s) to last result seenAdvance through result set(s) to last result seen

Application still sees aborted transactionsApplication still sees aborted transactions

SQLServer

ODBC Session Context

33

SummarySummary Phoenix goal:Phoenix goal: transparent app persistence transparent app persistence

No application programmingNo application programming Higher availability and scalabilityHigher availability and scalability

PCOMPCOM by Phoenix/COM via by Phoenix/COM via Logging non-deterministic eventsLogging non-deterministic events

By infrastructure via wrappingBy infrastructure via wrapping Component replay if system crashesComponent replay if system crashes

TCOM TCOM by Phoenix/ODBC viaby Phoenix/ODBC via Inducing DBMS logging for database recoveryInducing DBMS logging for database recovery

Via wrapping ODBC and modifying SQL requestsVia wrapping ODBC and modifying SQL requests Inducing DBMS recovery for session stateInducing DBMS recovery for session state

34

Phoenix/COM Project URLPhoenix/COM Project URL

1.1. Barga, Lomet, Weikum:  Recovery Guarantees for General Barga, Lomet, Weikum:  Recovery Guarantees for General Multi-tier Applications. (submitted for publication)  Multi-tier Applications. (submitted for publication) 

2.2. Barga, Lomet: Phoenix/COM: Barga, Lomet: Phoenix/COM: Enabling persistent component-Enabling persistent component-based applications. (Internal white paper)based applications. (Internal white paper)

http://www.research.microsoft.com/research/db/phoenixhttp://www.research.microsoft.com/research/db/phoenix

Phoenix/COM ReferencesPhoenix/COM References

35

Phoenix/ODBC ReferencesPhoenix/ODBC References

1.1. Barga, Lomet:  Measuring and Optimizing a System for Barga, Lomet:  Measuring and Optimizing a System for Persistent Database Sessions. Persistent Database Sessions. Proceedings of ICDE,Proceedings of ICDE, Heidelberg, Germany (April 2001)   Heidelberg, Germany (April 2001)  

2.2. Barga, Lomet, Baby, Agrawal: Persistent Client-Server Barga, Lomet, Baby, Agrawal: Persistent Client-Server Database Sessions. Database Sessions. Proceedings of EDBT Conference, Proceedings of EDBT Conference, Lake Lake Constance, Germany (Mar. 2000) 462-477. Constance, Germany (Mar. 2000) 462-477.

3.3. Barga, Lomet:Barga, Lomet: Phoenix: Making Applications Robust. (demo Phoenix: Making Applications Robust. (demo paper) paper) Proceedings of ACM SIGMOD ConferenceProceedings of ACM SIGMOD Conference, , Philadelphia, PA (June, 1999) 562-564.Philadelphia, PA (June, 1999) 562-564.

4.4. Lomet, Weikum: Efficient Transparent Application Recovery Lomet, Weikum: Efficient Transparent Application Recovery in Client-Server Information Systems. (Best Paper Award) in Client-Server Information Systems. (Best Paper Award) Proceedings of 1998 ACM SIGMOD ConferenceProceedings of 1998 ACM SIGMOD Conference, Seattle, WA , Seattle, WA (June 1998) 460-471. (June 1998) 460-471.