Real-Time Database Systems and Data Services: Issues and Challenges

Sang H. SonDepartment of Computer ScienceUniversity of VirginiaCharlottesville, Virginia 22903son@cs.virginia.edu

OutlineIntroduction: real-time database systems and real-time data servicesWhy real-time databases?Misconceptions about real-time DBS Paradigm comparisonCharacteristics of data and transactions in real-time DBSOrigins of time constraintsTemporal consistency and data freshnessTime constraints of transactionsReal-time transaction processingPriority assignmentScheduling and concurrency controlOverload management and recovery

Outline (contd)Advanced real-time applicationsActive, object-oriented, main-memory databasesFlexible security paradigm for real-time databases Embedded databasesReal-world applications and examplesReal-time database projects and research prototypesBeeHive systemResearch issues, trends, and challengesExercises

I. IntroductionOutline

Motivation: Why real-time databases and data services?A brief review: real-time systems Misconceptions about real-time DBS Comparison of different paradigms:Real-time systems vs real-time database systemConventional DBS vs real-time DBS

Some Facts about Real-Time DatabasesFact 1: As the complexity of real-time systems and application is going up, the amount of information to be handled by real-time systems increases, motivating the need for database and data service functionality (as opposed to ad hoc techniques and internal data structures)

Fact 2: Conventional databases do not support timing and temporal requirements, and their design objectives are not appropriate for real-time applications

Fact 3: Tasks and transactions have both similarities and distinct differences, i.e., traditional task centric view is not plausible to real-time databases.

Real-Time Databases and Services: ExamplesThey are used to monitor and control real-world activitiesTelecommunication systemsrouters and network management systems telephone switching systemsControl systemsautomatic tracking and object positioning engine control in automobilesMultimedia servers for real-time streamingE-commerce and e-businessstock market: program stock tradingfinancial services: e.g., credit card transactionsWeb-based data services

Something to Remember ...

Real-time FAST

Real-time nonosecs or secs

Real-time means explicit or implicit time constraints

A high-performance database which is simply fast without the capability of specifying and enforcing time constraints are not appropriate for real-time applications

A Brief Review: Real-Time SystemsA system whose basic specification and design correctness arguments must include its ability to meet its time constraints.Its correctness depends not only on the logical correctness, but also on the timeliness of its actions.

Review: Real-Time SystemsCharacteristics of real-time systemstimeliness and predictabilitytypically embedded in a large complex systemdependability (reliability) is crucialexplicit timing constraints (soft, firm, hard)A large number of applicationsaerospace and defense systems, nuclear systems, robotics, process control, agile manufacturing, stock exchange, network and traffic management, multimedia computing, and medical systemsRapid growth in research and developmentworkshops, conferences, journals, commercial productsstandards (POSIX, RT-Java, RT-COBRA, )

Time Constraintsdtv(t)v0d2tv(t)v0d1Hard and firm deadline Soft deadline

Databases for Real-Time SystemsCritical in real-time systems (any computing needs correct data)real-time computing needs to access data:real-world applications involve time constrained access to data that may have temporal propertytraditional real-time systems manage data in application-dependent structuresas systems evolve, more complex applications require efficient access to more dataFunction of real-time databasesgathering data from the environment, processing it in the context of information acquired in the past, for providing timely and temporally correct response

What is a Real-Time Database?A real-time database (RTDB) is a data store whose operations execute with predictable response, and with application-acceptable levels of logical and temporal consistency of data, in addition to timely execution of transactions with the ACID properties.

C. D. Locke Chief Scientist, TimeSys Co.

DefinitionsDB - Database DBS - Database SystemRTDB - Real-Time DatabaseTemporal consistency - validity of dataLogical consistency - integrity of dataDurability / Permanence - committed updates must persist Atomicity - no partial results seen or remain (all or nothing)Isolation - each transaction should not be aware of othersSerial execution - inefficient but correct execution of concurrent transactionsSerializability (SR) - interleaved execution equivalent to serial execution: conflict SR and view SR

What is the gain of using RTDBS?More efficient way of handling large amounts of dataSpecification and enforcement of time constraints Improved overall timeliness and predictabilityApplication semantic-based consistency and concurrency controlSpecialized overload management and recoveryExploitation of real-time support from underlying real-time OS Reduced development costs

Gain of Using RTDBS (More Specifically)Presence of a schema - avoid redundant data and its descriptionBuilt-in support for efficient data management - indexing, etcTransaction support - e.g. ACID propertiesData integrity maintenanceBut Not all data in RTDB is durable: need to handle different types of data differently (will be discussed further later)Correctness can be traded for timeliness- Which is more important? Depends on applications, but timeliness is more important in many casesAtomicity can be relaxed: monotonic queries and transactionsIsolation of transactions may not always be neededTemporally-correct serializable schedules serializable schedules

Objectives of Real-Time DatabasesCorrectness requirements:consistency constraintstime constraints on data and transactionsObjectivestimeliness and predictability:dealing with time constraints and violationsPerformance goals:minimize the penalty resulting from actions either delayed or not executed in timemaximize the value accruing to the system from actions completed in timesupport multiple guarantee levels of quality for mixed workloads

Why Not Using Conventional Databases?Inadequacies of conventional databases:poor responsiveness and lack of predictabilityno facility to support for applications to specify and enforce time constraintsdesigned to provide good average response time, while possibly yielding unacceptable worst case execution timeresource management and concurrency control in conventional database systems do not support the timeliness and predictability

Differences from Traditional DatabasesTraditional database systemspersistent data and consistency constraintsefficient access to datatransaction support: ACID propertiescorrect execution of transactions in the context of concurrent execution and failuredesigned to provide good average performanceDatabases for real-time systemstemporal data, modeling a changing environmentresponse time requirements from external worldapplications need temporally coherent viewactively pursue timeliness and predictability

Misconceptions on Real-Time Databases....

Misconceptions about RTDBS (1)Advances in hardware till take care of RTDBS requirements.fast (higher throughput) does not guarantee timing constraintsincrease in size and complexity of databases and hardware will make it more difficult to meet timing constraints or to show such constraints will be methardware alone cannot ensure that transactions will be scheduled properly to meet timing constraints or data is temporally validtransaction that uses obsolete data more quickly is still incorrectReal-time computing is equivalent to fast computing.minimizing average response time vs satisfying individual timing constraintspredictability, not speed, is the foremost goal

Misconceptions about RTDBS (2)Advances in standard DBS technology will take care of RTDB requirements.while novel techniques for query processing, buffering, and commit protocols would help, they cannot guarantee timeliness and temporal validitytime-cognizant protocols for concurrency control, commit processing and transaction processing are mandatoryThere is no need for RTDBS because we can solve all the problems with current database systemsadding features such as validity intervals and transaction deadlines to current database systems is in fact moving towards to developing a real-time database systemsuch approach (adding features in ad hoc manner) will be less efficient than developing one from the ground up with such capabilities

Misconceptions about RTDBS (3)Using a conventional DBS and placing the DB in main memory is sufficient.although main-memory resident database eliminate disk delays, conventional databases have many sources of unpredictability, such as delays due to blocking on locks and transaction schedulingincreases in performance cannot completely make up for the lack of time-cognizant protocols in conventional database systemsA temporal database is a RTDB.while both of temporal DB and RTDB support time-specific data operations, they support different aspects of timein RTDB, timely execution is of primary concern, while in temporal DB, fairness, resource utilization, and ACID properties of transactions are more important

Misconceptions about RTDBS (4)Problems in RTDBS will be solved in other areas.some techniques developed in other areas (e.g., RTS and DBS) cannot be applied directly, due to the differences between tasks and transactions, and differences in correctness requirementsthere are unique problems in RTDBS (e.g., maintaining temporal consistency of data)RTDBS guarantee is meaningless unless H/W and S/W never failstrue, in part, due to the complexity involved in predictable and timely executionit does not justify the designer not to reduce the odds of failure in meeting critical timing constraints

Reference: Stankovic, Son, and Hansson, Misconceptions About Real-Time Databases, IEEE Computer, June 1999.

Comparisons of Different Paradigms ...

Notion of TransactionTransaction

partially ordered set of database operationsa complete and consistent computation (i.e., they are designed to terminate correctly, leaving the database in a consistent state)units of user activity and system recoveryhave dynamic runtime behavior (dependent on the state of the database, i.e., data values)data is a resource (transaction can be blocked in accessing data objects)preemption may lead to abort

Conventional vs. Real-Time TransactionsConventional TransactionsLogically correct and consistent (ACID):atomicityconsistency isolationdurability

Real-Time TransactionsLogically correct and consistent (ACID)Approximately correcttrade quality or correctness for timelinessTime correctnesstime constraints on transactionstemporal constraints on data

Conventional vs. Real-Time Databases: Correctness CriteriaConventional Databases:Logical consistencyACID properties of transactions:Atomicity Isolation Consistency DurabilityData integrity constraints

Real-Time Database Systems:Logical consistency ACID properties (may be relaxed)Data integrity constraints Enforce time constraintsDeadlines of transaction External consistencyabsolute validity interval (AVI)Temporal consistencyrelative validity interval (RVI)

Real-time Systems vs. RTDBSReal-time systemsTask centricDeadlines attached to tasks

Real-time databasesData centricData has temporal validity, i.e., deadlines also attached to dataTransactions must be executed by deadline to keep the data valid, in addition to produce results in a timely manner

II. Characteristics of Data and TransactionsOutline

The origin of time constraintsTypes of time constraintsReal-time data and temporal consistencyReal-time transactions

The Origin of Time ConstraintsMeeting time constraints is of paramount importance in real-time database systems. Unfortunately, many of these time constraints are artifacts.If a real-time database system attempts to satisfy them all, it may lead to an over-constrained or over-designed system.

Issues to be discussed:1. What are the origins of (the semantics of) time constraints of the data, events, and actions?2. Can we do better by knowing the origins of time constraints?3. What is the connection between time-constrained events, data, and real-time transactions?

Example #1: Objects on Conveyor Belts on a Factory FloorRecognizing and directing objects moving along a set of conveyer belts on a factory floor.

Objects features captured by a camera to determine its characteristics.

Depending on the observed features, the object is directed to the appropriate workcell.

System updates its database with information about the object.

Example #1 (contd)Features of an object must be collected while the object is still in front of the camera.Current object and features apply just to the object in front of the cameraLose validity once a different object enters the system.Objects features matched against models in database.Based on match, object directed to selected workcell.Alternative: discard object and later bring it back again in front of the camera.

Example #2: Air Traffic ControlSystem makes decisions concerning

incoming aircrafts flight paththe order in which they should landseparation between landings

Parameters: position, speed, remaining fuel, altitude, type of aircrafts and current wind velocity.

Aircraft allowed to land => subsequent actions of this aircraft become critical: cannot violate time constraints

Alternative: Ask aircraft to assume a holding pattern.

Factors that Determine Time ConstraintsFocus: externally-imposed temporal propertiesThe characteristics of the physical systems being monitored and controlled:speed of the aircraft, speed of conveyer belt, temperature and pressureThe stability characteristics as governed by its control laws:servo control loops of robot hands, fly-by-wire, avionics, fuel injection rateQuality of service requirements:sampling rates for audio and video, accuracy requirement for resultsHuman (re)action times, human sensory perception:time between warning and reaction to warning

Events, data and actions inherit time constraints from these factorsThey determine the semantics (importance, strictness) of time constraints.

All Time Constraints are Artifacts?May be not all of them, but even many externally-imposed constraints are artifacts:

Length of a runway or speed of an aircraft - determined by cost and technology considerations;

Quality of service requirements - decided by regulatory authorities;

Response times guaranteed by service providers - determined by cost and competitiveness factors

Designer ArtifactsSubsequent decisions of the database system designer introduce additional constraints:The type of computing platform used (e.g. centralized vs. distributed)The type of software design methodology used (e.g., data-centric vs. action-centric)The (pre-existing) subsystems used in composing the systemThe nature of the actions (e.g., monolithic action vs. graph-structured or triggered action)

Time constraints reflect the specific design strategy and the subsystems chosen as much as the externally imposed timing requirements

Decisions on Time ConstraintsDifficulty of optimal time constraintsDetermining all related time constraints in an optimal fashion for non-trivial systems is intractable => divide and conquer (and live with acceptable decisions)Multi-layer decision processThe decisions made at one level affect those at the other level(s)While no decision at any level is likely to be unchangeable, cost and time considerations will often prevent overhaul of prior decisions

Decisions on Time Constraints (2)Decisions to be madeWhether an action is periodic, sporadic, or aperiodicThe right values for the periods, deadlines, and offsets within periodsImportance or criticality valuesFlexibility (dynamic adaptability) of time constraints

Time Constraints of EventsThree basic types of time constraints1. Maximum: delay between two eventsExample: Once an object enters the view of the camera, object recognition must be completed within t1 seconds2. Minimum: delay between two eventsExample: No two flight landings must occur within t2 seconds3. Durational: length of an eventExample: The aircraft must experience no turbulence for at least t3 seconds before the seat-belt sign can be switched off once againConstraints can specify between stimulus and response events (max, min, and duration between them can be stated)

Time Constraints of Events (2)The maximum and minimum type of time constraints of recurring (stimulus) events: rate-based constraintsTime constraints determine the constraints on transactions:Rate-based constraints -> periodicity requirements for the corresponding actionsTime constraints relating a stimulus and its response -> deadline constraintsSpecifications of minimal separation between response to a stimulus and the next stimulus -> property of the sporadic activity that deals with that stimulus

Data in Real-Time Database SystemsData items reflect the state of the environmentData from sensors - e.g., temperature and pressureDerived data - e.g., rate of reactionInput to actuators - e.g., amount of chemicals, coolantArchival data - e.g., history of (interactions with) environmentStatic data as in conventional database systems

Time Constraints on DataWhere do they come from?state of the world as perceived by the controlling system must be consistent with the actual stateRequirementstimely monitoring of the environmenttimely processing of sensed informationtimely derivation of needed dataTemporal consistency of dataabsolute consistency: freshness of data between actual state and its representationrelative consistency: correlation among data accessed by a transaction

Representation of Temporal ConsistencyAbsolute consistency: Absolute Validity Intervals (AVI)state of environment and reflection in DBd : (value, avi, ts) value = current state of d ts = time of observation avi = absolute validity interval (current_time - ts) aviRelative consistency: Relative Validity Intervals (RVI) among data used to derive other data " d D , ts(d) - ts(d) Drvi d D, D: relative consistency set for a transaction

Static Data and Real-Time DataStatic datadata in a typical databasevalues not becoming obsolete as time passesReal-time (Temporal) dataarrive from continuously changing environmentrepresent the state at the time of sensinghas observed time and validity intervalusers of temporal data need to see temporally coherent views of the data (state of the world)When must the data be temporally consistent?ideally, at all timesin practice, only when they are used by transactions

An ExampleData object is specified by(value, absolute validity interval, time-stamp)Interested in {temperature and pressure}with relative validity interval of 5Let current time = 100

temperature = (347, 10, 95) and pressure = (50, 20, 98)-- temporally consistent

temperature = (347, 10, 98) and pressure = (50, 20, 91)-- temporally inconsistent

What Makes the Difference?We have a set of predicates to be satisfied by dataWhy not use standard integrity maintenance techniques?Not executing a transaction will maintain logical consistency, but temporal consistency will be violatedSatisfy logical consistency by CC techniques, such as 2PLSatisfy temporal consistency by time-cognizant transaction processingAVI and RVI may change with system dynamics, e.g. mode changes

Time Constraints Associated with ActionsTime constraints dictate the behavior of the environment constrain the rates and times at which inputs arrive at the systemExample: seek permission to land only when aircraft is 10 mins from airportTime constraints prescribe performance of the systemdictate the responsiveness of the system to these inputsExample: respond to a landing request within 30 secondsTime constraints are imposed to maintain data temporal consistencyExample: actions that update an aircrafts dynamic parameters in 1 second

Distinct Types of Transactions Write-only transactions (sensor updates): obtain state of the environment and write into the databasestore sensor data in database (e.g., temperature)monitoring of environmentensure absolute temporal consistencyUpdate transactions (application updates)derive new data and store in databasebased on sensor and other derived dataRead-only transactionsread data, compute, and report (or send to actuators)

Time Constraints on TransactionsTime constraints on transactionssome come from the need to maintain temporal consistency of datasome come from the requirements on reaction time, dictating the responsiveness of the systemsome come from the designers choice, specifying the rates and times at which inputs arrive at the systemtransactions value depends on completion time

Types of Time ConstraintsBased on type of time constraints:Periodic- Every 10 secs Sample wind velocity- Every 20 secs Update robot positionAperiodic- If temperature > 1000 within 10 secs add coolant to reactorBased on Value:Hard: must execute before deadlineFirm: abort if not completed by deadlineSoft: diminished value if completed after deadline

Dealing with Time Constraint ViolationsLarge negative penalty => a safety-critical or hard time constrainttypically arise from external considerationsimportant to minimize the number of such constraints

No value after the deadline and no penalty accrues => a firm deadlinetypically, alternatives exist

Result useful even after deadline => a soft deadlinesystem must reassign successors parameters - so that the overall end-to-end time constraints are satisfied

Firm and soft time constraints offer the system flexibility - not present with hard or safety-critical time constraints

Examples of Time Constraints Specified using ECA (Event-Condition-Action) RulesThe time constraints can be specified using ECA rules

ON (10 seconds after initiating landing preparations)IF (steps not completed)DO (within 5 seconds abort landing)

ON (deadline of object recognition)IF (action not completed)DO (increase importance, adjust deadlines)

ON (n-th time violation within 10 secs)IF (crisis-mode)DO (drop all non-essential transactions)

Time Constraints: DiscussionUnderstand the issues underlying the origin and semantics of time constraintsnot all deadlines are given.need ways to deriving time constraints (and semantics) in the least stringent mannerflexibility afforded by derived deadlines must be exploiteddeadline violation must also be handled adaptively

Control strategies can be specified by ECA rules

III. Real-Time Transaction Processing

Outline

Priority assignment Scheduling paradigms Priority inversion problem Concurrency control protocols Predictability issues Overload management and recovery

Priority AssignmentDifferent approachesEDF: earliest deadline firsthighest value (benefit) firsthighest (value/computation time) firstcomplex function of deadline, value, slack timePriority assignment has significant impact on database system performanceAssignment based on deadline and value has shown good performance

Non-Real-Time SchedulingLevel: operating systems, database systems, etc.Primary Goal: maximize performanceSecondary Goal: ensure fairness Typical metrics: minimize response timemaximize throughpute.g., FCFS (First-Come-First-Served), RR (Round-Robin), fair-share scheduling

Goals of Real-Time Transaction SchedulingMaximize the number of transactions (both sensor and user) that meet deadlinesKeep data temporally valid on overload, allow invalid intervals on data (note that data with invalid interval may not be used during that invalid time)overload management by trading off quality for timeliness and schedule contingency (or alternative) versions of transactions more on overload management later ...

Execution Time of Transactionstexec = tdb + tI/O + tint + tappl + tcomm

tdb = processing of DB operations (variable)tI/O = I/O processing (variable)tint = transaction interference (variable)tappl = non-DB application processing (variable & optional)tcomm = communication time (variable & optional)

Scheduling ParadigmsScheduling analysis or feasibility checking of real-time computations can predict whether timing constraints will be met

Several scheduling paradigms emerge, depending on whether a system performs schedulability analysisif it does, whether it is done statically or dynamically, andwhether the result of the analysis itself produces a schedule or plan according to which computations are dispatched at run-time

Different Paradigms1. Static Table-Driven approaches:Perform static schedulability analysisThe resulting schedule is used at run-time to decide when a computation must begin execution2. Static Priority Driven Preemptive Approaches:Perform static schedulability analysis but unlike in the previous approach, no explicit schedule is constructedAt run-time, computations are executed (typically) highest-priority- firstExample: rate-monotonic priority assignment - priority is assigned proportional to frequency

Different Paradigms (2)3. Dynamic Planning Based Approaches:Feasibility is checked at run-time, i.e. a dynamically arriving computation is accepted for execution only if it found feasible (that is, guaranteed to meet its time constraints)One of the results of the feasibility analysis is a schedule or plan that is used to decide when a computation can begin execution.4. Dynamic Best-effort Approaches:No feasibility checking is doneThe system tries to do its best to meet deadlines, but since no guarantees are provides, a computation may be aborted during its execution

Dealing with Hard DeadlinesAll transactions have to meet the timing constraintsbest-effort is not enougha kind of guarantee is requiredRequires periodic transactions onlyresource requirements known a priori worst-case execution time of transactions are knownUse static table-driven or priority-driven approachschedulability analysis is necessaryrun-time support also necessary

Dealing with Soft/Firm DeadlinesTwo critical functions: assign transaction prioritiesresolve inter-transaction conflicts using transaction parameters: deadline, criticality, slack time, etc.For firm deadlines, abort expired transactionsFor soft deadlines, the transaction is continued to finish in general, even if the deadline is missedVarious time-cognizant concurrency controls developed, many of which are extensions of two-phase locking (2PL), timestamp, and optimistic concurrency control protocols

Time-cognizant Transaction Scheduling

Earliest deadline first (EDF)Highest value firstHighest value density first (value per unit computation time)Weighted formula: complex function of deadline, value, and remaining work, etc.Earliest Data Deadline First: considering the validity intervalExample: DD(Y) is used as the virtual deadline of transaction T ActivateTR TBeginTR TRead XRead YDeadline of TR TDD(X)DD(Y)

Example 1 : Commit CaseActivateTR TBeginTR TRead XRead YDeadlineof TR TCommitX and Y are validTR T makes deadlineDD(X)DD(Y)DD = Data deadline

Example 2 : Abort CaseActivateTR TBeginTR TRead XRead YDeadlineof TR TDD(Y) DD(X)ABORT

Example 3 : Forced WaitActivateTR TBeginTR TRead XRead YDeadlineof TR TDD(X)DD(Y)Force TR T to Wait for Update to Y since it will occur soon!

Example 4 : With Data SimilarityActivateTR TBeginTR TRead XRead Y15.70Deadlineof TR TDD(X)CommitDeadline of TR T is metData X is OKData Y is similar (defined in DB) DD(Y) - Y updated to 15.78

Transactions: Concurrency ControlPessimisticOptimistic (OCC)Hybrid (e.g., integrated real-time locking)SpeculativeSemantic-based Priority ceiling

Pessimistic Concurrency ControlLocks are used to synchronize concurrent actionsTwo-Phase Locking (2PL)all locking operations precedes the first unlock operation in the transactionexpanding phase (locks are acquired)shrinking phase (locks are released)suffers from deadlockpriority inversion

Example of 2PL: Two transactionsT1:write_lock (X); read_object (X);X = X + 1;write_object (X);unlock (X);

Priority T1 > Priority of T2T2:read_lock (X);read_object (X);write_lock (Y);unlock (X);read_object (Y);Y = X + Y;write_object (Y);unlock (Y);

Example of 2PL: DeadlockT1:

read_lock (X);read_object (X);write_lock (Y); [blocked]::=> DEADLOCK !T2:read_lock (Y);read_object (Y);

write_lock (X); [blocked]::

Conflict Resolution in 2PL2PL (or any other locking schemes) relies on blocking requesting transaction if the data is already locked in an incompatible mode. What if a high priority transaction needs a lock held by a low priority transaction? Possibilities are ...let the high priority transaction waitabort the low priority transactionlet low priority transaction inherit the high priority and continue executionThe first approach will result in a situation called priority inversionSeveral conflict resolution techniques are available, but the one that use both deadline and value show better performance

Priority Inversion Problem in Locking ProtocolsWhat is priority inversion?A low priority transaction forces a higher priority transaction to waithighly undesirable in real-time applicationsunbounded delay may result due to chained blocking and intermediate blocking:

Example: T0 is blocked by T3 for accessing data object, then T3 is blocked by T2 (priority T0 > T2 > T3)

Example of 2PL: Priority InversionT1: write_lock (X); [blocked] read_object (X); X = X + 1; write_object (X); unlock (X);T2:read_lock (X); read_object (X); write_lock (Y); unlock (X); read_object (Y); Y = X + Y; write_object (Y); unlock (Y);Priority inversion

Solutions to Priority Inversion ProblemPriority abortabort the low priority transaction - no blocking at allquick resolution, but wasted resources Priority inheritance execute the blocking transaction (low priority) with the priority of the blocked transaction (high priority)intermediate blocking is eliminatedConditional priority inheritancebased on the estimated length of transactioninherit the priority only if blocking one is close to completion; abort it, otherwise

Conditional Priority Inheritance ProtocolTi requests data object locked by Tjif Priority (Ti) < Priority (Tj)then block Tielse if (remaining portion of Tj > threshold)abort TjelseTi waits while Tj inherit the priority of Ti to execute

Why Conditional Priority Inheritance? Potential problems of (blind) priority inheritance:life-long blocking - a transaction may hold a lock during its entire execution (e.g., strict 2PL case)a transaction with low priority may inherit the high priority early in its execution and block all the other transactions with priority higher that its original priorityespecially severe if low priority transactions are longConditional priority inheritance is a trade-off between priority inheritance and priority abortNot sensitive to the accuracy of the estimation of the transaction length

Performance Results Priority inheritance does reduce blocking times. However, it is inappropriate under strict 2PL due to life-time blocking of the high priority transaction. It performs even worse than simple waiting when data contention is highPriority abort is sensitive to the level of data contentionConditional priority inheritance is better than priority abort when data contention becomes high Blocking is a more serious problem than resource waste, especially when deadlines are not tight In general priority abort and conditional priority inheritance are better than simple waiting and priority inheritanceDeadlock detection and restart policies appear to have little impact

Optimistic Concurrency ControlNo checking of data conflicts during transaction execution read phase: read values from DB; updates made to local copiesvalidation phasebackward validation or forward validationconflict resolutionwrite phase:if validation ok then local copies are written to the DBotherwise discard updates and (re)start transaction Non-blockingDeadlock freeSeveral conflict resolution policies

OCC: Validation phaseIf a transaction Ti should be serialized before a transaction Tj, then two conditions must be satisfied:Read/Write ruleData items to be written by Ti should not have already been read by TjWrite/Write ruleTis should not overwrite Tjs writes

OCC ExampleT1:

read_object (X); X = X + 1; write_object (X);validation

T3:read_object (Y); Y = Y + 1; write_object (Y);...T2:read_object (X); read_object (Y); Y = X + Y; write_object (Y);validation

OCC: Conflict ResolutionWhen a transaction T is ready to commit, any higher-priority conflicting transaction is included in the set H

Broadcasting commit (no priority consideration)T always commits and all conflicting transactions are abortedWith priority consideration: if H is non-empty, 3 choicessacrifice policy: T is always abortedwait policy: T waits until transactions in H commits; if they do commit, T is abortedwait-X policy: T commits unless more than X% of conflicting transactions belong to H

OCC: ComparisonBroadcasting commit (no priority consideration)not effective in real-time databasesSacrifice policy: wasteful theres no guarantee the a transaction in H will actually commit; if all in H abort, T is aborted for nothingWait policy: address the above problemif commit after waiting, it aborts lower priority transactions after waiting, which may have not enough time to restart and committhe longer T stays, the higher the probability of conflictsWait-X policy: compromise between sacrifice and waitX=O: sacrifice policy; X=100: wait policyperformance study shows X=50 gives the best results

Priority Ceiling Protocol Why?to provide blocking at most once propertythe system can compute (pre-analyze) the worst case blocking time of a transaction, and thus schedulability analysis for a set of transaction is feasible A complete knowledge of data and real-rime transactions necessary: for each data object, all the transactions that might access it need to be knowntrue in certain applications (hard real-time applications)not applicable to other general applications

Priority Ceiling Protocol For each data object O: write-priority ceiling: the priority of the highest priority transaction that may write Oabsolute priority ceiling: the priority of the highest priority transaction that may read or write Or/w priority ceiling: dynamically determined priority which equals absolute priority ceiling if O is write-locked; equals write priority ceiling if O is read lockedCeiling rule: transaction cannot lock a data object unless its priority is higher that the current highest r/w priority ceiling locked by other transactionsInheritance rule: low priority transaction inherits the higher priority from the ones it blocksGood predictability but high overhead

External consistency: Data being stored represents the the state of real-world artifacts at any point in time. (if the database contains an aircraft velocity of 825 km/h, then the actual velocity is 825+-1 km/h.Temporal consistency: Each data element represents the accruate state of the real-world artifacts at a point in time acceptably close to the time at which other related data elements are known to be accurate (e.g., position is determined by both latitude and longitude)Logical consistency: No data elements are visisble to an aapplication unless all ligically dependent data elements are also visible from the same transaction (e.g., if a data element shows a simualted aircraft destroyed, another data element does not describe a subsequent attack of that aircraft)Permanence: When a data element is changed, that data change will continue to be visible until it is changed again, or until no possible references to it will be made in the context of the system mission.

Misconceptions: Real-time systems people: cant use schemas, disks, etc. Traditional database people: can solve with commercial DB (just make it fast)

Misconceptions: Real-time systems people: cant use schemas, disks, etc. Traditional database people: can solve with commercial DB (just make it fast)

Misconceptions: Real-time systems people: cant use schemas, disks, etc. Traditional database people: can solve with commercial DB (just make it fast)

Misconceptions: Real-time systems people: cant use schemas, disks, etc. Traditional database people: can solve with commercial DB (just make it fast)

Tasks and transactions abstractions are units of work, and therefore also units of design, i.e., implements a well understood function or performs a well understood serviceTasks and transactions are both threads of control, i.e., they are the units of schedulingMany tasks are designed to be non-terminating. (All transactions are designed to terminate.The atomic nature of a transactions requires that the effects of any failed transaction not be visible to any other transaction. Hence, concurrently executing transactions must be isolated from each other; correct interleavings of transaction operations are serializable. Concurrently executing tasks are cooperative and inter-communicative; they are explicitly aware of each other, and as they request services for each other, are explicitly not serializable.Run-time behavior of tasks are statistically predictable. Transactions not When disk-resident data is involved, worst case estimates of transaction times may be wildly pessimistic.Tasks wait only for other tasks; transactions also wait for the completion of I/O events. Resources accessed by more than one task in a RTS are few in number and generally known at design time. They take the form of critical regions or rendez-vouz containing procedures for updating shared data.The resource (database data granules) which may be accessed by multiple transactions are very many in number However, the database is much larger than its active, in-use subset, at any moment. There is generally a very low probability of conflict between transactions. a very low probability that any data granule will be needed by two concurrently executing transactions. . amount of potential conflict is high, ethe amount of actual conflict is low. Hot spots etc..One of the main challenges in applying real-time technology (e.g. scheduling) to DB systems is that resources needed by a transaction may not be known a priori. A priori knowledge of transaction requirements is important for building a predictable system, even more so for meeting hard deadlines. Possessing complete knowledge of transaction requirements reduces resource management problems (e.g., concurrency control, memory and buffer management) to scheduling problems.Set of data objects to be read/written by a transaction may be dependent on user input, sensory inputs, or values in the DB. Hence, a priori reservation of resources (e.g., locks) to guarantee a particular WCET becomes hard/impossible..Temporal correctness:Transaction reads temporally consistent sets of data, i.e., within absolute validity interval for each data item, and meet relative consistency constraints among data items.Temporal constraints on data is referred to as external consistency.

Conventional DatabasesData are permanentMaximize throughputOften on disk, designed to minimize the number of disk accesses (not necessarily appropriate for RT)

RTDBSAll data are not permanent; some data is temporal Maximize completion ratioEnsure logical consistencyJAS: Maps are stored in Pascal memory structures. Is that a reason why it crashed I dont knowOSE Delta and ENEA

Integrated Vehicle Systems ControlAutonomous nodes

controlling individual subsystemshandling large amounts of data locally- under hard timing constraints(e.g., fuel injection, ignition control)requiring parameters from other subsystems- much less critical time scale(e.g., from the transmission, environment sensors).Similar features in

automated manufacturing power distribution control time-constrained distributed naming services

It is imperative that the state of the environment, as perceived by the controlling system, be consistent with the actual state of the environment. Hence, timely monitoring of the environment as well as the timely processing of the sensed information is necessary.Sensed data is processed further to derive new data, e.g., temperature progression (includes data access from archival storage)In Backward validation, the set of items read by the transaction attempting to commit is examined for any item written by a transaction that committed after the committing transaction started. Since the committed transactions are serialized before the committing transaction, the committing transaction should have seen the new value. Since writes are deferred until commit in optimistic protocols, that may not have happened. Thus when such an item exists, the committing transaction is abortedIn Forward validation, the set of items written by the committing transaction is compared to the sets of items read by any currently active transactions. If there are any nonempty intersections, then a serialization error will occur if all the transactions are committed. In this case however, it is possible to choose to abort either the committing transaction or all of the transactions with which it conflicts. [Graham, 1992].R-W and W-W rule: (Huang and Stankovic, 1991)Any of the following conditions should hold:1) Ti completes its execution before Tj started (i.e., no interleaving)2) The write set of Ti does not intersect with the read set of Tj (thus enforcing the R/W rule), and Ti completes its write phase before Tj starts its validation phase (this enforces the W/W rule).

In Backward validation, the set of items read by the transaction attempting to commit is examined for any item written by a transaction that committed after the committing transaction started. Since the committed transactions are serialized before the committing transaction, the committing transaction should have seen the new value. Since writes are deferred until commit in optimistic protocols, that may not have happened. Thus when such an item exists, the committing transaction is abortedIn Forward validation, the set of items written by the committing transaction is compared to the sets of items read by any currently active transactions. If there are any nonempty intersections, then a serialization error will occur if all the transactions are committed. In this case however, it is possible to choose to abort either the committing transaction or all of the transactions with which it conflicts. [Graham, 1992].In Backward validation, the set of items read by the transaction attempting to commit is examined for any item written by a transaction that committed after the committing transaction started. Since the committed transactions are serialized before the committing transaction, the committing transaction should have seen the new value. Since writes are deferred until commit in optimistic protocols, that may not have happened. Thus when such an item exists, the committing transaction is abortedIn Forward validation, the set of items written by the committing transaction is compared to the sets of items read by any currently active transactions. If there are any nonempty intersections, then a serialization error will occur if all the transactions are committed. In this case however, it is possible to choose to abort either the committing transaction or all of the transactions with which it conflicts. [Graham, 1992].Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.Deadlock resolution techniques:- Abort a transaction that already passed its deadline.- Abort a transaction with the longest deadline- Abort a transaction that is least critical

Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.In Backward validation, the set of items read by the transaction attempting to commit is examined for any item written by a transaction that committed after the committing transaction started. Since the committed transactions are serialized before the committing transaction, the committing transaction should have seen the new value. Since writes are deferred until commit in optimistic protocols, that may not have happened. Thus when such an item exists, the committing transaction is abortedIn Forward validation, the set of items written by the committing transaction is compared to the sets of items read by any currently active transactions. If there are any nonempty intersections, then a serialization error will occur if all the transactions are committed. In this case however, it is possible to choose to abort either the committing transaction or all of the transactions with which it conflicts. [Graham, 1992].R-W and W-W rule: (Huang and Stankovic, 1991)Any of the following conditions should hold:1) Ti completes its execution before Tj started (i.e., no interleaving)2) The write set of Ti does not intersect with the read set of Tj (thus enforcing the R/W rule), and Ti completes its write phase before Tj starts its validation phase (this enforces the W/W rule).

Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.Various CC algorithms differ in what time conflicts are detected and in the way they are resolved:Pessimistic CC protocols detect conflicts as soon as they occur, and resolve them using blocking.Optimistic CC protocols detect conflicts at transaction commit time, and resolve them using rollbacks.Most RT CC schemes could be be viewed as extensions of either PCC- or OCC-based protocols.Hybrid (Kim and Srivastava): the decision regarding the exact serialization order of transactions is delayed as much as possible to allow urgent transactions to commit. This is done through the dynamic allocation and adjustment of timestamp intervals.Speculative (Bestavros and Braoudakis):a transaction is allowed to use more resources so as to achieve better speculation and hence improve its chances for timely commitment.

While the priority ceiling protocol is very useful in providing a bound on transaction blocking delay and schedulability analysis, it suffers from unnecessary blockings due to the conservative transaction scheduling in data access. Recently, several approaches are being proposed to improve the performance by applying the notion of dynamic adjustment of serialization order, avoiding unnecessary blockings.