Lecture – Final Review of...
Transcript of Lecture – Final Review of...
December 3, 2014 Sam Siewert
CS317 - File and Database Systems
Lecture – Final Review of Concepts
http://www.thedailybeast.com/articles/2013/03/22/less-is-moo-the-genius-of-gary-larson.html
FIRST HALFWeek 1-6, Week 7 Mid-Term
Sam Siewert 2
Week 1-6 – As Describe in SyllabusNotes on File Systems Presented Week-1
– Meta-data, Namespace, I-nodes
Connolly-Begg Chapters 1 to 13– Part-I - DBMS Background
Chapters 1, 2, 3 – Background on DBMS, History P. 26Chapter 3 – Architecture (ANSI SPARC, P. 36-37)
– Part-II - Relational Model, E.F. Codd, SQL DML/DDLChapters 4, 5 – Relational Model, Algebra and Calculus
– Relational Algebra (P. 132)– Tuple Relational Calculus (P. 133 to 136)– NO DOMAIN RELATIONAL CALCULUS
Chapters 6, 7, 8 – Core SQL, SQL/PSMChapter 9 – SQL:2011 ORDBMS Extensions
– OOA, OOD, OOP and OODBMS Alternative (Persistence of Instantiated Objects from Classes)
– OODBMS Discussed in concept, but deferred until Chapter 27
– Part-III - Database Analysis and DesignChapter 10 – DBMS Development LifecycleChapter 11 – DreamHome Case Study (Assignment #3), P. 112Chapter 12 – Basic ER Models (Chen’s Diagrams)Chapter 13 – Enhanced ER Models
Sam Siewert 3
Study SuggestionsReview Mid-Term
Scan Book Chapters
Read Through Notes and Study any Unfamiliar Concepts
Sam Siewert 4
NORMALIZATIONWeek 8
Sam Siewert 5
Normalization – Week 8Most Starting Points for RDBMS Design include UNF Data (e.g. Spreadsheets) – Duplication in Columns!
We studied Process to go from UNF to 3NF
We studied Residual Issues with 3NF and BCNF as Potential Solution
Goal in UNF->3NF->BCNF Normalization is to Eliminate Insert, Delete and Modify Hazards while Preserving Loss-less Join and Dependencies
Tends to Produce More Tables, Simpler, with FKs!
Sam Siewert 6
7
The Process of Normalization
Ignore Higher Forms
8
The Process of Normalization
Preview of UNF -> 3NFMinimizes Update Anomalies [Insert, Update, Delete], Page 420 to 426 – One Client Renting Multiple Properties – Typical of Spreadsheet
Sam Siewert 9
cNo cName pNo pAddr rentStart rentFinish Rent oNo oName
CR76 John Kay PG4,
PG16
6 Lawrence Street,5 Novar Drive
7/12,
9/13
8/31,
9/14
350,
50
CO40,
CO93
Tina Murphy,
Tony Shaw
…
Client
PropertyOwner
PropertyForRent OwnerRental
CONCEPTUAL / LOGICAL DBMS DESIGN
Week 9
Sam Siewert 10
Conceptual DBMS Design - Week 9MySQL Workbench CASE Tool
ER/EER Use to Specify an RDBMS Schema
Forward and Reverse Engineering of DB Schemas
Goals and Concepts for Conceptual and Logical Design from Chapter 16 & 17
General Familiarity with MySQL Workbench Major Features and Methods of Use
Sam Siewert 11
PHYSICAL DBMS DESIGNWeek 10-11
Sam Siewert 12
DBMS Logical and Physical Design –Week 10/11
Chapter 16 & 17 – Conceptual DBMS Design Methods Mapping onto Physical
Physical Design– Indexing – B-Tree, B+ Tree, Hash Tables– Storage Engines – MyISAM, InnoDB (File or Partition),
MEMORY– RAID – RAID 1, RAID 0, RAID 5; RAID 50, RAID 10 (Ignore
other RAID levels)
Differences between B-Tree and B+ Tree
Value of R-Tree Sam Siewert 13
DBMS SECURITYWeek 11-12
Sam Siewert 14
DBMS Security - Week 11/12Authorization AuthenticationOne-Way HashSymmetric EncryptionPublic Key EncryptionSigning (E.g. Driver Signing)Networked DBMS Client to DBMS Server
Threats – What are they, how Significant to DBMS
Basic Encryption Concepts and Methods
RAID and RTO/RPO Disaster Recovery Concepts Sam Siewert 15
DBMS TRANSACTIONSWeek 12
Sam Siewert 16
DBMS Transactions – Week 12Recall That Many DBMS Clients can Issue Many Transaction Requests– Thread Per Connection– Thread Created for Each Transaction Request– ACID Goals (Atomicity, Consistency, Isolation, Durability)
Two Solutions in Use Today – Strict 2PL (Locks with Automatic Scoping), Time-stamp Request Queue– Similar Issues in OS, but Not Exactly the Same [DBMS - Lost
Update, Uncommitted Dependency, Inconsistent Analysis]– Similar Solutions in OS, but Not Exactly the Same (E.g.
Semaphores and Locks)
Alternatives for Scheduling Reads/Writes encapsulated in Transactions are Challenging (NP-Hard) and/or Insufficient– Conflict Serialization – Insufficient to Solve all 3 Problems– View Serialization – NP Hard Algorithm, Not Practical
Sam Siewert 17
Comparison of Methods
Pearson Education © 2014 18
Orders conflicting operationsAs “some” serial schedule
Including those with data corruptionAnd inconsistent DB states
Less Restrictive Operation OrderingAs “some” serial schedule[Impractical – NP-Complete]
DBMS Transaction SummaryConcurrency due to Multiple Clients and Transactions Mapped to OS Threads [Readers, Writers]
Causes Problems with Split Transactions– Lost Update– Dirty Read or Uncommitted Dependency– Fuzzy Read, Phantom Read
Serialization Using Locks – Not Sufficient for Consistency if Lock Scope Too Small and Can Cause Deadlock!Recoverability – Atomicity Requirement2PL – Growing and Shrinking Phase – Helps with Lock Scope, but Still have Deadlock/LivelockCascading Roll-back and Rigorous/Strict 2PLDeadlock Problem – Prevention, Detect & BreakAlternative Timestamp Methods
Rigorous/Strict 2PL with Deadlock Prevention or Detect & Break or Timestamp is Best
Sam Siewert 19
DISTRIBUTED DBMSWeek 13
Sam Siewert 20
Distributed DBMS – Week 13Why? – Scaling and Disaster Recovery
DR– Datacenters in 2 Geo Locations – Natural Disaster, Terrorist
Attack, WAN outages, Power grid failure, etc.– RTO / RPO – Recovery Time and Point Objectives– Asynchonrous Mirrors, Active / Passive Fail-Over and Fail-Back
Cluster Scaling – Shared Nothing– Load Balance Large Number of Clients and Transactions– Scaling Beyond Single Server– Synchronous Low-latency / High-bandwidth Network (e.g. 10GE,
40G Infiniband, 100G Converged Enhanced Ethernet)
Sam Siewert 21
MySQL ReplicationUsed for DR [Disaster Recovery] and for Geo Content and Services Distribution– East Coast / West Coast – ERAU Daytona, ERAU Prescott– ACTIVE-ACTIVE for Reads, Writes to MASTER
http://dev.mysql.com/doc/refman/5.0/en/replication.htmlAsynchronous Compared to Clustering [Synchronous]One Server is Master - MySQL Replication Solutions
Sam Siewert 22
MySQL Replication - DRFail-Over Master in Case of DisasterBEFORE MASTER FAILURE
Sam Siewert 23
MySQL Replication – DR Fail-OverFail-Over Master in Case of DisasterAFTER MASTER FAILUREChallenges – Fail-Back [Manual] and Split-Brain
Sam Siewert 24
To Restore MASTER We Must:
1) Repair Issue [E.g. HW fix]2) Re-Sync Writes3) Fail-Back to Original MASTER
MySQL ClusterMySQL Shared Nothing Cluster – NDBD, Network DB DaemonSynchronous – So Ideally Gigabit, 10GE, 40G IB or Better ClusterSimilar to PNFS Concept – Parallel Network File System (NDB MgtBottleneck?)
Sam Siewert 25
DBMS CONNECTORSWeek 13-14
Sam Siewert 26
RDBMS Client Connector - Week 13/14
Data Processing and User Interface Client Applications Need to Connect to DBMS Server
Improved Features Relative to Direct SQL Command Line for Users
Language Specific Connectors– ODBC – Open Database Connection, Middleware API Standard,
Typically Implemented in C Language– JDBC – Java Database Connector (ODBC Bridge)– MySQL C++ API Connector– MySQL C API and Driver for C Connector
Study C Examples – conntest.c
Sam Siewert 27
OODBMS AND UMLWeek 14
Sam Siewert 28
OODBMS and UML – Week 14OODBMS, Object Stores, and NoSQL Alternatives to RDBMS
Basic Definitions of Class Hierarchy, Inheritance, Encapsulation, Abstract Class, Concrete or Refined Class, Instantiation of Object, Dynamic Binding, Parametric and Ad-hoc Polymorphism
OOP – Programming Languages for Most Direction Implementation of OOA/OOD
PPL – Persistent Programming Language – One OODBMS Approach to Integrate OOP with DBMS
Type Gap in RDBMS and Type Mapping Between Store and Memory in OODBMS and Object Stores
Sam Siewert 29
Recall Object Definition – From Week 4Instance of a Class [Hierarchy from Abstract that Can’t Be instantiated to Concrete Sub-classes]Classes Define Public and Private Data and Methods to Operate on that DataSub-classes Inherit from Parent Classes and Refine Data Abstraction and MethodsOOPS – Java, C++, …, back to Smalltalk and Lisp CLOSEncapsulation and Abstraction is the GoalImplementation Hiding and Interface DefintionEach OOP Has Variations on Support [E.g. Multiple Inheritance, Abstract Classes and Methods, Polymorphism [Parametric, Ad-hoc, Operator Overloading]E.g. Oracle’s Java Object Tutorial
Sam Siewert 30
Origins of the Object-Oriented Data Model
Pearson Education © 2014 31
Two-Level Storage Model for RDBMS
Pearson Education © 2014 32
Single-Level Storage Model for OODBMS
Pearson Education © 2014 33
OODBMS Type MappingPersistent OOPL Approach– Need to Store Instantiated Objects with Persistence Attributes– Store Attribute Data and Methods (Code)
Pointer Swizzling – OIDs (Index into Look-Up Table) Provides Mapping between Methods (Function Addresses) and Data (Data Addresses)– Similar to General Descriptor Concepts – E.g. a File pointer and a File
descriptor (index that is simple integer)– Involves OID Look-Up Indirection
No Swizzle – VM Paging of Objects from Storage to Memory– VM Basic Concept – Logical Memory Space Larger than Physical, Disk
Backs Up Pages in Virtual Memory Space not Loaded in RAM– VM Addressing is Logical Memory Address translated to Physical Address
with Page Fault to Handle Unloaded Pages– OODBMS can Page Objects from Disk to Memory– VM Pages are Not Normally Durable, but Objects Paged Must be!
Sam Siewert 34
Accessing an Object with an OODBMS
Pearson Education © 2014 35
Client-Server Architecture
Pearson Education © 2014 36
SummaryIgnore FDM [Functional Data Model] and Paging Details in Chapter 27
OODBMS is a Work in Progress, but Promising
NoSQL, Object Stores, Innovation in File Systems
RDBMS Remains Ideal for Transaction Workloads
OODBMS, Object Stores, NoSQL – Growing Alternative for Newer Workloads – Object Oriented Data (BLOBs) and Big Data Analytics
Sam Siewert 37
UML – Week 14Unified Set of OO Diagraming Methods for OOA/OODBasic Concepts on Indicated Diagram Types
Sam Siewert 38
UML – Design GoalsProvide ready-to-use, expressive visual modelinglanguage …
Provide extensibility and specialization …
Be independent of particular programming languagesand development processes.Provide a formal basis …
Encourage growth of object-oriented tools market.Support higher-level development concepts such ascollaborations, frameworks, patterns, and components.
Integrate best practices.
Pearson Education © 2014 39
UML - DiagramsStructural:– Class diagrams (similar to EER with Hierarchy,
Attributes, and Class Name, Methods Added)– object diagrams (similar to ER)– component diagrams– deployment diagrams.
Behavioral:– use case diagrams– sequence diagrams– collaboration diagrams– statechart diagrams– activity diagrams.
Pearson Education © 2014 40
UML Class DiagramVery Similar to EER (Almost Identical, Just Added Detail and Extensions) – Lecture-Week-6-2– Encapsulation of Attributes and Methods to Operate on Them– Recall RDBMS has SQL/PSM and ORDBMS Has Extensions for
Methods– EER However Has Class Hierarch and Attributes, but No
Methods Specified
Sam Siewert 41
http://www.agilemodeling.com/artifacts/classDiagram.htmLecture-Week-6-2, Slide 11 to 16
UML – Object DiagramsModel instances of classes and used to describe system at a particular point in time.
Can be used to validate class diagram with “real world” data and record test cases.
Similar to ER (Lecture Week-6-1)
Pearson Education © 2014 42
UML – Deployment DiagramsDepict configuration of run-time system, showinghardware nodes, components that run on thesenodes, and connections between nodes.
Pearson Education © 2014 43
UML – Use Case Diagrams
Pearson Education © 2014 44
UML – Use Case Diagrams
Pearson Education © 2014 45
UML – Statechart DiagramsShow how objects can change in response toexternal events.Usually model transitions of a specific object.
Pearson Education © 2014 46