Growing in the Wild. The story by CUBRID Database Developers.
CUBRID Inside - Architecture, Source & Management Components
-
Upload
cubrid -
Category
Technology
-
view
3.169 -
download
0
description
Transcript of CUBRID Inside - Architecture, Source & Management Components
CUBRID InsideFirst Look Inside CUBRID
Jan 18, 2009
저작권 Copyright Notice
CREATIVE COMMONS 저작자표시 -동일조건변경허락 2.0 대한민국
이용자는 아래의 조건을 따르는 경우에 한하여 자유롭게
• 이 저작물을 복제 , 배포 , 전송 , 전시 , 공연 및 방송할 수 있습니다 .
• 이차적 저작물을 작성할 수 있습니다 .
다음과 같은 조건을 따라야 합니다 :
• 저작자표시 . 귀하는 원저작자를 표시하여야 합니다 .
• What does "Attribute this work" mean?
The page you came from contained embedded licensing metadata, including how the creator wishes to be attributed for re-use. You can use the
HTML here to cite the work. Doing so will also include metadata on your page so that others can find the original work as well.
• 동일조건변경허락 . 귀하가 이 저작물을 개작 , 변형 또는 가공했을 경우에는 , 이 저작물과 동일한 이용허락조건하에서만 배포할 수 있습니다 .
• 귀하는 , 이 저작물의 재이용이나 배포의 경우 , 이 저작물에 적용된 이용허락조건을 명확하게 나타내어야 합니다 .
• 저작권자로부터 별도의 허가를 받으면 이러한 조건들은 적용되지 않습니다 .
• Nothing in this license impairs or restricts the author's moral rights.
CREATIVE COMMONS Attribution-Share Alike 2.0 Korea
You are free:
• to Share — to copy, distribute and transmit the work
• to Remix — to adapt the work
Under the following conditions:
• Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your
use of the work).
• What does "Attribute this work" mean?
The page you came from contained embedded licensing metadata, including how the creator wishes to be attributed for re-use. You can use the HTML here
to cite the work. Doing so will also include metadata on your page so that others can find the original work as well.
• Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.
• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this web page.
• Any of the above conditions can be waived if you get permission from the copyright holder.
• Nothing in this license impairs or restricts the author's moral rights.
http://creativecommons.org/licenses/by-sa/2.0/kr/
Copyright 2009 Search Solution Corporation. All Rights Reserved.
CUBRID
41 First Look Inside CUBRID
Contents
1. System Architecture and Processes
2. Design & Implementation Concepts
3. Module and Source Structure
4. In The Next CUBRID Inside
4 /
41 First Look Inside CUBRID
System Architecture and Pro-cesses• What is CUBRID?
• System Architecture
• CUBRID Processes and IPC
• Query Processing Workflow
1
5 /
41 First Look Inside CUBRID
System Architecture and ProcessesWhat is CUBRID?
□ CUBRID is
Open Source DBMS owned and developed by NHN
• Hosted at nForge site
Targeted to a database platform for Internet portal service
Relational database system with object extensions – Object-Relational Model
1,000,000 line of source code scaled project
• Engine – C/C++, CUBRID Manager (GUI) and JDBC driver – Java
□ CUBRID provides
Full-featured database server engine
• Server and Broker architecture
GUI management and query tool
• CUBRID Manager
Various API (JDBC, ODBC(OLE DB), PHP, and CCI)
• Python and Ruby as contributed projects
1.1
6 /
41 First Look Inside CUBRID
System Architecture and ProcessesSystem Architecture
1.2
□ Server Subsystem
Multi-threaded storage engine part which runs as DB server
Consists of Transaction Manager, Object Manager, Lock Manager, Query Manager,
Log Manager, and Storage Manager
□ Client Subsystem
Client library part providing native C API
Consists of Query Parser & Optimizer, Object & Lock Cache, Transaction Manager
and Object Manager
□ Broker Subsystem
Linked with client library and implemented CCI protocol
Does job queuing, connection pooling, monitoring and logging
□ Connectors
JDBC driver
CCI library
• Many drivers are implemented with CCI library – ODBC, OLEDB, PHP, Python, and Ruby
7 /
41 First Look Inside CUBRID
System Architecture and ProcessesSystem Architecture
1.2
AdminUtility
Server
Broker
Inter-face
ConnectionPooling
Monitoring Logging
JDBC
JobQueu-
ing
ODBC CCI
PHPOLE DB
Python
Ruby
Backup /Restore
CUBRIDManager
CUBRIDManager
GUI
CM Server
Create, Delete,
Copy, Re-name
Load /Unload
Compact /Optimize
Add Vol-ume
Check /Diag
QueryParser &
Opti-mizer
QueryManager
LockManager
LockCache
ObjectManager
Transac-tion
Manager
StorageManager
LogManager
DataVol-ume
IndexVol-ume
TempVol-ume
Ac-tiveLogAr-
chiveLog
File Based
Objects
8 /
41 First Look Inside CUBRID
AdminUtility
Server
Broker
Inter-face
Parser ObjectManager
SchemaManager
Transac-tion
ManagerQuery
Transform
QueryOptimizer
PlanGenera-
tion
Workspace Manager
Memory Manager
Native C API
Communication Module
ConnectionPooling
Monitoring Logging
JDBC
JobQueu-
ing
ODBC CCI
PHPOLE DB
Python
Ruby
Client Library
Communication Module
Transac-tion
Manager
LogManager
LockManager
QueryManager
AccessMethod
B+TreeModule
File Man-ager
SystemCatalog Module
Buffer Manager
Disk Manager
Storage Engine
DataVol-ume
IndexVol-ume
TempVol-ume
Ac-tiveLogAr-
chiveLog
Backup /Restore
CUBRIDManager
CUBRIDManager
GUI
CM Server
Create, Delete,
Copy, Re-name
Load /Unload
Compact /Optimize
Add Vol-ume
Check /Diag
File Based
Objects
System Architecture
9 /
1.2
41 First Look Inside CUBRID
System Architecture and ProcessesCUBRID Processes and IPC
□ Master Process ‘cub_master’
Daemon process listening on the
TCP port for clients’ connection
• csql, broker, and admin utilities
Registrar of server processes using
Unix Domain Socket
□ Server Process ‘cub_server’
Multi-threaded process serving
clients’ requests
• Has thread pool and job queue
Accesses database volume and log
files
• One server process per one data-
base
□ Broker Process ‘cub_broker’
Daemon process listening on the
TCP port for connectors’ connec-
tion
Controls cub_cas processes (fork/
kill) with connection queue
□ CAS Process ‘cub_cas’
Connects to the server process in
behalf of connectors
• According to the database and user
name
□ CSQL Program ‘csql’
Interactive SQL execution
1.3
10 /
41 First Look Inside CUBRID
cub_cascubridc-
s.so
cub_cascubridc-
s.socsql
cubridc-s.so
System Architecture and ProcessesCUBRID Processes and IPC
Replication
StandbyServer
Broker
APWeb Server
ActiveServer
1.3
volume filevolume file
log filelog file
cub_master
cub_servercubrid.so
cub_broker
JDBC driver
CCI library
mount(read/write)
descriptor pass
UDS
TCP
request &response
request &response
register
databases.txt
read
cubrid.conf
port listening job queuemulti-thread
connect
parse
parse
port listening
query &result
query &result
shared memory
descriptor pass
cubrid_broker.conf
parse
connect
fork
cub_admincubrid-sa.so
11 /
41 First Look Inside CUBRID
System Architecture and ProcessesQuery Processing Workflow
Query Pars-ing•Parse tree
Semantic Check•Attributed parse tree
View Trans-formation•Parse tree
Query Rewrite Optimization•Parse tree
Cost Based Optimization•Query graph
Plan Genera-tion•XASL
Scan Open•Heap/Index scan
B+tree Search•OID list
Object Fetch•Lock, Buffer page
Memory Rep-resentation•DB_VALUE
Predicate Eval-uation•Qualified tuple
Query Result•List(temp) file
Cursor Open•List file scan
Result Tuple Fetch•List file page
Cursor Close•Query end
1.4
12 /
41 First Look Inside CUBRID
System Architecture and ProcessesQuery Processing Workflow
1.4
Query ManagerTransaction
ManagerLock Manager
Query Result Cache
Query Plan Cache
Query Evalua-tor
(Interpreter)
Storage Man-ager
Scan Man-agement
Extended Access
Spec Lan-guage(XASL)
UNION
DIFFERENCE
INTERSECTION
OBJFETCH
SETFETCH
BUILD_LIST
BUILD_VALUE
SCAN
MERGE_LIST
READ
UPDATE
DELETE
INSERT
XASL Type
13 /
41 First Look Inside CUBRID
Design & Implementation Con-cepts• Client-Server and Standalone
Mode
• Object, Class and OID
• Representation, Record Descrip-
tor, and Memory Object Pointer
• Workspace – Object & Lock
Caching
• 2PL vs. MVCC
• XASL (Extended Access Spec
Language)
• Heap, B+tree, Catalog, and Temp
• Click Counter
2
14 /
41 First Look Inside CUBRID
Design & Implementation ConceptsClient-Server and Standalone Mode
Client-Server Mode• Database engine is
separated into server and client process
• CUBRID is a kind of Client-server DBMS architecture
• Server process (cub_server) mounts the data-base
• Client processes, which are linked with client library (libcubridcs.so), sends requests to the server process
• Broker (cub_cas) is a typical client process
• Applications (con-nectors) are com-municating with the broker.
Standalone Mode• Database engine is
combined into one process
• Standalone pro-grams, which are linked with stand-alone library (libcubridsa.so), mount the data-base without the server
• csql with –S and admin utilities are typical standalone process
• Note that the database can be mounted only one process, cub_server or standalone pro-gram
2.1
15 /
41 First Look Inside CUBRID
Design & Implementation ConceptsObject, Class and OID
Object
• Instances and classes: A relational database is a collection of tables, and a CUBRID data-base is a collection of classes.
• The data in a relational table is contained in rows that coincide with the instances of a class.
Class
• Objects that share the same set of attributes and methods are grouped into entities called classes.
• Class hierarchy is a single rooted and connected directed acyclic graph (DAG) of classes; superclass and subclass
• The root class, db_root, is a system defined class, which is the root of the class hierarchy
OID
• A permanent unchangeable object identifier is assigned to an object when the object is stored on a heap or when an OID is requested for an object.
• The OID uniquely identifies the object and directly indicates its physical disk address: (vol-ume identifier, page identifier, slot identifier)
• OIDs can be reclaimed only during database compaction by resolving references to the re-claimed OIDs
2.2
16 /
41 First Look Inside CUBRID
Design & Implementation Concepts Representation, Record Descriptor, and Memory Object Pointer
2.3
Disk Representation•Format of object(record) when it is stored on disk•A formatted and compacted se-quence of bytes optimized for query processing•Record descriptor: RECDES•Disk structure: DISK_REPR, DISK_ATTR
Memory Representation•Format of object when it is cached on memory•A hierarchy of compound structures optimized for program access•Memory Object Pointer: MOP•Memory structure: SM_CLASS, SM_ATTRIBUTE
Representation ID: REPR_ID•Version of class representation•Whenever a change is made to a class, new representation id is assigned instead of reflect-ing the change to all instances
Cache Coherency Num-ber: CHN•To synchronize in-mem-ory cached copy of the object with its corre-sponding disk object
17 /
41 First Look Inside CUBRID
Design & Implementation ConceptsWorkspace – Object & Lock Caching
Client memory cache of database object• B
asic concept of Client-server DBMS architecture
• Whenever the object is accessed, the object locator fetches the object from the server into workspace
• Cached objects may be decached when its state is not consistent with the one stored in the database
• Note that query results are not objects but tuples
Hash table for workspace objects with linked lists• M
aps OID to MOP via workspace hash table
• MOP is a handle to a cached object
• MOP data structure contains OID, object pointer, class pointer, cached lock mode, class link, dirty link, hash link, commit link, and state flags
Acquired lock cache• B
asic concept of Client-server DBMS architecture
• Whenever an object is fetched, necessary lock is acquired before and cached with the object (lock mode)
• Note that lock manager is centralized in the server
Memory manager for object cache• Q
uick fit allocation
• Works with garbage collector
• Memory object contents are allocated within workspace memory
Workspace
2.4
18 /
41 First Look Inside CUBRID
Design & Implementation Concepts2PL vs. MVCC
Concurrency Control in DBMS
2 Phase Locking•CUBRID, SQL Server, DB2•A concurrency control locking protocol to guarantee serializability•Transaction handles its locks in two distinct, consecutive phases•Phase 1: locks are acquired and no locks are released•Phase 2: locks are released and no locks are acquired•2PL is subject to deadlocks and has deadlock resolver
Multi Version Concurrency Control•Oracle, InnoDB, PostgreSQL, AltiBase•Uses timestamps or sequence number to achieve serializability•Each version would have a write timestamp or sequence number•Read transaction does not block by maintaining several versions of an object•Drawback is the cost of maintaining multiple versions
2.5
19 /
41 First Look Inside CUBRID
Design & Implementation ConceptsXASL (Extended Access Spec Language)
• Run time structure that represent executable queries
• The XASL interpreter (query evaluator) uses the XASL NODEs to determine which query actions are to be performed for the query
• UNION_PROC, DIF-FERENCE_PROC, IN-TERSECTION_PROC, BUILDLIST_PROC, BUILDVALUE_PROC
• OBJFETCH_PROC, SETFETCH_PROC, SCAN_PROC, MERGE-LIST_PROC, READ_PROC
• UPDATE_PROC, DELETE_PROC, IN-SERT_PROC
• Scan info: defines the access method for the data
• Value lists: determine which values are needed in query re-sults
• Predicates: used to restrict query results
• Pointers to other XASL NODEs: sub-query, join, union, etc
• Heap scan: sequen-tial scan or full scan
• Index scan: B+tree search
• List file scan: tempo-ral and final query result
• Set scan: set data• Method scan: call-
back to the client for method execution
XASL Tree XASL Type
XASL NODESCAN Type
2.6
20 /
41 First Look Inside CUBRID
Design & Implementation ConceptsHeap, B+tree, Catalog, and Temp
File
Heap• Object storage
in data vol-umes
• One per class
B+tree• Index storage
in index vol-umes
• One per index
Catalog (Ex-tendible hash)• Class represen-
tation• One per system
Temp• Temporal or fi-
nal query result• In temp vol-
umes
2.7
21 /
41 First Look Inside CUBRID
Design & Implementation ConceptsClick Counter
□ In Terms of User
Whenever the user clicks an article, the read counter of the article is increased.
• Counts the clicks on the article
□ In Terms of Service Application Developer
The read counter is very important for bulletin board type service.
Combine getting article data and increasing read counter together
• get_article() + increase_counter() = get_article_and_increase_counter()
• (SELECT article_info) + (UPDATE read_counter+1)
□ In Terms of Database Engineer
Do SELECT and UPDATE in one SQL statement
• SELECT trigger? SELECT FOR UDPATE?
UPDATE the record with the short term lock
□ Example
SELECT doc_id, title, INCR(read_counter), post_date FROM board WHERE
doc_id=?
2.8
22 /
41 First Look Inside CUBRID
Design & Implementation ConceptsClick Counter
2.8
Parsing
Plan Generation
Access Spec
Scan Open
B+tree Search
Address of Record
Object Locking
Object Representation
Copy to Result Set
Return
Fetch
Parsing
Plan Generation
Access Spec
Scan Open
B+tree Search
Address of Record
Object Locking
Object Representation
Change Values
Logging
Return
Parsing
Plan Generation
Access Spec
Scan Open
B+tree Search
Address of Record
Object Locking
Object Representation
Copy to Result Set
Change Values
Logging
Return
Fetch
SELECT UPDATE Click Counter
Don’t walk the same passage twice!When you get there, do as many things as possible.
Killing two birds with one stone.
23 /
41 First Look Inside CUBRID
Module and Source Structure• Overall Structure
• Query Processing Component
• Transaction Management Com-
ponent
• Server Storage Management
Component
• Client Storage Management
Component
• Object Management Component
• Job and Thread Management
Component
• Client-Server Communication
Layer
• Broker Component
3
24 /
41 First Look Inside CUBRID
Module and Source StructureOverall Structure
src
api
base
broker
cci
cmserver
communication
compat
connection
executables
heaplayersjdbcjsp
object
odbc
oledb
optimizer
parser
php
query
storage
thread
transaction
25 /
3.1
Object Management
Query Processing
CUBRID Manager
Query Processing
Query Processing
Broker
Client-Server Comm
Job & Thread Management
Transaction Management
Server Storage Management
Client Storage Management
Memory Manager
Native C API
CCI Library
Job & Thread Management
Java Stored Procedure
PHP Connector
JDBC Driver
C APICommon & Support
Utilities
41 First Look Inside CUBRID26 /
3.1
Parser ObjectManager
SchemaManager
Transac-tion
ManagerQuery
Transform
QueryOptimizer
PlanGeneration
Workspace Manager
Memory Manager
Native C API
Communication Module
ConnectionPooling
Monitoring Logging
JDBC
JobQueuing
ODBC CCI
PHPOLE DB
Python
Ruby
Client Library
Communication Module
Transac-tion
Manager
LogManager
LockManager
QueryManager
AccessMethod
B+TreeModule
File Man-ager
SystemCatalog Module
Buffer Manager
Disk Manager
Storage Engine
Backup /Restore
CUBRIDManager
GUI
CM Server
Create, Delete,
Copy, Re-name
Load /Unload
Compact /Optimize
Add Volume
Check /Diag
Module and Source Structure
src/query
src/storage
src/transaction
src/compat
src/broker
src/cmserver
src/communication
src/jdbc
src/object
src/oledb
src/php
src/cci
src/odbc
src/optimizer
src/base
src/executables
src/heaplayers
src/parser
src/thread
src/connection
41 First Look Inside CUBRID
Module and Source StructureQuery Processing Component
□ Scanner/Parser
Parses query string and builds parse tree
Uses ANTLR v1.x as parser generator
• Will be replaced with ‘bison’ at R2.0
Source files in src/parser
• Data structure: PT_PARSER, PT_NODE, …
□ Semantic Checker
Type check and binding, name resolution,
semantic checking, view translation, and so
on
Source files in src/parser
• Parse tree traversing function:
parser_walk_tree()
□ Optimizer
Rewrite optimization by heuristic rules
Select a query execution plan by CBO (Cost
Based Optimization) method
Source files in src/parser and src/optimizer
• Data structure: QO_NODE, QO_PLANNER, …
□ XASL Generator
Generate XASL tree from the query execution
plan
XASL contains scan information (heap/index/
list file/set/method scan), value list, and predi-
cates
Source files in src/parser
• Data structure: XASL_NODE, REGU_VARIABLE, …
□ Query Manager (Executor)
Executes XASL tree (XASL interpreter)
Includes query evaluator, query plan cache,
query result cache, query result file, and scan
management
Source files in src/query
• Main function: qexec_execute_mainblock()
□ Cursor Manager
Extracts tuple from the list file page shipped
from the server Query Manager
Source files in src/query
3.2
27 /
41 First Look Inside CUBRID
Server
Broker
Client-Server Communication Layer
Client-Server Communication Layer
Module and Source StructureQuery Processing Component
3.2
Query Manager
Storage Manage-ment Component
(Server)
Semantic Checker Scanner/Parser Optimizer
XASL Generator
Object Manage-ment Component
Cursor Manager
Transaction Man-agement Compo-
nent
Query Plan/Result Cache
Query Evaluator(Interpreter)
Scan Management
28 /
41 First Look Inside CUBRID
Module and Source StructureTransaction Management Component
□ Transaction Manager
Controls transaction – start, commit/abort,
savepoint, and top operation
• Works tightly with lock and log manager
Supports 2PC protocol for global transaction
Initialize other related modules such as lock,
log, and recovery manager
Source files in src/transaction
• Data structure: LOG_TDES, TRANTABLE, …
□ Object Locator
Controls the flow of objects between the
workspaces and the database pages
• Translates between memory representation and
disk representation
Caches objects from the server into the
workspace with the granted locks
• Cached object and lock provides fast access to
the objects and reduces interacting with the lock
manager on the server
Source files in src/transaction
□ Lock Manager
Based on strict 2PL protocol
Includes support for OR feature
• class lock and instance lock
Deadlock detector with WFG
• As separate thread
Source files in src/transaction
• Data structure: LK_ENTRY, LK_TRAN_LOCK, …
□ Log Manager
Transaction logging and recovering
• REDO/UNDO recovery protocol
• WAL protocol
• Physical logging and logical logging
Supports group commit and asynchronous
commit feature
Source files in src/transaciton
• Data structure: LOG_GLOBAL,
LOG_HDRPAGE, LOG_PAGE,
3.3
29 /
41 First Look Inside CUBRID
Module and Source StructureTransaction Management Component
3.3
Server
Broker
Client-Server Communication Layer
Object Locator(Server)
Storage Manage-ment Component
(Server)
Lock Manager
Client-Server Communication Layer
Storage Manage-ment Component
(Client)
Object Management Component
Transaction Man-ager
(Client)
Transaction Man-ager
(Server)
Log Manager
Object Locator(Client)
30 /
41 First Look Inside CUBRID
Module and Source StructureServer Storage Management Component
□ File I/O Manager
Provides disk I/O access functions and implements file
structure
Coordinates the allocation and deallocation of un-
structured files and pages
• File>Sector>Page
• File manager requests sectors from the disk manager
Source files in src/storage
• Identifier: vdes, PAGEID
□ Disk Manager
Volume file management – implements volume struc-
ture in a OS file
• A volume corresponds to a singe file
• A set of contiguous pages of fixed size
Source files in src/storage
• Identifier: volid
□ Page Buffer Manager
Data buffer pool with LRU replacement algorithm
Uses buffer hash table and fix/unfix protocol (latch)
Source files in src/storage
• Identifier: VPID
• Data structure: PGBUF_BUFFER_POOL
□ Slotted Page Manager
Manages insertions, deletions, and modifications of
records on pages
• Each record has an associated slot identifier
Source files in src/storage
• OID = (pageid, slotid, volid)
□ Overflow File Manager
Larger object than the size of a page is placed on
overflow pages
Source files in src/storage
□ Object Heap Manager
Inserts/deletes/updates objects within a file
• Permanent unchangeable OID is assigned to an object
when it is stored on a heap
• Provides heap scan method (sequential scan) and direct
fetch method
An object heap holds the instances of one class
Source files in src/storage
• Identifier: HFID
□ B+-tree Manager
Implements prefix B+-tree index with KVL
• Provides index scan method – unique find and range
search
3.4
31 /
41 First Look Inside CUBRID
Module and Source StructureServer Storage Management Component
3.4
Server
Broker
Client-Server Communication Layer
Client-Server Communication Layer
I/O Manager
Page Buffer Man-ager
File Manager
Catalog Manager
Object Heap Manager
B+-tree ManagerLarge Object
ManagerExtendible Hash
Manager
Query Processing Compo-nent
(Server)
Transaction Management Component
(Server)
Disk Manager
Slotted Page Manager
Overflow File Manager
32 /
41 First Look Inside CUBRID
Module and Source StructureClient Storage Management Component
□ Workspace Manager
Maintains a cache of database objects
within the virtual memory address
space of client process
• Maintains hash table that maps a OID
into MOP
• MOP has OID field and a pointer to
memory object
Source files in src/object
□ Quick Fit Allocator
Memory manager for workspace
Source files in src/object, src/base
□ Garbage Collector
To collect and free the
unused(dangled) memory objects
Normally GC is disabled
□ Object and Lock Caching
When the contents of object(s) is(are) re-
quired, the object locator is called to fetch
the object(s) from the database via the
server-side object locator
The object locator translate the disk repre-
sentation of the objet into the memory rep-
resentation and allocates MOP to points it
While accessing the object in the database,
the requested lock is held by the lock man-
ager in the server and the memory object
has the cached lock mode
The cached objects can be decached at any
time when its state is not consistent with the
disk objects, or will be decached at the time
of transaction abort
• At the time of transaction commit, only the
cached locks are released; objects remains
with null lock
3.5
33 /
41 First Look Inside CUBRID
Module and Source StructureClient Storage Management Component
3.5
Server
Broker
Client-Server Communication Layer
Client-Server Communication Layer
Garbage Collector Quick File AllocatorWorkspace Manager
Object Management Component
Transaction Man-agement Compo-
nent Object Locator
(Client)
34 /
41 First Look Inside CUBRID
Module and Source StructureObject Management Component
□ Object Accessor
Interface for object creation, deletion, inspec-
tion, and modification
Source files in src/object
□ Object Representation
Transforms (or translates) between the disk
representation and the memory representation
of the object
• Resolves byte ordering also
Source files in src/object, src/base
• Functions: or_xxx(), tf_mem_to_disk(),
tf_disk_to_mem(), …
□ Data Type and Domain
Internal data structure for data type and do-
main
Source files in src/object
• Data structure: TP_DOMAIN
□ Sets
Source files in src/object
• Data structure: SETOBJ, COL
□ Schema Manager
Implements OR model regarding database schema
• Definitions of the class object – attributes, method,
class inheritance, and conflict resolution
• Memory representation of class object
Source files in src/objet
• Data structure: SM_CLASS, SM_ATTRIBUTE, …
□ Authentication and Authorization
User authentication and database privilege
• User, GRANT, REVOKE, …
Source files in src/object
□ Trigger Manager
Implements trigger feature
Source files src/object
□ Dynamic Loader
Support dynamic loading of shared object as
‘method’
• Depreciated feature
□ Large Object
Implements glo, elo, fbo classes and supporting
methods
Source files in src/object
3.6
35 /
41 First Look Inside CUBRID
Module and Source StructureObject Management Component
3.6
Server
Broker
Client-Server Communication Layer
Client-Server Communication Layer
Set
Schema ManagerAuthentication
& AuthorizationTrigger Manager
Dynamic Loader
Object Representa-tion
Storage Manage-ment Component
(Client)
Data Type and Do-main
Object Accessor
36 /
41 First Look Inside CUBRID
Module and Source StructureJob and Thread Management Component
□ Connection Manager
Connection establishment and man-
agement and request handler
Source files in src/communication, src/
connection
• Data structure: CSS_QUEUE_ENTRY,
CSS_CONN_ENTRY, …
□ Job Queue
Master thread monitors socket fds (se-
lect()), receives clients’ request packet,
en-queues it, and then wake up a worker
thread in the thread pool
Source files in src/connection
• Data structure: CSS_JOB_ENTRY, …
□ Critical Section
Implements semaphore
Source files in src/thread
□ System Threads
Master thread
Deadlock detection thread
• Run deadlock detection algorithm periodically
• Resolves deadlock by aborting unilaterally vic-
tim transaction(s)
Checkpoint thread
• Executes periodic checkpoint action
OOB handling thread
• Catch OOB signal
Page flush thread
• Flushes dirty data buffer pages to the volume
files in the background
• To reduce burden of page replacement
Log flush thread
• Flushes dirty log buffer pages to the log file in
the background
• Implements group commit and asynchronous
commit features
Source files in src/thread
3.7
37 /
41 First Look Inside CUBRID
Module and Source StructureClient-Server Communication Layer
□ Socket Transceiver
Socket utility functions
• TCP/IP, Unix domain socket, and
named pipe
Source files in src/connection
• Linux and Windows port
□ Packet Handler
Sends requests and receives re-
sponse between client and server
• Packet types: request packet, data
packet, close packet, out-of-band
packet, error packet, abort packet
• Packet format: packet header + pay-
load
Queuing request and data packets
Source files in src/connection
□ Communication Interface
Layer
RPC stub style functions
• Implements object serialization
Source files in src/communication
3.8
38 /
41 First Look Inside CUBRID
Module and Source StructureBroker Component
□ Broker Process
Accepts connection requests from
the Connectors (JDBC, CCI, …)
Allocates and/or assigns CAS
process to the request
Controls and monitors CAS pro-
cesses
Has four threads
• main: manages cas process
• receiver_thread: does accept() and
enqueues new job
• dispatch_thread: dequeue job and
assigns cas process
• cas_monitor_thread: monitors cas
process to restart
Source files in src/broker
□ CAS Process
Linked with CUBRID client library
• So, run as a single thread
Receives and processes requests from
the application(Connectors)
• Connect to database
• Prepare and execute query
• Fetch the result
• Commit/rollback transaction
• Get/set parameters
Has four states
• IDLE: not connected with AP
• BUSY: processing request
• CLIENT_WAIT: connected and waiting for
request; within a transaction
• CLOSE_WAIT: connected and waiting for
request; out of a transaction
Source files in src/broker
3.9
39 /
41 First Look Inside CUBRID
In The Next CUBRID Inside4
40 /
41 First Look Inside CUBRID
In The Next CUBRID Inside
Academic theory?
Implemen-tation de-
tails?
New fea-tures?
OrFuture of
the project?
Processing flow?
Practical is-sues?
Details of Transaction Manage-ment
Communication between Broker and Connectors
Replication and/or HA feature
4
What topic do you want to see covered in the next CUBRID Inside?
Suggesting Topics
41 /
Question?