2005 JavaOneSM Conference | Session XXXX
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
High PerformancePersistence
Donald Smith
Oracle Corporation
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Show how you build high performance
J2EE applications that scale using an
object-relational persistence layer
Goal
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Java Access of Relational Data
Direct JDBC
Direct SQL calls hard coded in Java
Use result sets (“rows”) directly
Persistence layer
Accessed as objects or components
Transparent that the data is stored in RDB
Persistence layer in middle tier handlesobject-relational mapping and infrastructure
Required if doing business logic in the middle tier!
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
JDBC—Java DatabaseConnectivity
Java standard foraccessing databases
JDBC is simply thedatabase connectionutilities Javadevelopers need tobuild upon
SQLrows
JDBC
SQLrows
Java &Java &
WebWeb
ServicesServices
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
JDBC
Simply – data source connection
OK with:“Window on data” applications
Business logic entrenched ondatabase
Java nothing more than GUI tool
Is it your only option for “highperformance persistence”? –NO!
SQLrows
JDBC
SQLrows
Java &Java &
WebWeb
ServicesServices
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Persistence Layer
Abstracts persistence details from theapplication layer
SQLrows
Objects
Persistence Layer
Objects
object-level
querying and creation
results are objects
results are
returned as
raw data
API uses SQL
or database
specific calls
object creation and
updates through
object-level API
Java &Java &
WebWeb
ServicesServices
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Persistence Layer
SQLrows
Objects
JDBC
Objects
object-level
querying and creation
results are objects
results are
returned as
raw data
API uses SQL
or database
specific calls
object creation and
updates through
object-level API
J2EE &J2EE &
WebWeb
ServicesServices
Persistence Layer
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
ORM Persistence Entity Types
EJB 2 Persistence: Entity Beans
BMP: Developer must hand code persistence “lifecycle” calls generate by J2EE Container
CMP: More Automatic Persistence
“Plain Old Java Objects” (POJO) Persistence
Oracle TopLink
Open Source: JBoss Hibernate
DAO Pattern: Custom, Generators
JDO Implementations
EJB 3.0 Persistence: Entity Beans == POJOs
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
iBATIS – Somewhere inbetween…
iBATIS is not meant to replace Object-Relational Mapping(ORM) tools. It is a low-level framework that allows you todirectly hand-code your SQL statements and map them toJava object. You don’t need to lookup DataSource, getconnections, create prepared statements, parse ResultSetor even cache the results – iBATIS does it all for you.Under the covers, iBATIS creates a PreparedStatement, setsthe parameters (if any), executes the statements and buildsa Map or JavaBean object from the ResultSet.
Summarized from Vinny Carpenter’s BLOG entry:
http://www.j2eegeek.com/blog/2005/01/31/ibatis-where-have-you-been-all-my-life/
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Impedance Mismatch
Difference in relational and objecttechnology know as “object-relationalimpedance mismatch”
Challenging problem
Requires relational and object expertise
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Impedance Mismatch
Factor J2EE RDBMSLogical Data
Format
Objects, methods,
inheritance
Tables, SQL, stored
procedures
Scale Hundreds of megs Gigabytes, terabytes
Relationship Object references Foreign keys
Uniqueness Internal object identity Primary keys
Key Skills Java development,
object modeling
SQL, Stored Procedures, data
management, Data modeling
Tools IDE, Source code
management, Object
Modeler
Schema designer, query
manager, performance
profilers, database config
Corporate
Org.
Structure
“Newer technology”
often with weak
organizational ties to
database mgmt
Often mature infrastructure
with significant legacy
considerations
Technic
al
Busin
ess
Polit
ical
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Mapping
Object model and Schema must be mapped
True for any persistence approach
Most contentious issue facing designers
Which classes map to which table(s)?
How are relationships mapped?
What data transformations are required?
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Mapping Process
Map Object Model to schema
Java IDEJava source
MappingTool
Schema
Mapping
Metadata
(XML/Annotation)
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Mapping Tools
Theunderlyingmappingflexibility isveryimportant
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Modeling and Mapping
Options:
Generate Object Model from schema
Generate schema from Object Model
Meet in the Middle
• Existing Object Model
• Existing Schema
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Modeling and MappingGeneration
Excellent productivity boost at start ofproject
Don’t take the generated code/schema asgospel!
Tools are smart not psychic
Many legacy schemas will lead to classicproblems discussed shortly…
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Recommendations
Ensure ORM layer allows customization ofmappings
Match the approach to current and long termproject requirements
New/Flexible schema versus legacy staticschema
Future enhancements
Maintenance
If absolutely tied to a schema/model with nohope of change – reconsider strategy
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Impedance Mismatch:Inheritance
Java
Used to extend (share) common state &behavior
Relational
Shared data can be normalized into commontable
Adds additional unique constraint withincommon table
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Inheritance
Very straight forward at first, but canbecome challenging because of the numberof possible solutions
Compounded with relationships
Can be source of contention with DBA –generally leads to lots of joining on queries!
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“Leaf Table Mapping”
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
Map only concrete classes
SUB1
ID VAR
SUB22
7 Foo
SUB1VAR
Pish
ID VAR
13 Tim
SUB2VAR
677
SUB22VAR
Flim
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“Leaf Table” Issues
How do you query for the Root class?
Data is not normalized
Performance Impact:
PRO: No joins or table contention between types
CON: No heterogeneous query results
Is this your fastest option? If you only do leafqueries, perhaps it is…
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“All Table Mapping”
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
Map allclasses ROOT
ID VAR
SUB2
7 Foo
13 Bar
ID
13
SUB2VAR
677
SUB22VAR
Flim
SUB1
ID
7
SUB1VAR
Pish
ID
13
SUB22
Problem!How to telltype?!
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Side Bar: Identifying Type
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
Map allclasses ROOT
ID VAR
SUB2
7 Foo
13 Bar
ID
13
SUB2VAR
677
SUB22VAR
Flim
SUB1
ID
7
SUB1VAR
Pish
ID
13
SUB22
Need TypeIdentifier!
TYPE
S1
S22
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Side Bar: Identifying Type
ROOT
ID VAR
7 Foo
13 Bar
Lots of strategies exist withvarious performanceimpacts…
TYPE
S1
S22 ROOT
ID VAR
7 Foo
13 Bar
TYPE
com.foo.Sub1
com.foo.Sub22
ROOT
ID VAR
7 Foo
13 Bar
if id < 10
type = Sub1
else
type = Sub22
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Side Bar: Identifying Type
Another option to “type identifier fields”exists Just read through ALL Leaf classesand figure it out in memory… Performance?
Sub1sub1var: String
Root {a}
id: int
var: String
Sub2 {a}sub2var: int
Sub22sub22var: String
ROOTID VAR
SUB2
7 Foo
13 Bar
ID
13
SUB2VAR
677
SUB22VAR
Flim
SUB1ID
7
SUB1VAR
Pish
ID
13
SUB22
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“All Table” Issues
Lots (and lots) of joining
Type identification can be inefficient
Performance Impact:
PRO: Root Level/Abstract Class query resultspossible
CON: Additional queries or joins required
CON: Type Identification
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“Single Table Mapping”
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
Map to one table
ROOT
ID VAR
7 Foo
13 Bar
SUB1VAR
Pish
null
SUB2VAR
null
677
SUB22VAR
null
Flim
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Identifying Type
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
ROOT
ID VAR
7 Foo
13 Bar
SUB1VAR
Pish
null
SUB2VAR
null
677
SUB22VAR
null
Flim
TYPE
S1
SUB22
if SUB1VAR != null
type = Sub1
else
type = Sub22
Two options fortype detection
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“Single Table” Issues
DBA will freak out – de-normalized database
Type identification can be inefficient
Performance Impact:PRO:• Heterogeneous query results possible• No extra queries or joins for subclasses
CON: Additional table size, unused columns
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
“Combination Table Mapping”
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
Mapappropriateclasses
ROOT
ID VAR
7 Foo
13 Bar
SUB2VAR
677
SUB22VAR
Flim
SUB1
ID
7
SUB1VAR
Pish
ID
13
SUB22
TYPE
S1
S22
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Which Strategy to Use?
Leaf TableWhen never querying at abstract class level
All TableWhen not doing much querying• Too many joins
Single TableWhen one subclass is much more prevalent, andyour DBA doesn’t care about normalization
CombinationUse to optimize above situations
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Wouldn’t It Be Cool?
Sub1
sub1var: String
Root {a}
id: int
var: String
Sub2 {a}
sub2var: int
Sub22
sub22var: String
Map to view for reads…ROOT (View)
ID VAR
7 Foo
13 Bar
SUB1VAR
Pish
null
SUB2VAR
null
677
SUB22VAR
null
Flim
ROOT
ID VARSUB2
7 Foo
13 BarID
13
SUB2VAR
677
SUB22VAR
Flim
SUB1ID
7
SUB1VAR
Pish ID
13
SUB22
Map to tables for writes…
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Locking
J2EE Developers want to think of locking atthe object level
Databases may need to manage lockingacross many applications
Persistence layer or application server mustbe able to respect and participate in locks atdatabase level
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Optimistic Locking
DBA may wish to use version, timestampand/or last update field to representoptimistic lock
Java developer may not want this in theirbusiness model
Persistence layer must be able to abstract this
Must be able to support using any fieldsincluding business domain
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Optimistic Locking
EMP
ID NAME
7 Don
13 Trisha
VER
3
8
SAL
40,000
55,000
UPDATE EMP SET NAME=‘DONALD’,
VER=4 WHERE ID=7 AND VER=3
UPDATE EMP SET SAL=50,000,
VER=4 WHERE ID=7 AND VER=3
SQL from Transaction B
SQL from Transaction A
ID NAME
7 Don
13 Trisha
VER
4
8
SAL
50,000
55,000
ID NAME
7 Donald
13 Trisha
VER
4
8
SAL
40,000
55,000
If A “Wins”:
B gets “Opt Lock Exception
If B “Wins”:
A gets “Opt Lock Exception
OR
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Pessimistic Locking
Requires careful attention as a JDBCconnection is required for duration ofpessimistic lock
Should support SELECT FOR UPDATE[NOWAIT] semantics
Tim
e
Begin UOW
Commit UOW
Begin Txn
Commit Txn
Pess Lock
J2EE &J2EE &
WebWeb
ServicesServices
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Concurrency Protection:Locking
Proper locking semantics is a must
Recommendations:
Use optimistic locking
• Reads can avoid unnecessary refresh by comparing lock values
Minimize pessimistic locking
• Reduce database resource contention with longer transactions
• Use when retry is more costly then locking
• Typically used in data processing or non-user interactivemodules
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Querying
Critical element of ORM tuning
Most applications have more reading then writing
GOALS
Reducing the amount of SELECT statements
Optimize the speed of each SELECT statement
RECOMMENDED SOLUTIONS
Just in time reading
Result caching – object or result container
Optimized SQL generation
Data projections versus full object loading
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Object Traversal – Lazy Reads
J2EE Applications work on the scale of afew hundreds of megabytes
Databases routinely manage gigabytesand terabytes of data
Persistence layer must be able totransparently fetch data “just in time”
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Classic Support Case
Customer – “Uh, Hi. I am trying to read 1object and get 3,000 SQL statements.”
Support – “Have you turned on indirection?”
Customer – *silence*
Support – “Turn on Proxies and let us knowwhat happens”
Customer – “Oh, 3 SQL statements now.Thanks ”
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Just in Time Reading – FaultingProcess
Customer
Orders
Proxy
1. Accessing relationshipfor first time
2. Get relatedobject basedon FK3b. SQL if
not cached3a. CheckCache
4. PlugresultintoProxy
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Lazy Loading – Just in TimeReading
Use of proxy to defer reading until required
Very valuable performance feature
Several Implementation Options
Explicit proxy
Dynamic proxy (java.lang.reflect.Proxy)
Development time class enhancement (source or byte codes)
Dynamic weaving (AOP)
Customer AddressValueHolder
List PhoneNumber
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
N+1 Reads Problem
Many persistence layers and applicationservers have an N+1 reads problem
Causes N subsequent queries to fetchrelated data when a collection is queried for
A side effect of the impedance mismatch andpoor mapping and querying support inpersistence layers
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
N+1 Reads Problem
Pool of Created
Objects or Beans
Persistence
Layer or EJB
Container
findByCity()
1
findByCity()
2
For each Customer
Fetch their Address
Address
46
4
n
5 5
3 Returns collection
n
If Address had related
objects, they too may be
fetched 2n+1 Reads!
Container returns results
C C C C
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Classic Support Case #2
Customer – “Uh, Hi. I am trying to read 1object and get 3,000 SQL statements.”
Support – “Have you turned on indirection?”
Customer – “Yes, and it’s still at 3,000 SQLs.”
Support – “Turn on join/batch reading”
Customer - * SILENCE *
Customer – “Oh, 3 SQL statements now.Thanks ”
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Example Object Model
PurchaseOrder
id : int
customer : Customer
Customer
id : int
address : Address
phoneNumbers : List
Address
id : int
street : String
city : String
state : String
postalCode : String
country : String
PhoneNumber
customer : Customer
number : String
type : String
1
1
1
*
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Query Example
Find all pending PO’s with a customerin San Francisco and display customerdetailsEJB QL
SELECT po FROM PurchaseOrder po
WHERE po.status = 'ACTIVE' AND
po.customer.address.city = 'SFO‘
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Query Example: Initial SQL
SELECT PO.* FROM PO, CUST, ADDR WHEREPO.STATUS = ‘ACTIVE’ AND PO.CUST_ID =CUST.ID AND CUST.ADDR_ID = ADDR.ID ANDADDR.CITY = ‘SFO’
{Returns N Purchase Orders}
SELECT * FROM CUST WHERE CUST.ID = 1 …{N}
SELECT * FROM ADDR WHERE ADDR.ID = 100 …{N}
SELECT * FROM PHONE WHERE PHONE.CUST_ID =1 … {N}
RESULT: 3N+1 queries (100 PO’s = 301)
“N+1 Query – explosion of SQL”
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
N+1 Reads
Must have solution to minimize queries
Need flexibility to reduce to 1 query, 1+1query or N+1 query where appropriate
1 Query when displaying list of customers andaddresses – known as a “Join Read”
1+1 Query when displaying list of customers anduser may click button to see addresses – known asa “Batch Read”
N+1 Query when displaying list of customers butonly want to see address for selected customer
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Where Do I Define This?
Many options exist, depends on persistencelayer and spec
In the query language syntax
On the mapping
Development time/runtime
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Join Reading 1:1’s
Join ReadingJoin rows together in a single SELECT
Example of generated SQLSELECT PO.*,CUST.*,ADDR.* FROM PO, CUST,ADDR WHERE PO.STATUS = ‘ACTIVE’ ANDPO.CUST_ID = CUST.ID AND CUST.ADDR_ID =ADDR.ID AND ADDR.CITY = ‘SFO’
{Returns N Purchase Orders with Customers& Addresses}
SELECT * FROM PHONE WHERE PHONE.CUST_ID = 1
…{N calls}
RESULT: N+1 queries (100 Phone #’s = 101)
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Join Reading All Relationships
Join ReadingRead in rows that can be joined together in asingle SELECT
ExampleSELECT PO.*,CUST.*,ADDR.*, PHONE.*FROM PO, CUST, ADDR, PHONEWHERE PO.STATUS = ‘ACTIVE’ AND PO.CUST_ID= CUST.ID AND CUST.ADDR_ID = ADDR.ID ANDADDR.CITY = ‘SFO’ AND PHONE.CUST_ID =CUST.ID
{Returns N Purchase Orders withCustomers, Addresses, and PhoneNumbers}
RESULT: 1 query (100 PO’s)
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Batch and Join Reading
Use SELECT per child with join to originalclause
ExampleSELECT PO.*,CUST.*,ADDR.* FROM PO, CUST, ADDR WHERE
PO.STATUS = ‘ACTIVE’ AND PO.CUST_ID = CUST.ID ANDCUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’
{Returns N Purchase Orders with Customers &Addresses}
SELECT PHONE.* FROM PHONE, PO, CUST, ADDR WHEREPO.STATUS = ‘ACTIVE’ AND PO.CUST_ID = CUST.ID ANDCUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’ ANDPHONE.CUST_ID = CUST.ID
RESULT: 2 queries (100 PO’s)
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Joining + Batch Reading
Use SELECT per child with join to originalclause
ExampleSELECT PO.* FROM PO, CUST, ADDR WHERE PO.STATUS =
‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND CUST.ADDR_ID= ADDR.ID AND ADDR.CITY = ‘SFO’
SELECT CUST.* FROM PO, CUST, ADDR WHERE PO.STATUS =‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND CUST.ADDR_ID= ADDR.ID AND ADDR.CITY = ‘SFO’
SELECT ADDR.* FROM PO, CUST, ADDR WHERE PO.STATUS =‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND CUST.ADDR_ID= ADDR.ID AND ADDR.CITY = ‘SFO’
SELECT PHONE.* FROM PHONE, PO, CUST, ADDR WHEREPO.STATUS = ‘ACTIVE’ AND PO.CUST_ID = CUST.ID ANDCUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’ ANDPHONE.CUST_ID = CUST.ID
RESULT: 100 PO’s = 4 SQL
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Join & Batch Reading
Reducing the quantity of SQL calls willproduce a significant performance gain
Joining and batching can be combined
Be careful not to extend the depth too deep
The cost of the join could become too expensive
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Query Optimization Options Reading 100 PO’s with relationships
Source: Simple test environment
# SQL Time
Default 301 200 ms
Join 1:1’s 101 150 ms
Join All 1 60 ms
Batch All 4 40 ms
Batch & Join 2 20 ms
Fewer SQL ! Always = Faster Execution
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Queries: Minimize Data Read
Only read the entire object when required
When modifying the object
When caching will assist concurrent clients
All fields of the object are required
Consider projections versus object queries
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Query: Projection Example
EJB QL: SELECT new PODetails(po.id, po.num,
po.amount, c.lastName, a.city)
FROM PurchaseOrder po JOIN po.customer c,
JOIN po.customer.address a
WHERE po.status = 'ACTIVE‘ AND a.city =
'SFO‘
SQL: SELECT PO.ID, PO.NUM, PO.AMOUNT, CUST.L_NAME,
ADDR.CITY FROM PO, CUST, ADDR WHERE PO.STATUS =
‘ACTIVE’ AND PO.CUST_ID = CUST.ID AND
CUST.ADDR_ID = ADDR.ID AND ADDR.CITY = ‘SFO’
New YorkJones500.00CD4562
OttawaSmith100.00AB1231
CityLast$PO #ID
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Object Traversals
How would a developer using a typicalpersistence layer find “The averageorder amount for customers in city X”?
In other words, how do Java developerswrite “report” style queries?
CustomerOrder OrderItem
Productname
city
orders
orderDate
cost
orderItems
quantity
discount
product
productName
cost
weight
1
1 11* *
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Example Codepublic double findAverageOrderSizeFor(String city) {
int count=0; double value=0.0;
Customer c = new Customer();
c.setCity(city);
Collection custs = pLayer.executeQueryByExample(c);
iterate i over custs {
Collection ords = custs[i].getOrders();
iterate j over ords {
value += ords[j].getValue(); count++;
}
}
return value/count;
}
This approach causes unnecessary
interactions with the database and
brings too much data into the
appserver…
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
A Better Solution
EJB QL: SELECT avg(ord.amount)
FROM Customer cu JOIN cu.order o,
WHERE cu.city = :city
SELECT AVG(t0.VALUE) FROM ORDERS t0, ADDRESS t2, CUSTOMER t1
WHERE ((t2.NAME = city) AND ((t0.CUST_ID = t1.ID) AND (t2.ID = t1.ADD_ID)))
Generates
Projections allow Java developers to leverage
the database instead of trying to do all the
computation in JVM
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Query: Projection Optimizations
Specify the required fields in terms of theobject model
Use relationships to indicate joins
Use aggregate functions
SUM, MIN, MAX, COUNT, AVERAGE
Produce easy to render results
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Transactions: Short & Sweet
J2EE apps typically support many clients sharingsmall number of db connections
Ideally would like to minimize length of transactionon database
Tim
e
Begin UOW
Commit UOWBegin Txn
Commit Txn
J2EEJ2EE
AppApp
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Transactions: Minimal Writes
Persistence layer tracks/calculates changes
Only UPDATE modified columns
Order UPDATE, INSERT, & DELETES usingreferential integrity rules and consistentorder
Benefits
Simplified development
Reduced TX costs
Minimize deadlock occurrences
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Transactions: Minimize Context
Each object in a transaction context has acost
Only include objects which may be changedin the transaction context
Symptoms
Long commit cycles
Increased garbage creation
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Minimize Context
Bad:
Better:
begin UOWread 10,000 objectspick 1change 1
commit UOW
read 10,000 objectspick 1begin UOW
change 1commit UOW
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Transactions: Other Optimizations
Data Access Optimizations
Parameter Binding
ResultSet streaming
Streaming of large objects (BLOB/CLOB)
Cache Interaction
Transaction updates to shared object cache
Distribution of changes to clustered caches
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Why Have a Cache?
Object Identity
Performance – Avoid extra database trips
Reference Read-Only
Read-mostly
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Caching
OO Query SQL Query
Results(s)
Does PK for rowexist in cache?
YES – Getfrom Cache
NO – Buildbean/objectfrom results
Returnobjectresults
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
ORM Caching Options
Isolated cache
Per Client/Thread
Transactional
Conforming
Shared cache
EntityManager
Session
EntityManager
Factory
Server
UnitOfWork
Cache Cache
Cache
J2EEJ2EE
AppApp
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Caching Architecture
ServerShared
Cache
SessionIsolated
Cache
UnitOfWork
TX
Cache
ServerShared
CacheSession Bus
JDBC/JCA
JMS (MDB)
RMI
CORBA
IIOP
Distributed/Coordinated
Configurable/Flexible
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Cache Configuration
Categorize your persistent entity types
Isolated: short cache life
Shared: medium to long cache life (app/dbspecific)
Reference: Long cache life
Configure each type
Cache type
Initial and/or maximum size
Invalidation (time-to-live)
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Caching –Stale Cache Management
Non-locking read = stale data
Trying to eliminate this issue can lead tomajor performance penalties
Addressing stale cache
Configurable cache life-cycle
Refresh using query when required
Use optimistic lock value to avoid extra refresh
Avoid turning off the cache
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Distributed/CoordinatedCaching
Following successful database commit
Changes are merged into shared cache
Changes messaged to clustered sessions
Ensure messaging is warranted
Fail-over
Shared objects between users
Server
Shared
Cache
Server
Shared
CacheSession Bus
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
AgendaPersistence Architecture Overview
Performance Points:Models and Mapping
Inheritance
Locking
Querying
Lazy Reading
N+1 Reads
Projections
Transactions
Caching
Leveraging the Database and JDBC
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Leverage the RelationalDatabase
Schema adjustments
Indexes
Order of calls & columns
Stored Procedures
PK Sequencing (pre-allocation)
Hints in SQL
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Leverage JDBC Features
Prepared Statement Caching
Parameter binding
Large data element streaming
Result scrolling
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Work with Your DBA
Some ORM approaches ignore RDBMS best-practices
Don’t restrict the evolution of the relationalschema
Allow for the use of indices and storedprocedure where necessary
Leverage advanced features of the databaseHints in SQL
Custom types (ORM data types)
XML storage
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Summary
Know or learn your application data andusage
Know your ORM persistence layer
Leverage the database & JDBC capabilities
General Performance Tuning
Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Oracle Corporation
Donald Smith — High Performance Persistence Page
Q&A
Top Related