Java developers: make the database work for you

Lucas Jellema

AMIS

Plain JDBCIbatis, Spring

JPA (Hibernate)

EJB (CMP) WS*

JDBC

RDBMSRDBMS

“NO SQL”

Cache

Data Grid

Java Applications & Database

Position of Database

• It almost feels like “a necessary evil”• Database is abstracted away as much as

possible • It’s the persistent data store

– It does CRUD (Create, Retrieve, Update & Delete)

• What else could there be to it?

Database (Vendor) Independence

• Applications should not depend on a specific vendor’s database– Only use common functionality (available in ‘all’)– Do not leverage special features of any database

• Abstract database away through frameworks– Use generic and/or generated SQL

• Do as little as possible relating to the RDBMS – … even if the organization happens to have

enterprise editions and specialized db veterans

“We could also do that in the database”

• in the database? Huh?

RDBMSRDBMS ≈

Stored Procedures

• Stored Procedures executing procedural programming units– PL/SQL, Transact-SQL, SQL/PL,

SPL, pl/perl, pl/php, …

– Java

Stored Procedures

After the polarization (peak 2002)pragmatism struck…

• EJB 2.x => JPA and EJB 3.0 (JEE 5)• Consensus

– Leverage enterprise database for what it is good at (you pay for it, may as well use it)

– Most applications will only be used on one vendor’s database ever

– Even portable applications should still leverage database strengths

• Through generic APIs with database specific implementations

Project: VP - Rich UI, Complex Data Manipulation

JSF (Rich Faces)

SEAM

JPA (Hibernate)

Oracle RDBMSOracle

RDBMS

Team & Design that combines strengths of all technologies…

- Ease and Elegance of Implementation- Functionality (in an affordable way)- Productivity- Performance

JSF (Rich Faces)

SEAM

JPA (Hibernate)

Oracle RDBMSOracle RDBMS

Database Strengths

• Integrity• Fine grained (data) security and auditing• Data Retrieval

– joining tables together, leveraging indexes

– hierarchical, network-like traversals

– advanced analytics, historical queries, mining

• Aggregation and Sorting• Complex & Massive Data Manipulation

Zooming in – using Oracle

• Have to pick one• Largest market-share• Fairly representative (e.g. ANSI SQL)• The one I personally know best

Oracle RDBMSOracle RDBMS

Sample Data Model:Departments & Employees

Primary, Unique and ForeignKey Constraints

• Definition in Database is Declarative• Implementation is optimized• Imagine the programming and performance

cost of a middle tier based implementation

Database

RDBMS not always exclusively accessed through one Java API

SOA, ESB, WebServices

Data Replication &

Synchronization

Batch Bulk Processes Standard

Applications

LegacyApplications

Other data constraints

• Not Null• Data Type:

– string, numeric, date (time), xml– maximum length, integer/floating point

• Data Rules– COMM < 0.3 * SAL– COMM IS NULL or JOB = ‘SALESMAN’– MGR != EMPNO

• Implemented using Column Definitionsand Check Constraints

Database Triggers – decorating Data Manipulation

• Triggers execute before or after Insert, Update or Delete of database records

Employees

Before Insert trigger: sal=…

insert, update

, delete

Purpose of triggers

• Set default values on new records– if :new.job=‘SALESMAN’ then :new.comm = 1000

• Calculate & Derive values upon insert, update or delete

• Notify third parties of data manipulation• Perform complex validation on the data

changes applied by the transaction– Per Department: Max Salary < 1.8 * Average

– Per Manager: #subordinates < 15

JPA refreshing entities after triggers have applied new values

@Entity@Table(name = "EMP")public class Employee…@Column(name=“sal”)private Double salary

persist

Employees

Before Insert trigger: sal=…

@ReturnInsert //EclLnk

@Generated (value=GenerationTime.INSERT) // Hibernate

The far reaches of SQL vsthe limit(itation)s of JPQL

• Many Java ORM-frameworks – including JPA via Hibernate or EclipseLink – generate SQL– Usually “database independent” SQL

– By and large only leveraging the common functionality across databases

• As a consequence:– Many Java applications do not exploit the

wealth of (the SQL of) their databases

– And use what they do leverage in a what is possibly a suboptimal way

Aggregation & Rollup

• Data for reporting purposes can be prepared by database queries– Including aggregations

(max/min/avg/count/sum)

– and Sub Totals

– and Grand Total

– and String Aggregation

Sub and Grandtotals with Rollup

• Rollup instructs databaseto aggregate at every levelstarting from the right– deptno, job

– deptno

– (grand total)

• Also see:– Cube

– GroupingSets

Analytical Functions – spreadsheet-style row processing

• Analytical Functions allow SQL queries to perform inter-row comparison & aggregation

• For example: in a single query, for each employee– show salary rank in department and job

– show salary difference with colleague next higher in rank (on the list per department)

– show average salary in the department

– show csv list of colleagues in department

Analytical Functions - example

Flashback Query

select emp.*, dept.dnamefrom emp AS OF TIMESTAMP (SYSTIMESTAMP - INTERVAL '1' DAY) , deptwhere emp.deptno = dept.deptno

Flashback Versions

• Retrieve all states each record has been in– Every transaction that touched a row left a version

of it– Pseudocolumns: xid, operation, starttime, endtime

Employee Version-history with Analytical and Flashback

Trees

Trees

Retrieving Hierarchical data sets with single SQL statements

• Database has optimized algorithms– Starting at any node in the tree or network

– Drilling down to the specified number of levels

– Order siblings within parent

– Indicate leaf and parent nodes; detect cyclesEMPID ENAME MGR DEPTNO LEVEL--------------- ---------- ---------- ---------- ---------- 7839 KING 10 1 7698 BLAKE 7839 30 2 7499 ALLEN 7698 30 3 7900 JAMES 7698 30 3 7654 MARTIN 7698 30 3 7844 TURNER 7698 30 3 7521 WARD 7698 30 3 7782 CLARK 7839 10 2 7934 MILLER 7782 10 3

Encapsulate Database specific SQL in a View API

• Views – for encapsulation of data model, multi-table join, (advanced) SQL hiding, authorization rules– Note: a view looks like a table to the client

View

The read-only cursor API

• A Cursor is a reference to a query result set• Database can open a cursor

for a SQL query• And return it to the application

to fetch the rows from• Cursor == JDBC

ResultSet• A cursor can be

nested: containdetails …

Employees

Stored Procedure

Departments

cursor

JDBC ResultSetwhile rs.next { … }

Cursor for Master-Detail resultset

Stored Procedure

Using Complex Views for Hiding Legacy Data Models

Providing a ‘business object’ API

• DML API: a View – aided by an Instead Of trigger

• Insert of one new row inUSERS_VW (e.g. a JPApersist operation) can actually be four new records– USER, PERSON,

EMAIL_TYPEEMAIL_ADDRESS

USERS

PERSONS EMAIL_ADDRESSES

EMAIL_TYPE

USERS Instead Of DML trigger

*

* *

*

The Hollywood Principle: Query ResultSet Change Notification

Cache

Cache Refresh triggered by DB

PL/SQL

Cache

SQL query

Register

DatabaseChange

Notification

Oracle RDBMS

invokes Java Listener

with event details

Reaching out from the database

Database

Email conversations

Database receiving and sending emails – from people or applications

RESTful resource navigation

RESTful architecture

RESTful PL/SQL APIexposed through dbms_epg

http

http

http

JEE Application Server

Enterprise Service Bus

?

Database informing and leveraging the middle tier

HTTP calls using the

UTL_HTTP package

Other Database Features worth investigating

• Virtual Private Database & Fine Grained Authorization

• XMLType, XMLDB & FTP/HTTP/WEBDAV server

• Object Types and Collections

• Data type Interval & Time Zone support

• Fine Grained Auditing

• System Triggers, for example “after logon”

• (Global) Application Context

• Autonomous Transaction

• Advanced Queuing (& JMS interaction)

• Creating advanced job execution schedules

• Edition Based Redefinition (versioning of database objects)

• Statistics and Data Mining

• Virtual Columns

Summary & Conclusions

• Databases can do much more than• Java applications can benefit!• Strike the right balance:

– Leverage database forwhat it can do best

• Make Java and Database work together in a smooth way

Use the right tool for the job

• Render HTML• Enforce Application

Logic• Handle User

Interaction• Create graphics• Interact with Internet

• (bulk) copy of data• Guard

Uniqueness• (large) Sort or

Aggregation• (complex) SQL• Enforce data rules

Summary & Conclusions

• Databases can do much more than• Java applications can benefit!• Strike the right balance:• Make Java and Database work together• Cater for ‘multiple database

consumers’• Acquire expertise on your team• Arrive at architectural design choices and

best development practices

Best Practices & Principles

• Prevent data to travel to the middle tier unless it has to– Performance (network and object

instantiation) & Resource Usage (memory)

• When data is on the middle tier: ensure it has the required freshness

• Encapsulate database (specific) functionality– NO SQL (in the middle tier)

– Decoupling and database (vendor) & framework independence

Best Practices & Principles

• Use Views and Stored Procedures to create APIs that encapsulate database functionality– Note: the database brings constraints and

triggers to the party – weaved in like Aspects

– Cursors mapping to ResultSets allow retrieval of nested data structures through simple calls

• Leverage the database for what it’s worth• Include ‘database developer’ in your team• Never be dogmatic

Want to know more?

• Have the sources for the demos

• Have this presentation presented & discussed at your organization

• Learn about Java and the Database (Oracle)

• Inject (Oracle) Database expertise – in the context of Java development - into your team

• Receive a paper with more details on ‘making the database work for you & for ’

• Send me an email: lucas.jellema@amis.nl

• Visit our blog: http://technology.amis.nl/blog

Master Class ‘Java Developers make the database work for you’

• Friday 17 December 2010(AMIS, Nieuwegein):– One day master class:

‘Java Developer make the database work for you’

• For information and registration:– lucas.jellema@amis.nl