There and Back Again As Quick As a Flash: Tuning ... · Hibernate (MySQL) MongoDB (net) DB4O...

There and Back AgainAs Quick As a Flash:

Tuning Performance in CDOStefan Winkler

Independent Software Developer and IT ConsultantCDO Committer

[email protected]

@xpomul on Twitter

Note: Extended version of these slides available on the ECE page of this talk!

1Dienstag, 23. Oktober 12

mailto:[email protected]

mailto:[email protected]

CDO in a nutshell

Server

Client

Client

Client

Storage


CDO in a nutshell

Server

Client

Client

Client

Storage

CDOID

attribute

reference

CDOID


CDO in a nutshell

Server

Client

Client

Client

Storage

CDOID

attribute

reference

CDOID

CDOID


CDO in a nutshell

Server

Client

Client

Client

Storage

CDOID

attribute

reference

CDOID


Storage

CDO in a nutshell

Server

Client

Client

Client

Storage

CDOID

attribute

reference

CDOID


Storage

CDO in a nutshell

Server

Client

Client

Client

Storage

attribute

reference

CDOID

attribute

reference


Storage

CDO in a nutshell

Server

Client

Client

Client

Storage

attribute

reference

CDOID

CDOID

attribute

reference


Storage

CDO in a nutshell

Server

Client

Client

Client

Storage

attribute

reference

CDOIDCDOID

attribute

reference


Storage

CDO in a nutshell

Server

Client

Client

Client

Storage

attribute

reference

CDOID

CDOID

attribute

reference


Layers

AppLogic

CDO Client

EMFModel

CDOServer

IStoreCode

Storage


AppLogic

CDO Client

EMFModel

CDOServer

IStoreCode

Storage


Choose the Right Storage!

MEM

Lissome

DB (h2 in-proc)

DB (h2 net)

DB (MySQL)

Hibernate (MySQL)

MongoDB (net)

DB4O (in-proc)

0 200.000 400.000 600.000

1.701

519.063

22.314

9.381

4.269

1.486

1.475

1.121

time to read 10,000 objects (msec)

0 2.500 5.000 7.500 10.000

2.268

1.503

10.151

6.686

5.546

721

431

283

time to write 10,000 objects (msec)


Choose the Right Storage!

Store Usage Scenarios Pros Cons

MEMStore prototyping, testing supports all CDO features

not persistent

LissomeStore custom CDO file-based store, optimized for performance

performance, full support planned

custom file format (no external access)

DBStore common/default solution supports all CDO features

DB selection and tuning adds to complexity

HibernateStore custom DB schema required custom DB schema and mapping

only supports subset of CDO features, slower

MongoDB Store No SQL DB wanted,fast writing speeds required

fast writing speeds slow reading speeds,limited commit size

DB4OStore No SQL DB wanted,fast reading speeds required

fast reading speeds slower writing speedsdual licence

ObjectivityStore OODB with commercial support needed

performance ?(not measured)

commercial licence, performance not measured


Choose the Right Database!

h2 (net)

MySQL (net)

PostgreSQL (net)

0 2.500 5.000 7.500 10.000

4.231

9.381

4.269

time to read 10,000 objects

5.000 5.833 6.667 7.500

6.606

6.686

5.546




h2 (net)

MySQL (net)

PostgreSQL (net)

0 2.500 5.000 7.500 10.000

4.231

9.381

4.269


5.000 5.833 6.667 7.500

6.606

6.686

5.546


Standard Configurations !!!



h2 (net)

MySQL (net)

PostgreSQL (net)

0 2.500 5.000 7.500 10.000

4.231

9.381

4.269


5.000 5.833 6.667 7.500

6.606

6.686

5.546


Standard Configurations !!!

... and tweak it!


AppLogic

CDO Client

EMFModel

CDOServer

IStoreCode

Storage


Choose the Right Repository Mode!

Time

v1 v2 v3 v4 v5 v6 v7

Audit Mode



Time

v1 v2 v3 v4 v5 v6 v7

Non-Audit Mode



Time

0:v1 v2 v3 v4 v5 v6Branching Mode

1:v1

2:v1

1:v2

2:v2 2:v3


Choose the Right DBStore Mapping!

•Audit:

• plain vs. range-based list mapping

•Branching:

• plain vs. range-based list mapping

• copy-lazy vs. copy-on-write list branching



datesum

Orderdatesum

Productdatesum

Productnumberprice

Product

OrderOrderOrderOrder

id version date sum

1 2 3/29/12 102.34

ProductProductProductProduct

id version number price

2 2 1-423-4 39.12

3 2 1-485-3 60.71

4 2 1-584-2 2.51

Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List

id version index value

1 2 0 2

1 2 1 3

1 2 2 4

• non-audit repo mode

• no historic („audit“) views

• writes deltas




id version date sum

1 2 3/29/12 102.34



2 2 1-423-4 39.12

3 2 1-485-3 60.71

4 2 1-584-2 2.51

Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List

id version index value

1 2 0 2

1 2 1 3

1 2 2 4

• audit repo mode

• classic

• always writes complete list

• range-based

• writes deltas




id version date sum

1 2 3/29/12 102.34



2 2 1-423-4 39.12

3 2 1-485-3 60.71

4 2 1-584-2 2.51

• audit repo mode

• classic

• always writes complete list

• range-based

• writes deltas

Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List

id from to index value

1 1 null 0 2

1 1 1 1 4

1 2 null 1 3

1 1 1 2 3

1 2 null 2 4



• branching repo mode

• classic vs. range-based

• copy-lazy (default) vs. copy-on-branch

Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List

id branch from to index value

1 1 1 null 0 2

1 1 1 1 1 4

1 1 2 null 1 3

1 1 1 1 2 3

1 1 2 null 2 4








1 1 1 null 0 2

1 1 1 1 1 4

1 1 2 null 1 3

1 1 1 1 2 3

1 1 2 null 2 4



1 2 1 null 1 5








1 1 1 null 0 2

1 1 1 1 1 4

1 1 2 null 1 3

1 1 1 1 2 3

1 1 2 null 2 4








1 1 1 null 0 2

1 1 1 1 1 4

1 1 2 null 1 3

1 1 1 1 2 3

1 1 2 null 2 4



1 2 1 null 0 2

1 2 1 null 1 5

1 2 1 null 2 4



0

2.250

4.500

6.750

9.000

non-audit audit branching branching w/ ranges

721

1.4911.481

688

4.581

1.4731.483

736

8.941

1.5061.442

681

time to insert 100 objects at beginningtime to insert 100 objects in the middletime to insert 100 objects at the end


AppLogic

CDO Client

EMFModel

CDOServer

IStoreCode

Storage


Choose the Right Transport Layer!

CDO Client

CDOServer

CDO Client

CDOServer

Computer 1 Computer 2

TCP (true client/server)

TCP (same computer)


Choose the Right Transport Layer!

CDO Client

CDOServer

CDO Client

CDOServer

CDO Client

CDOServer

Computer 1 Computer 2

TCP (true client/server)

TCP (same computer)

JVM (same JVM, direct calls)


A Word On Caching

• Revision Caches in CDO use Soft references

• Caches grow until memory limit is reached

• View Cache can be configured to use strong or weak references instead

CDO Client

CDOServer

ViewCache

(Soft Ref)

SessionCache

(Soft Ref)

RepositoryCache

(Soft Ref)

view.options().setCacheReferenceType(ReferenceType.STRONG);


AppLogic

CDO Client

EMFModel

CDOServer

IStoreCode

Storage


Choose the Right Model Configuration!

Native

Reflective

Legacy

0 3.000 6.000 9.000 12.000

11.302

8.882

8.581


0 2.500 5.000 7.500

4.460

3.076

2.802



AppLogic

CDO Client

EMFModel

CDOServer

IStoreCode

Storage


Make Use of CDO Loading Policies!

• Lazy Collection Loading

session.options().setCollectionLoadingPolicy( CDOUtil.createCollectionLoadingPolicy( /* initial = */ 5, /* onResolve = */ 5));

• Explicit Prefetching

CDOObject fromCDO = getMyObjectFromCDO();fromCDO.cdoPrefetch(/* depth = */ 3);

• Prefetching by Policy

view.options().setRevisionPrefetchingPolicy( CDOUtil.createRevisionPrefetchingPolicy( /* size = */ 5));


Leftovers

• Things I did not talk about

• unordered collections

• store queries (use with caution!)

• LOBs & streaming


Summary• Choose the Right Storage!

• Choose the Right Database!

• Choose the Right Repository Mode!

• Choose the Right DBStore Mapping!

• Choose the Right Transport Layer!

• Choose the Right Model Configuration!

• Make Use of CDO Loading Policies!

More of CDO: Now that I‘ve got my model - where is my Application? Thu 10:30-11:30

Newsgroup: eclipse.tools.emf


There and Back Again As Quick As a Flash: Tuning ... · Hibernate (MySQL) MongoDB (net) DB4O...

Documents

Transcript of There and Back Again As Quick As a Flash: Tuning ... · Hibernate (MySQL) MongoDB (net) DB4O...