www.devoxx.com

Using content repositories: Nuxeo Core, JCR 2, CMIS

Florent GuillaumeHead of R&DNuxeo

www.devoxx.com

Overall presentation goal

Learn how to use content repositories APIsto store and manipulate rich documents

www.devoxx.com

Speaker’s qualificationsFlorent Guillaume is Head of R&D at Nuxeo, a leading vendor of Open Source ECM platforms

Florent Guillaume is a member of international normalization technical committees: JSR-283, Oasis CMIS

Florent Guillaume architected and wrote an ECM system (Nuxeo CPS) and is a lead architect on another (Nuxeo EP)

Florent Guillaume wrote a low-level storage engine for Nuxeo Core

3

www.devoxx.com

Store information

Attached file (binary stream)

Metadata

Retrieve information

Search

Security

Be visible and editable

Out of scope for a repository

4

What is a document?

www.devoxx.com

What is a “rich” document?Depends on your definition of “rich”

5

www.devoxx.com

What is a “rich” document?Depends on your definition of “rich”

5

www.devoxx.com

What is a “rich” document?Depends on your definition of “rich”

Associated binary streams

Multiple attached files

Vignettes

Renditions

Complex metadata

Lists of records

Sub-records (XML-like)

6

www.devoxx.com

What storage APIs?Filesystem

Relational database

JCR (JSR-170, JSR-283)

Nuxeo Core

CMIS

7

www.devoxx.com

FilesystemDecide where to store files

Decide how to uniquely identify files

Use java.io.FileOutputStream for binaries

Decide how to serialize metadata

All by hand

java.io.ObjectOutputStream, Serializable

etc.

8

www.devoxx.com

Read/write

Filesystem example

9

public void createDocument(Long id, String title) throws IOException { File file = new File(ROOT_DIR, id.toString()); OutputStream out = new FileOutputStream(file); try { out.write(title.getBytes("UTF-8")); } finally { out.close(); } }

public MyDocument getDocument(Long id) throws IOException { File file = new File(ROOT_DIR, id.toString()); InputStream in = new FileInputStream(file); try { byte[] bytes = new byte[100]; int n = in.read(bytes); String title = new String(bytes, "UTF-8"); MyDocument doc = new MyDocument(); doc.setId(id); doc.setTitle(title); return doc; } finally { in.close(); } }

www.devoxx.com

Filesystem drawbacksSimplistic, no actual model

Everything has to be done manually

No search

10

www.devoxx.com

Filesystem using JAXBMap Java objects to/from XML representations

Define a schema (model)

Marshall/Unmarshall using JAXB APIs

11

www.devoxx.com

Relational databaseJDBC

Hibernate

...other ORMs

JPA

12

www.devoxx.com

JDBCDefine a SQL model for your data

Tables, Columns, Data types

Decide where to store BLOBs

Column or filesystem

Emit SQL statements to do all read/write operations

13

www.devoxx.com

JDBC exampleClass defining the model

14

class MyDocument {

private Long id;

private String title;

public Long getId() { return id; } public void setId(Long id) { this.id = id; }

public String getTitle() { return title; } public void setTitle(String title) { this.title = title; }}

www.devoxx.com

Establish a connection, Write

JDBC example

15

public void init() throws ClassNotFoundException { Class.forName("com.myvendor.TheDriverClass"); } public Connection getConnection() throws SQLException { return DriverManager.getConnection("jdbc:myvendor:/my/database/info", "myLogin", "myPassword"); }

public void createDocument(Long id, String title) throws SQLException { PreparedStatement ps = null; Connection conn = getConnection(); try { ps = conn.prepareStatement("INSERT INTO MyDocuments (id, title) VALUES (?, ?)"); ps.setLong(1, id); ps.setString(2, title); ps.execute(); } finally { if (ps != null) ps.close(); conn.close(); } }

www.devoxx.com

Read

JDBC example

16

public MyDocument getDocument(Long id) throws SQLException { PreparedStatement ps = null; Connection conn = getConnection(); try { ps = conn.prepareStatement("SELECT title FROM MyDocuments WHERE id = ?"); ps.setLong(1, id); ResultSet rs = ps.executeQuery(); if (!rs.next()) { return null; } String title = rs.getString(1); MyDocument doc = new MyDocument(); doc.setId(id); doc.setTitle(title); return doc; } finally { if (ps != null) ps.close(); conn.close(); } }

www.devoxx.com

JDBC drawbacksStill very much manual

No document abstraction

Reads from ResultSets have to know proper types

17

www.devoxx.com

HibernateModel your documents as classes

Define an object-relational mapping in XML

Use Hibernate to automate read/writes

18

www.devoxx.com

Hibernate exampleXML configuration

19

<hibernate-mapping> <class name="example.MyDocument" table="DOCUMENTS"> <id name="id" column="DOC_ID"> <generator class="native"/> </id> <property name="title"/> </class></hibernate-mapping>

www.devoxx.com

Hibernate exampleRead/write

20

class DocumentManager {

public void createDocument(Long id, String title) { Session session = HibernateUtil.getSessionFactory().getCurrentSession(); session.beginTransaction(); MyDocument doc = new MyDocument(); doc.setId(id); doc.setTitle(title); session.save(doc); session.getTransaction().commit(); session.close(); }

public MyDocument getDocument(Long id) { Session session = HibernateUtil.getSessionFactory().getCurrentSession(); session.beginTransaction(); Query query = session.createQuery("FROM MyDocument WHERE id = :id"); query.setParameter("id", id); MyDocument doc = (MyDocument) query.uniqueResult(); session.getTransaction().commit(); session.close(); return doc; }}

www.devoxx.com

Hibernate drawbacksDocument classes have to be defined by hand

No standard, just application-defined classes

Object-relational mapping too flexible

Too much choice can be a curse

Binaries are still a problem (no streaming)

Hibernate’s “binary” is a byte[] → Memory hog

Blob support inconsistent

21

www.devoxx.com

JPAModel your documents as classes

Define an object-relational mapping using annotations

Use JPA to automate read/writes

22

www.devoxx.com

JPA exampleClass defining the model and the mapping

23

@Entityclass MyDocument {

private Long id;

private String title;

@Id public Long getId() { return id; } public void setId(Long id) { this.id = id; }

public String getTitle() { return title; } public void setTitle(String title) { this.title = title; }}

www.devoxx.com

JPA exampleRead/write

24

@Statelessclass DocumentManagerBean {

@PersistenceContext EntityManager em;

public void createDocument(Long id, String title) { MyDocument doc = new MyDocument(); doc.setId(id); doc.setTitle(title); em.persist(doc); }

public MyDocument getDocument(Long id) { return em.find(MyDocument.class, id); }

public List<MyDocument> listDocuments() { Query query = em.createQuery("SELECT doc FROM MyDocument doc"); return query.getResultList(); }}

www.devoxx.com

JPA drawbacksSame as Hibernate

Although annotations are very convenient

25

www.devoxx.com

Beyond mere storage

www.devoxx.com

Beyond mere storageStandard document abstraction

Security

Locking

Versioning

Full text search

Types and inheritance

Folders? ordering?

27

www.devoxx.com

JCRContent Repository API for Java™ Technology

JSR-170, released in June 2005

Initiated by Day Software

Also BEA, Documentum, FileNet, IBM, Oracle, Vignette and others

Apache Jackrabbit is the RI

28

www.devoxx.com

JCR goalsJava API

Fine-grained, hierarchical storage model

Be the “SQL” of hierarchical storage

Lots of functionality

29

www.devoxx.com

JCR featuresCRUD

Hierarchy of nodes

Simple properties, Lists, Binaries

Queries

Versioning, Locking, References, ...

30

www.devoxx.com

JCR 2JSR-283

First public review July 2007

Final release expected early 2009

Nuxeo is a contributor to the specification

31

www.devoxx.com

JCR 2 featuresFix JSR-170 inconsistencies

Several compliance levels

New property types

Improved features

Versioning, Access control, Observation

Retention & Hold

Shareable nodes

Java query API

32

www.devoxx.com

JCR – nodes and properties

33

Node Type

foo

Property Type

value

bar

childchild

www.devoxx.com

JCR – node hierarchy

34

(root)

Folder Folder

Document

Document

Folder

foo bar gee

File

docthing

vignette

www.devoxx.com

JCR – properties types

35

String

Binary

Date

Long

Double

Boolean

Name

Path

Reference

JCR 2

Decimal

WeakReference

URI

www.devoxx.com

JCR – hierarchy with properties

36

(root)

Folder Folder

Document

Document

String

my doc

String

...

Folder

foo bar gee

Date

2008-12-11

String

florentFile

String

img.png

Binary

<binary>

docthing

title descriptiondate creatorvignette

filename data

www.devoxx.com

JCR – node hierarchy

37

(root)

Folder Folder

Document

Document

title: my doc

description: ...

date: 2008-12-11

creator: florent

Folder

foo bar gee

File

filename: img.png

date: <binary>

docthing

vignette

www.devoxx.com

JCR – typesDefine the model

38

<my='http://my.example.com/ns'>

[my:document]- my:id (long) mandatory- my:title- my:description- my:creator- my:date (date)+ my:vignette (nt:resource)

www.devoxx.com

JCR – APIEstablish a connection

39

public Repository repository;

public javax.jcr.Session session;

public void init() throws IOException { repository = new TransientRepository(); }

public void open() throws LoginException, RepositoryException { Credentials credentials = new SimpleCredentials("username", "password".toCharArray()); session = repository.login(credentials); }

public void close() { session.logout(); }

www.devoxx.com

JCR – APIRead/write

40

public void createDocument(Long id, String title) throws RepositoryException { Node root = session.getRootNode(); Node doc = root.addNode(id.toString()); doc.setProperty("my:id", id); doc.setProperty("my:title", title); session.save(); }

public Node getDocument(Long id) throws RepositoryException { Node root = session.getRootNode(); Node doc = root.getNode(id.toString()); return doc; }

www.devoxx.com

JCR – APIQuery

41

public List<Node> getDocuments() throws RepositoryException { QueryManager queryManager = session.getWorkspace().getQueryManager(); Query query = queryManager.createQuery( "//element(*, my:document)", Query.XPATH); query = queryManager.createQuery( "SELECT * from my:document", Query.SQL); List<Node> documents = new LinkedList<Node>(); NodeIterator it = query.execute().getNodes(); while (it.hasNext()) { documents.add(it.nextNode()); } return documents; }

www.devoxx.com

NuxeoFounded in 2000

Sustained growth for 8 years

Pioneering Open Source ECM software vendor

International organization, customers, partners, community

40+ employees

Business oriented Open Source

42

www.devoxx.com

Nuxeo ECM3+ years of work

Focus on the platform

Document Core

Nuxeo EP, Nuxeo WebEngine, Nuxeo RCP

Components everywhere (OSGi)

Large feature set

Big deployments

43

www.devoxx.com

Nuxeo CoreHigh-level document-oriented Java API

Complex document abstraction

Independent of actual storage backend

EJB remoting

REST bindings (JAX-RS)

SOAP bindings (JAX-WS)

44

www.devoxx.com

Nuxeo Core basicsCRUD

Hierarchy of document

Complex properties

Binaries

Security

Locking

Versioning

Publishing, Proxies

45

www.devoxx.com

Nuxeo CoreDefine the model

46

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:nxs="http://www.nuxeo.org/ecm/schemas/mydocument" targetNamespace="http://www.nuxeo.org/ecm/schemas/mydocument">

<xs:include schemaLocation="core-types.xsd"/>

<xs:element name="id" type="xs:integer"/> <xs:element name="title" type="xs:string"/> <xs:element name="description" type="xs:string"/> <xs:element name="creator" type="xs:string"/> <xs:element name="date" type="xs:date"/> <xs:element name="vignette" type="nxs:content"/> <xs:element name="fullname" type="nxs:fullname"/>

<xs:complexType name="fullname"> <xs:sequence> <xs:element name="firstname" type="xs:string"/> <xs:element name="lastname" type="xs:string"/> </xs:sequence> </xs:complexType>

</xs:schema>

www.devoxx.com

Nuxeo CoreDefine the model

47

<extension target="org.nuxeo.ecm.core.schema.TypeService" point="schema"> <schema name="mydocument" src="schemas/mydocument.xsd" prefix="mydoc"/></extension>

<extension target="org.nuxeo.ecm.core.schema.TypeService" point="doctype"> <doctype name="MyDocument" extends="Document"> <schema name="common"/> <schema name="mydocument"/> <facet name="Versionable"/> </doctype></extension>

www.devoxx.com

Nuxe Core – APIRead/Write

48

@Scope(ScopeType.STATELESS)@Name("myDocumentActions")public class DocumentActionsBean {

@In(create = true) protected transient NavigationContext navigationContext;

@In(create = true) protected transient CoreSession documentManager;

public void createDocument(Long id, String title) throws ClientException { DocumentModel current = navigationContext.getCurrentDocument(); DocumentModel doc = documentManager.createDocumentModel( current.getPathAsString(), null, "MyDocument"); doc.setProperty("mydocument", "id", id); doc.setPropertyValue("mydoc:title", title); doc.setPropertyValue("mydoc:fullname/firstname", "Florent"); documentManager.createDocument(doc); documentManager.save(); }}

www.devoxx.com

CMISDraft v 0.5 published in September 2008 by EMC, IBM, Microsoft

Alfresco, Open Text, Oracle, SAP also on board from the start

Oasis TC formed in November 2008

Adullact, Booz Allen Hamilton, Day, Ektron, Exalead, Fidelity, Flatirons, Magnolia, Mitre, Nuxeo, Saperion, Sun, Vamosa, Vignette (as of 2008-12-01)

CMIS 1.0 expected mid-2009

49

www.devoxx.com

CMIS goalsSimple document model

Independent of protocol

SOAP, REST (AtomPub) bindings

Not tied to a programming language

Platform, vendor independent

Basic set of ECM functions

“Greatest common denominator”

50

www.devoxx.com

CMIS basicsCRUD

Hierarchy folders, documents

Simple properties, lists

One binary

Policies

Versioning

Relationships

Queries

51

www.devoxx.com

CMIS advancedMulti-filing

Advanced queries

Joins

Full text

... maybe more to come

52

www.devoxx.com

CMIS basic propertiesObjectId

Uri

ObjectTypeId

CreatedBy

CreationDate

LastModifiedBy

LastModificationDate

ChangeToken

Name (doc, folder)

version-related props (doc)

ParentId (folder)

AllowedChildObjectTypeIds (folder)

SourceId (rel)

TargetId (rel)

PolicyName (policy)

PolicyText (policy)53

www.devoxx.com

CMIS objects

54

Folder DocumentRelationship Policy

www.devoxx.com

CMIS folders

55

Folder(root)

Folderfoo

Folderbar

Folderstuff

Foldergee

Folderblah

www.devoxx.com

CMIS documents

56

Folder(root)

Folderfoo

Folderbar

Folderstuff

Foldergee

Folderblah

Doc123

Doc456

Doc789

Doc001

www.devoxx.com

CMIS relationships

57

Folder

(root)

Folder

foo

Folder

bar

Folder

stuff

Folder

blah

Doc

456

Doc

001

555

333

Doc

123

www.devoxx.com

CMIS policies

58

Folder(root)

Folderfoo

Folderbar

Doc123

Doc001

Policy

PolicyPolicy

www.devoxx.com

CMIS policies uses

59

ACLs

Locks

Retention & Hold

Automatic transformations

Repository hints

etc.

www.devoxx.com

CMIS queries

60

www.devoxx.com

Standard SQL-92

Extensions for multi-valued properties

Extensions for hierarchical searches

Extensions for fulltext search

CMIS query example

61

SELECT OBJECT_ID, SCORE() AS SC, DESTINATION, DEPARTURE_DATES FROM TRAVEL_BROCHURE WHERE IN_TREE( , ‘ID00093854763’) AND CONTAINS( , 'PARADISE ISLAND CRUISE') AND '2010-01-01' < ANY DEPARTURE_DATES AND CONTINENT <> 'ANTARCTICA'ORDER BY SC DESC

www.devoxx.com

Additional headers for behavior control

MIME types

CMIS AtomPub bindings

62

Service application/atomsvc+xml

Feed application/atom+xml;type=feed

Entry application/atom+xml;type=entry

Query application/cmisquery+xml

AllowableActions application/cmisallowableactions+xml

www.devoxx.com

CMIS Web Services bindings

63

All the WSDL files are provided

Check the spec ;-)

www.devoxx.com

CMIS repository servicesgetRepositories

getRepositoryInfo

getTypes

getTypeDefinition

64

www.devoxx.com

CMIS navigation servicesgetDescendants

getChildren

getFolderParent

getObjectParents

getCheckedoutDocuments

65

www.devoxx.com

CMIS object servicescreateDocument

createFolder

createRelationship

createPolicy

getAllowableActions

getProperties, updateProperties

getContentStream, setContentStream, deleteContentStream

moveObject, deleteObject, deleteTree66

www.devoxx.com

CMIS multi-filing servicesaddObjectToFolder

removeObjectFromFolder

67

www.devoxx.com

CMIS discovery servicequery

68

www.devoxx.com

CMIS relationships servicesgetRelationships

69

www.devoxx.com

CMIS policy servicesapplyPolicy

removePolicy

getAppliedPolicies

70

www.devoxx.com

CMIS versioning servicecheckOut

cancelCheckOut

checkIn

getPropertiesOfLatestVersion

getAllVersions

deleteAllVersions

71

www.devoxx.com

Mapping documents to JCRDocument → Node

Schema → Mixin type

Simple & Array properties → Properties

Complex type → Sub-node

Lists of complex types → Ordered sub-nodes

72

www.devoxx.com

Jackrabbit tables

73

www.devoxx.com

Mapping documents to SQLParent-child relationships → Dedicated “hierarchy” table

Schema → Table

Simple property → Column

Array property → “Collection” table

Complex type → Sub-document

Lists of complex types → Ordered sub-documents

74

www.devoxx.com

Visible SQL storage

75

www.devoxx.com

Visible SQL storage

76

www.devoxx.com

Nuxeo Core 2 and CMISNext-generation storage based on CMIS model

No “impedance mismatch” in models

Nuxeo extensions if needed

Leverage the Visible SQL Storage backend

Distributed and clusterable

Faster remote access and caching

True clusters

Facilitate cloud-based backends

77

www.devoxx.com

SummaryMany storage choices

Define your model, choose the right API

JCR is very capable

CMIS is coming fast

Nuxeo gives the best of both

78

www.devoxx.com

Concluding statement

Look for CMIS soon, and check out Nuxeo now!

www.devoxx.com

Q&A

Questions?

www.devoxx.com

Thank you for your attention!

http://www.nuxeo.comhttp://doc.nuxeo.org

http://jcp.org/en/jsr/detail?id=170http://jcp.org/en/jsr/detail?id=283

http://jackrabbit.apache.org/http://www.oasis-open.org/committees/cmis/