Content Mirror

Plone Content Serialization to a Structured/Relational Database

 Who am I

Kapil Thangavelu

    Lives in Washington, DC

       Works with Python              Uses zope3, sqlalchemy, repoze, z3c

What is this talk about  

Rethinking How to Use PloneData Portability

Developer Productivity

What its not about

Using relational database with Plone

Product vs. Framework

2003, First Plone conference NOLA, paul everitt posed the question.

The Product Won

Plone does alot of out the box, with alot of knobs and customization possible through the web.

More Complexity

- Features are great if your using them.- Size Matters- Orthogonality Maters

Complexity Measured

"I have measured out my life in coffee spoons", The Love song of J Alrefd Prufrock, TS. Eliot

(1m x 40r x 5d x 50w )/60 = 166h

The Plone Tax, aka time spent waiting on restarts

Simplicity is Good

- Pay for What You Use

- We can build fast, interactive applications

- Developer Productivity and the Learning Curve Matters  - Simple things have a better chance of being correct

Separating Deployment

2am December 6th 2002, Amsterdam, infrae zope3 sprint

Static Deployment

CMFDeployment - Flexible Static Deployment, Filtering, transforms, deployment skins, rendering, urlresolution.

http://plone.org/products/cmfdeployment

From Plone 1.0 - Plone 3.1

Data Deployment

Entransit, provides data deployment, with features like rollback in a workspace based site structure.

.. requires particular site structure/layout according to workgroups, provides advanced features like rollback of a deployment.

Too Complex

Still too complex, needs to work out of the box.

operation and setup need to be dirt simple regardless of the internal complexity (like a product ;-)

ContentMirror

 "A facility for mirroring the content of a Plone site into a structured external datastore."  - automated, transparent, synchronous serialization.   - default source, Archetypes Content   - default target, Relational Databases

ContentMirror

Supports Plone 2.5 - Plone 3.1 Support different Databases ( tested with Postgresql & Mysql )

Work out of the Box

Strong Test Coverage

Easy to Extend

Bootstrapping

 - Load the Archetypes - Transform to Database Schema - Create Peer Class

Schema Transformationtransform archetypes schemas into relational tables Using SQLAlchemy Uses a default schema ( dublin core, relations, files, identity ) each type gets its own table with common fields factored out

Peer BasedEach content type's class has a corresponding, database persisted class (orm mapped), with corresponding attributes. The peer/domain class can be specified or autogenerated (default).

Event Based

In order to serialize content as changes are happening in the CMS, we integrate into the application server's event stream and subscribe to content events. The subscribers in turn create sync operations via an operation factory.

Files

Don't copy files without need Stored in the database by default

Containment

Adjacency List Model  Modified Preorder Tree TraversalLightweight Mapper for Site Maps (use eager join option )

Relations

Archetypes

Bidirectional

Generic Between Types

Custom Attributes

Content Mirror

Relations Table

Doesn't support custom attributes without customization.

Installation   <configure xmlns="http://namespaces.zope.org/zope"            xmlns:db="http://namespaces.objectrealms.net/rdb"           xmlns:ore="http://namespaces.objectrealms.net/mirror">     <!-- setup a database connection -->   <db:engine url="postgres://localhost/contentmirror"             name="mirror-db"             echo="True"/>   <!-- associate the connection to the db structures -->   <db:bind engine="mirror-db"           metadata="ore.contentmirror.schema.metadata" /></configure>

Finishing Installation

# generate a schema ./bin/zopectl run Products/ContentMirror/ddl.py postgres  # create the database./bin/createdb mirror-example # load up the schema./bin/psql mirror-example < db.sql # bulk load the existing content./bin/zopectl run Products/ContentMirror/bulk.py portal

Playing with Content Demo

 SQL Command Line

 Django Admin Interface

Resources

Source Code, Issue Tracker, Documentation http://code.google.com/p/contentmirror Mailing Listhttp://groups.google.com/group/contentmirror 99% unit test coverage / 48 tests / Doctests

Questions   

?