TransWarp and the Future of ZPatterns

IPC 9 Zope Track

Ambiguous Praise For ZPatterns

"... if I could understand what it does I'd probably use it." - Brad Clements

"... everytime I've come across a technology I had trouble understanding, it turned out to be a worthless technology or a good one where the authors had trouble explaining their stuff. For the time being, I've placed ZPatterns in the latter category" - Cees de Groot

Praise For ZPatterns at IPC 9:

“It’s certainly made my life more… interesting.” – Steve Spicklemire

“We had been taught the value of commenting code in our computer classes, but this experience [with ZPatterns] really made us see why it is important to do so.”– Lex Berezhny & Jason Straw

Issues with ZPatterns

Difficult to teach/grasp Tedious development: same names retyped

– As ZClass properties– As SQL fields in queries and DDL– As form text and field names– In SkinScript– etc.

Inherently a hack, promoting further hacks ad infinitum

Our Quest for Automation

Write once, use many Metadata was the key UML is standard for metadata, XMI for

interchange of UML But no UML or XMI tools for Python Code generation from XSLT was ridiculous Had to create our own system

Introduction

Components: The Future (And Present)

Of Programming

Where did ZPatterns come from?

Ty and PJE needed a component model for Zopelications

Zope component capabilities were poor or non-existent for:– Distributing components without upgrade breakage– Configuring components to work with components

outside their own “product”– Separating data storage from logic and UI code

Concepts/Assumptions

Objects are domain objects, representing real-world things

Real-world things have data stored about them in many databases

For a given piece of data, there may be many “conceptual” objects

Thus, a useful framework must allow many-to-many mappings between physically stored data and conceptual objects

Therefore…

An object’s attributes should be able to be stored in any database, without forced one-to-one mappings

There should be a way to create or access instances of objects, knowing only their interface, not their implementation class or storage mechanisms

The RIPP Model

Not tool-specific or implementation specific, but a generally reusable architecture pattern

A way of structuring Zope or ZPublisher applications as:– Reusable, configurable components, with– Separation of application logic from storage and user

interface code, and– Separation of implementation class choices (variable)

from application roles (fixed)

RIPP Overview

I_Document

I_Category

DTML, etc.

I_Author

DTML, etc.

DTML, etc.

Component Architectures

Design Patterns/Idioms are key to an effective component architecture:– What components should exist?– How is the work divided among them?– How are they put together to form larger

structures?

The RIPP patterns are a first step in this direction - TransWarp is the next

TransWarp: It’s Not Just Components Any More

(It’s component architectures, which means patterns, which means

repetition, which means automation!)

The “Why” of TransWarp

“Easy as 1-2-3”– Draw applications with UML

– Add application-domain logic

– Mix with implementation frameworks

+ =

The “What” of TransWarp

WarpCORE (SQL DBs)

Messaging, Distribution

User Interface (HTML, GUI, ?)

Aspects, FeatureDefs, & Transforms

"Structural Models"

UML & XMI

. . . ?

CASE

GP

Frameworks

AOP

What is AOP?

Separately Specified Areas of Concern– e.g. Persistence, User interface, domain logic

Ability to “weave” these areas into a single program

Modular replacement of implementation for each area in a given application

Generative Programming (GP)

The Automation Assumption:

“If you can compose components manually, you can also automate this process.”

– Czarnecki and Eisenecker, “Components and Generative Programming”

Plan + Parts = Products

=

=

=

=

+

+

+

+

“Structural Aspect”

The aspect of a component that describes its class hierarchy and “structural features”

Structural feature = Attribute, Association Can be derived directly from UML model Reusable with different implementations of

domain logic, storage, user interface

Structural Models

Structural model provides implementation for a structural aspect

Defines classes to be used for “attribute”, “associationEnd”, etc.

Standardized storage interface for model updates and queries, including predicates

Based on the “Services, Elements, and Features” pattern

Services, Elements, And Features

Cool Tools

CASE, UML, XMI, SQL…It’s 100% Buzzword Compliant!

TransWarp CASE

UML Model Library– Built using a StructuralAspect representing the

UML MetaModel– Full StructuralModel features available:

• use predicate queries

• create new interfaces for UML model manipulation

• your choice of storage mechanisms

XMI Library

Objects + SQL = Warp CORE CORE = Customizable Object-Relational

Environment Pattern language for storing “business objects” in a

relational database Conceptually well-suited to RIPP model Handles aggregation, composition, temporal

relationships, inheritance, polymorphism Highly efficient design using few tables and well-

chosen indexes

CORE = A Design Pattern

Pattern language for database design, not persistence mechanism for arbitrary objects– (although you could implement one using

CORE as a base)

Not suitable for retrofitting existing databases - it is for designing new ones

Does not require any special tools - we’ve designed and built databases by hand with it

The CORE Tables

Using CORE with TransWarp

Generate databases from UML– Stereotypes and/or tagged values used to identify

event classes, other pragmas

Weave CORE storage aspect with structural model to create an app - no SQL or coding required for standard retrieves and updates

Put domain-specific queries (e.g. “getTasksForParty”) together in one aspect for easy reference/maintenance

So What’s Going ToHappen To ZPatterns?

(I’m so glad you asked me that… That’s an excellent question…)

Zope Directions

Component-driven architecture Interface-oriented More Python-centric Things are more “explicit” Content management framework to be

seperated from “pure” application server Less need for weird hacks

DataSkins: Stupid Zope Tricks

Implementing storage separation in a through-the-web configurable way turned out to be very hard in Zope

Zope 2 makes lots of internal assumptions:– ZODB-based persistence– Object-based (not action-based) transactions– Local roles stored as dictionary of lists– Standardized web-based management GUI

Whither goest ZPatterns?

Now on maintenance-only track Many former ZPatterns functions will

probably “disappear” into Zope core:– PlugIns may be replaced by a more interface-

savvy management UI– Factory mechanisms may allow straightforward

creation of objects without reference to a Specialist, allowing fewer Specialists to be used in many applications

Even if there is a “Z3Patterns”…

Will only implement RIPP model SkinScript, Providers may go away Primary development focus will be on

TransWarp, which will be much more useful and powerful for future Zopes

So When Can WeUse TransWarp?

(I had a feeling that you were going to ask me that…)

TransWarp Project Status

AOP tools available today GP, UML, and XMI tools being ported

from pre-AOP library WarpCORE patterns have been used for

production systems, but automation not developed.– Solid automation requires the GP, UML, and

XMI tools to exist first

Licensing & Contributions

License: BSD-style, minus any advertising or credit requirements. No warranty.

Contributions Wanted: “Horizontal” aspect libraries (UI, DB/persistence, logging, testing, documentation generation, code generation, etc.)

TransWarp Links

TransWarp Documentation Wiki http://www.zope.org/Members/pje/Wikis/TransWarp

TransWarp Software Releases http://www.zope.org/Members/pje/TransWarp/