1006 Z2 Intro Complete

© ZFabrik Software KG 2010

The z2 Environment- introduction -


… the solution to humane life-cycle management for Java solutions:

Self-updating from source,

Cost-effective and highly productive,

Extremely easy to distribute,

Transparent for developers and supporters,

Auditable and versioned,

Standards friendly and extensible

The z2 Environment is...


Motivation

Operating a continuous integration infrastructure is complex and costly

Deployment of application servers and applications is complex and time consuming - developers tend to delay integration

Operating developer build infrastructure is error-prone, complex, and hard to debug

So Don't Do It!


Runtime is fully capable of preparing all source code and configuration for execution – no external toolset required.

Re-use of standard versioned store that holds everything that defines a system's content – no external configuration required.

Store- and file system based integration with development tools – no complex development tool integration required.

Approach


MachineMachineMachine

Architecture: Deployment

Many <home>s share one root component repository - usually implemented over a subversion repository

Every <home> runs a <home> Java process that manages one or more worker Java processes, as defined by a home layout

Worker processes run whatever participates in their target states

One <home> may run over several component repositories, in particular a dev repo that is defined over a workspace folder structure

<home>

Worker 1 Worker 2

manage

Check for updates

Repository

e.g. subversion, a workspace,...


Cus

tom

/Ext

ensi

ble

z2 im

ple

men

tatio

n

Architecture: In-process

Servlet/JSP

Jetty

Ker

nel (

bina

ry)

Resource Management

Component Model &Component Factories

Pro

gra

mm

ing

Mod

els

& C

onve

n tio

ns

JRE

App

licat

ions

DB Migration

Migrate4j

Java Builder

ECJ

SVN Repo

SVNKit

JNDI Naming

OSGi Bundles

Equinox

Timer/Jobs

Worker Processes

Data Sources

System States


Components in z2

The component concept captures anything stored in the repositories that z2 understands at runtime, for example

Web Applications

Worker process configurations

Java modules

System states

At runtime, in particular during development, changes in the repositories translate to component state invalidations (e.g. unloading of a Web application)

Repository

C1 C2

Cn

Worker Process

Componentsas defined inthe repo

C1C1C1C1

C1C1C1

C2

C1C1C1C1Cn

Web App

Java Classes

JDBC connection pool

depends

e.g. a Web App

e.g. Java e.g. a datasource

corresponds

Connection props read from C2

Connection props read from C2

Source files readfrom C1 and compiled

Source files readfrom C1 and compiled

Web resource in WebContent folder of Cn

Web resource in WebContent folder of Cn


Synchronization is about letting go...

When synchronizing, changes in the repository (since the last synchronization) are detected and translated into component invalidations

Invalidations observe declared and implicit dependencies

Invalidations are orchestrated by the <home> process, so that the life-cycle of worker processes is also subject to change detection

After invalidation, the <home> process will try to have all worker processes attain their target states again

Repository

C1 C2

Cn

Worker Process

Componentsas defined inthe repo

C1C1C1C1

C1C1C1

C1

C1C1C1C1Cn

Web App

Java Classes

JDBC connection pool

depends

e.g. a Web App

e.g. Java e.g. a datasource

1

2

3

Example: Detection of modification triggers

invalidation chain

Example: Detection of modification triggers

invalidation chain


System States

System states abstract operational modes of a system into configuration objects

Examples are:

de.travelhands/up → Start everything needed for the Travelhands front end

environment/jobWorkerUp→ target state for batch jobs

Components (in particular system states) can participate in states and depend on states: “run” before attaining a state, or a state must be attained before the component “runs” resp.

state

de.travelhands/up

state

environment/up

state

de.travelhands/up_db

state

de.travelhands/up_dev

depends

participates

part

icip

ates

schema

de.travelhands/tenantDB

participates

web

de.travelhands.app/web

participates

web

de.travelhands.admin/web_tests

participates

Example: Travelhands state graph

Example: Travelhands state graph


Scenarios...


<home>* Installation and Operation

Best practice: Keep bootstrapping core distribution in same repository as the base system

Install and update <home> by issuing simple Subversion client commands

Core installation is a matter of seconds

All further configuration is stored in Subversion and loaded on-demand

As all configuration and code is in a versioned store, configuration changes are tracked, secured and can be reverted at any time.

Checked in core dist(~7 MB)

Checked in core dist(~7 MB)

svn co …svn up ...svn co …svn up ...

<home> 1 <home> 2 <home> 3


Development & Modification

Have a running local <home> for testing and provisioning of binary

Check out projects (say A) to modify from the Subversion repo into an Eclipse Workspace

Use the Eclipsoid plugin to “arm” those projects and to satisfy all project dependencies

Keep modifying, sync, and test until done.

Note: There is no deployment

Commit or revert changes

“Disarm” projects. Done

check

outA

A' sync A from here

sync all but A from here

Project inEclipse workspace

<home>

Sourcerepo


SCMSCM

SCM

System Creation and Cloning

A system is completely defined (incl. configuration) by a repository URL (and hence a repository)

New systems are created by basic repository operations (e.g. branch)

When cloning, only essential interfaces with the outer world, e.g. database URLs, must be adapted

Every system constitutes a complete development branch

Updates are promoted between repositories via Transports

Shipping means to deliver a repository dump

branch

copyOR


Transport

Transports work via tool-supported creation of delta-workspaces that allow reviewing of changes before committing to the transport target (in particular when considering configuration)

Transports promote changes between repositories (and hence systems)

A standard transport route is helpful but not restrictive: cross-transport are a necessary reality

Transports can be used for remote upgrades as well

prepare

commit

revi

ew

prepare

commit

revi

ew


Support

As the runtime is a faithful representation of its repository, there is generally no doubt about sources when debugging

Since adding another <home> is trivial, debugging without interfering with production runtimes is easily achieved

As every system is a potential development branch, emergency fixes can be applied on the spot

In general: Fixes can be supplied as unified diffs

ProductionSupport

debug, emergency fix

Part of system

check


Support WorkflowsBug fixing flow with z2:1. Analyze the bug report: What system?

2. Install local <home> for that system, possibly on site

3. Debug and fix on local <home>

4. Test fix. Submit fix. Transport fix or provide diff

5. Customer imports fix and tests again

6. Propagate fix to other releases via transport

Traditional bug fixing flow:1. Analyze the bug report: What release?

2. Identify patch level of product or module

3. Find code line for that release

4. Verify/Install suitable app server

5. Build application for given version (e.g. tag)

6. Deploy application

7. Debug, apply fix and test

8. Submit and build application again

9. Copy binaries to customer – hope that it does not contains lots of other changes

10. Convince customer to install new application and test


Positioning...

… w.r.t. to ANT and Maven


In Principle...Sources

Developers

Define what'shappening there

Modify

Test,

debug,

support

Dev

, Q

A,

or P

r od.

Run

time


So, Where is the Problem?

Java application servers do not live on sources but rather on binaries

It is up to the development organization to (somehow) provide those binaries and hence... … find out what binaries are affected by a change … find out what binaries belong to one solution … keep development environments up-to-date … integrate changes (continuously) … live with delays between commit and run


Deployables

The “classical” approach

Dev

. Ru n

time

Sources

Dev

elop

ers

(wor

ksp a

ce)

Check o ut and

comm

it change s

Nightly build(everything?)

deploy/push(everything?)

Deployables

Dev build(what?)

QA

. R

u ntim

e

debug

deploy/push(what?)

central

local


Deployables


Dev

. Ru n

time

Sources

Dev

elop

ers

(wor

ksp a

ce)

Check o ut and

comm

it change s

Nightly build(everything?)

deploy/push(everything?)

Deployables

Dev build(what?)

QA

. R

u ntim

e

debug

deploy/push(what?)

central

local

How “equal” are these?How “equal” are these?

How “hard” is it to get thisup to date?


What are the sources of what you are debugging (but others change)?


How “hard” is it to introduce a

structural change?

How “hard” is it to introduce a

structural change?



Ok for monolithic projects (only few deployables)

Breaks for modular systems (too complex to keep in sync: Due to the “push” approach, it's up to the developer to keep things straight)

Breaks for large projects (turn around times too long)

Summary: Simple but doesn't scale.


The Maven Way

Dev

. Ru n

time

Sources

Dev

elop

ers

(wor

ksp a

ce)

Check o ut and

comm

it change s

ContinuousIntegration

Deployables

QA

. R

u ntim

e

debugde

ploy

/pus

h

SNAPSHOT Repo

Release Repo

Dev Build(mvn)

rele

ase

depl

oy/p

ush

deploy/push

depoy

central

local


The Maven Way

Dev

. Ru n

time

Sources

Dev

elop

ers

(wor

ksp a

ce)

Check o ut and

comm

it change s

ContinuousIntegration

Deployables

QA

. R

u ntim

e

debugde

ploy

/pus

h

SNAPSHOT Repo

Release Repo

Dev Build(mvn)

rele

ase

depl

oy/p

ush

deploy/push

depoy

central

local

Who maintains all this

infrastructure?

Who maintains all this

infrastructure?

How hard is it to keep the

dependency version vector right?

How hard is it to keep the

dependency version vector right?

How “equal” are these?How “equal” are these?





binary release management requires

extreme care

binary release management requires

extreme care


The Maven Way

Maven is built around versioning, versioned dependency and release processes for binaries

It does not address the system consistency side of things

It is geared towards a distributed, independent community of producers of libraries

It does not care about source consistency Summary: Useful for the community, much less

interesting for solution providers.


And in z2

Sources

Debug

Pull what has changedsince the last pull

Pull from workspace with prio

Dev

. R

u ntim

eQ

A.

Ru n

time

central

Dev

elop

ers

(wor

ksp a

ce)

Check o ut and

comm

it change s

Pull what has changed

since the last pull


In z2

A runtime that updates itself on-demand, according to changes in source repositories

Developer runtime takes developer workspace into account with preference

The runtime is a faithful representation of the repository – no question where sources are.

Almost like a scripting environment


Outlook

What could be done but we did not need to so far:

1. z2 as life-cycle tool over any Java EE server: z2 manages update checking and invalidations Instead of start/stop perform deploy/undeploy

2. Component Repository over Maven Repository Integration of existing assets

3. z2 for PHP et al Consistent dev and production environment


More Information

Try it out at:

www.z2-environment.de

Contact us at:

[email protected]

1006 Z2 Intro Complete

Documents

Transcript of 1006 Z2 Intro Complete