Eclipse Gyrex OSGi based PaaS-Like Programming Stack - OSGi Cloud Workshop March 2012
Paremus Cloud and OSGi Beyond the VM - OSGi Cloud Workshop March 2012
-
Upload
mfrancis -
Category
Technology
-
view
2.205 -
download
2
description
Transcript of Paremus Cloud and OSGi Beyond the VM - OSGi Cloud Workshop March 2012
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Cloud & OSGi - Beyond the VM
Richard Nicholson: Paremus CEO President of the OSGi Alliance www.paremus.com
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Virtual Machine Man
Amazon EC2, RackSpace, Azure? They all suffer from the same fundamental design flaw.
Virtual Machine image centric Cloud solutions:• Cause excessive network infrastructure load.• Cause excessive storage consumption - caused by virtual machine
image sprawl.• Fail to address to primary issue - the ongoing cost of maintaining
applications.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Artefact Module Centric Image Centric
Machine Image ~ 1 to 2 Gbytes per node ~ 1 to 2 Gbytes per node
Application ~ 1 TO 100 Mbytes ~ 1 to 2 Gbytes per node
Application Patch ~ 100’s Kbytes ~ 1 to 2 Gbytes per node
Property / Config Changes ~ 1 Kbyte ~ 1 to 2 Gbytes per node
Type of Deployment Artefact
VM images: The Wrong deployment artefact!
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Observation
1. The Unit of Deployment is...• The unit of fix / maintenance.• The potential unit of re-use.
Also...2. Time to deploy - directly influences...• Time to recover a failed Business Service.• Time to scale an environment in response to business load.
DetectDecision
Response
Need to Optimise
Is Kirk
in the
Room
?
Patterns of Modular Architecture
(Kirk Knoernschild)
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Principle: Modularity and Abstraction
The smaller the deployment artefact the better
➟ BUT MUST AT ALL COST avoid Operational Complexity.
Automatically manage dependenciesAutomatically install required parts.
ABSTRACTION - operations must manage the WHOLE, not the constituent PARTS.
➟ The new OSGi Alliance R5.0 OBR (Resolver & Repository specifications)?
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Principle: Modularity and Abstraction
The smaller the deployment artefact the better
➟ BUT MUST AT ALL COST avoid Operational Complexity.
Automatically manage dependenciesAutomatically install required parts.
ABSTRACTION - operations must manage the WHOLE, not the constituent PARTS.
➟ The new OSGi Alliance R5.0 OBR (Resolver & Repository specifications)
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Principle: Modularity and Abstraction
The smaller the deployment artefact the better
➟ BUT MUST AT ALL COST avoid Operational Complexity.
Automatically manage dependenciesAutomatically install required parts.
ABSTRACTION - operations must manage the WHOLE, not the constituent PARTS.
➟ The new OSGi Alliance R5.0 OBR (Resolver & Repository specifications)
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
circa 2005
Paremus introduce the Service Fabric - a distributed OSGi / OBR based Cloud runtime.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
The Paremus Service Fabric
Java / Scala, [Blueprint, Declarative Services, WARs & generic artefacts].
(1..n) compute resource (physical or virtual) may contribute to create a Service Fabric
A highly modular OSGi™ / OSGi Resolver based cloud runtime.
(1..m) ‘Composite Applications’ (a.k.a. Systems) may be dynamically assembled and run upon a single Service Fabric.
IaaS
SaaS
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Runtime State
= 1
= 5
= 1
Target State
The Service Fabric is a model driven runtime. To change a running System, simply change its model in the required manner.
Re-Configure
Operational simplicity through Abstraction
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
= 1
= 5
= 1
M!del
Target State
"Structure#SLA$
Runtime
Entity
Monitor
Provision Delta
Planned Deltase.g. Configuration
changes
Unplanned Deltase.g. Resource
failures
Target State Runtime State
Updating the System
The runtime will respond...
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Today - 2012
Building consensus that next generation of cloud solution will be composite and will support dynamically assembly highly
modular applications.
Growing realisation that OSGi, and specifically the new OBR resolver specification, are key enablers.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Dependency management really is everything!
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
OBR Resolution Process
OSGi Resolver
RESOLVER ENVIRONMENT
Physical Repository
OSGi Repository A
OSGi Repository C
OSGi Repository B
‘Scoped’ Repositories(specific views of one - or more - physical
repositories)
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
OBR Resolution Process
OSGi Resolver
Resource/s + Constraints e.g. ‘Pricer (a,c)’
RESOLVER ENVIRONMENT
Physical Repository
OSGi Repository A
OSGi Repository C
OSGi Repository B
‘Scoped’ Repositories(specific views of one - or more - physical
repositories)
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
OBR Resolution Process
OSGi Resolver
Resource/s + Constraints e.g. ‘Pricer (a,c)’
RESOLVER ENVIRONMENT
Physical Repository
OSGi Repository A
OSGi Repository C
OSGi Repository B
‘Scoped’ Repositories(specific views of one - or more - physical
repositories)
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
OBR Resolution Process
OSGi Resolver
Resource/s + Constraints e.g. ‘Pricer (a,c)’
RESOLVER ENVIRONMENT
Physical Repository
OSGi Repository A
OSGi Repository C
OSGi Repository B
‘Scoped’ Repositories(specific views of one - or more - physical
repositories)
Part ‘Pricer’
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
OBR Resolution Process
OSGi Resolver
Resource/s + Constraints e.g. ‘Pricer (a,c)’
RESOLVER ENVIRONMENT
Physical Repository
OSGi Repository A
OSGi Repository C
OSGi Repository B
‘Scoped’ Repositories(specific views of one - or more - physical
repositories)
Part ‘Pricer’
WHAT NOW?
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Centralised Resolver: push / pull resultant composites?
OSGi Resolver
Part ‘Pricer’
Physical Repository
OSGi Repository A
OSGi Repository C
OSGi Repository B
2
Push / Pull The new software
Artefact?
1
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
• Better than a fully stacked VM (WHAT COULD BE WORSE?!).
• Heavy resolving load on centralised (one or a few) ‘Cloud‘ resolvers.
• Shipping assembled composites to target nodes loads loads network and source nodes.
• No visibility of local environment on target nodes. Resolution may be incorrect or not optimal.
• Example of approach - Apache ACE?
Centralised Resolver: push / pull resultant composites?
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Centralised Resolver: Push / Pull List of Bundles?
OBR Resolver
Part ‘Pricer’ Pricer bundle a bundle b bundle c bundle d bundle e bundle f bundle g bundle h
1
2
Physical Repository
OBR Repository A
OBR Repository C
OBR Repository B
*AR
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Centralised Resolver: Push / Pull List of Bundles?
• Better than pushing / pulling composite artefacts.• Avoids some network loading - but not the loading
on the centralised resolvers.• Still no visibility of local environment on target
nodes. As before - resolution may be incorrect or not optimal.
• Example of approach?
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Service Fabric approach - OSGi Resolver on each node ✔
OSGi Resolver
An OSGi runtime instance
$ ./poshParemus Service Fabric Evaluation License,expires Wed Jan 09 23:59:59 EST 2013.______________________________________Welcome to the Paremus Service Fabric![paremus-ran-2.local.0]% lsb 0 ACTIVE org.apache.felix.framework:3.2.1 1 ACTIVE com.paremus.core.types:1.4.5 2 ACTIVE com.paremus.core.hooks:1.4.5 3 ACTIVE com.paremus.posh.runtime:1.4.5 4 ACTIVE com.paremus.posh.shell:1.4.5 5 ACTIVE com.paremus.console:1.4.5 6 ACTIVE com.paremus.nimble.core:1.4.5 7 ACTIVE com.paremus.nimble.repos:1.4.5 8 ACTIVE com.paremus.nimble.cli:1.4.5 9 ACTIVE org.apache.felix.configadmin:1.2.810 ACTIVE com.paremus.util.cmds:1.4.5
Resolver centric OSGi runtimes are lightweight runtimes ! (example Paremus
Nimble).
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Inject Root Resource & OSGi repository scope
Physical Repository
OBR Repository A
OBR Repository C
OBR Repository B
Resource/s + Constraints e.g. ‘Pricer (a,c)’
neg
otia
teA negotiation process selects optimal nodes to host the ‘Pricer’ composite
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Inject Root Resource & OSGi Repository scope
Physical Repository
OBR Repository A
OBR Repository C
OBR Repository B
Resource/s + Constraints e.g. ‘Pricer (a,c)’
Each node performs a local resolution.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Inject Root Resource & OSGi Repository scope
Physical Repository
OBR Repository A
OBR Repository C
OBR Repository B
Resource/s + Constraints e.g. ‘Pricer (a,c)’
RESO
LVE
Each node performs a local resolution.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Each participating node asynchronous pulls artefacts
Physical RepositoryResource/s + Constraints e.g. ‘Pricer (a,c)’ Asynchronous flow of bundles from
one or more physical repositories toparticipating nodes
Each node asynchronously downloadsrequired bundles that are not already local.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
‘Persistence (b,c)’
‘Gateway (b)’
Repeat
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
System Models
Physical Deployment Running System = FX Trading
Pricer (b,c) [1..n] Persistence (a,c) [1..1] Gateway (b) [1..m]
In RFC-133 we defined the notion of a ‘System’: A runtime abstraction comprised of elements, wiring and runtime constraints.
<system name="gateway.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>
<!-- set SystemGroup for security control --> <nature group="demo" />
<system name="pricer" boundary="fibre"> <system.part category="msf" name="com.example.pricer"> <property name="type" value="firm" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> <replication.handler type="scalable"> <property name="scaleFactor" value="1" type="float" /> <property name="fixedDelta" value="-2" type="integer" /> <property name="minimum" value="1" type="integer" /> </replication.handler> </system>
<system name="gateway" boundary="fibre"> <system.part category="msf" name="com.example.gateway"> <property name="id" value="foo" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> </system></system>
System Model = FX Trading
<system name="fx.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>
<!-- set SystemGroup for security control --> <nature group="demo" />
<system name="pricer" boundary="fibre"> <system.part category="msf" name="com.example.pricer"> <property name="type" value="firm" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> <replication.handler type="scalable"> <property name="scaleFactor" value="1" type="float" /> <property name="fixedDelta" value="-2" type="integer" /> <property name="minimum" value="1" type="integer" /> </replication.handler> </system>
...
...
... </system>
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
A ‘System’ Model - (example Paremus Service Fabric)
Defines system name and version. Whether System is distributed.
The OSGi repositories to be used when assembling Parts.
<system name="fx.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>
<!-- set SystemGroup for security control --> <nature group="demo" />
<system name="pricer" boundary="fibre"> <system.part category="msf" name="com.example.pricer"> <property name="type" value="firm" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> <replication.handler type="scalable"> <property name="scaleFactor" value="1" type="float" /> <property name="fixedDelta" value="-2" type="integer" /> <property name="minimum" value="1" type="integer" /> </replication.handler> </system>
....
....
....
</system>
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
A ‘System’ Model - (example Paremus Service Fabric)
The Pricer Part. A managed service fabric based built from com.example.pricer. An instance of which is created with attribute ‘firm’
<system name="fx.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>
<!-- set SystemGroup for security control --> <nature group="demo" />
<system name="pricer" boundary="fibre"> <system.part category="msf" name="com.example.pricer"> <property name="type" value="firm" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> <replication.handler type="scalable"> <property name="scaleFactor" value="1" type="float" /> <property name="fixedDelta" value="-2" type="integer" /> <property name="minimum" value="1" type="integer" /> </replication.handler> </system>
....
....
....
</system>
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
A ‘System’ Model - (Paremus Service Fabric)
Remote Services using the Paremus implementation of the OSGI Alliance RSA specification.
In this example Essence RMI is the protocol - and SLP isused for discover. Normally we recommend DDS - which is the default.
AVRO?
<system name="fx.system" version="1.0" boundary="fabric" repopath="gateway,nimble-rs,aries,aries-ext,fabric,fabric-ext,nimble-cmpn" xmlns="http://schema.paremus.com/sf/1"> <description>Deploys a pricer and gateway component wired together using slp and essencermi</description>
<!-- set SystemGroup for security control --> <nature group="demo" />
<system name="pricer" boundary="fibre"> <system.part category="msf" name="com.example.pricer"> <property name="type" value="firm" /> </system.part> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.dsw.essencermi" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.discovery.slp" /> <system.part category="osgi.active.bundle" name="com.paremus.dosgi.topologymanager" /> <replication.handler type="scalable"> <property name="scaleFactor" value="1" type="float" /> <property name="fixedDelta" value="-2" type="integer" /> <property name="minimum" value="1" type="integer" /> </replication.handler> </system>
....
....
....
</system>
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
A ‘System’ Model - (Paremus Service Fabric)
Replication Handlers control the number of Part instances required in the runtime.
....
1
Many
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
DEMO TIME
1. Start a client. 2. Start single node Service Fabric & a simple OBR repository.3. Load Repository & System model.4. Deploy System. 5. Apply a property change to the running System.
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Control Loops & Hierarchies
Highly agile, stable & robust systems tend to use (fast) inner and (slow) outer provisioning control loops (reference http://www.ist-selfman.org/wiki/images/9/94/SelfmanFACS06talk.pdf).
Cloud Runtime
reactive Inner Control Loop (PaaS)Controls distribution
of software components across population of available resources
automatic or manual Outer Control Loop (IaaS)controlling number of VM resources
in the environment
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
BNDTools <> Service Fabric Integration
Code Run
An OBR centric end-to-end solution
Copyright © 2005 - 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
OSGi Dev Con 2012 - Cloud Workshop II March 2012
Modular Systems are Maintainable Systems
Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010
⎫⎬⎭
}Applicability w.r.t OPEX of VM based (non modular) private Cloud solutions.
Thanks
Copyright © 2012 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.
Service Fabric Overview February 2012
www.paremus.com
Service Fabric - www.paremus.com/psfNimble - www.paremus.com/nimble
Bndtools - http://bndtools.org