Modularity in the cloud a case study
-
Upload
marcel-offermans -
Category
Technology
-
view
613 -
download
2
description
Transcript of Modularity in the cloud a case study
![Page 1: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/1.jpg)
Modularity in the cloud
A case study
![Page 2: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/2.jpg)
Paul BakkerArchitect at Luminis Technologies
@pbakker
Marcel OffermansDirector at Luminis TechnologiesMember at Apache
@m4rr5
![Page 3: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/3.jpg)
The case
Educational system focussed on personalized learning
Used in high schools in The Netherlands
Expand to other countries in the near future
![Page 4: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/4.jpg)
![Page 5: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/5.jpg)
Cross device HTML 5 + JS
RESTful web services
OSGi services
Mongo
Apache Felix
Amdatu
S3
![Page 6: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/6.jpg)
Amdatu: OSGi Cloud components
JAX-RSMONGO DBBlobStoresMulti-tenancySearchOpenSocial...
Apache Licensed
open source
project
![Page 7: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/7.jpg)
Some numbers
190bundles
in a deployment
120PulseOn
bundles
![Page 8: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/8.jpg)
School B
Load
Deployment
Load Balancer
PulseOn nodePulseOn node
PulseOn node
Load Balancer
PulseOn node
PulseOn node
PulseOn node
School A
Mongo
Mongo
Mongo
MongoMongo
Mongo
![Page 9: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/9.jpg)
Availability zones
Load Balancer
PulseOn nodePulseOn node
PulseOn node
PulseOn nodePulseOn node
PulseOn node
Availability Zone 1
MongoMongo
Mongo
MongoMongo
Mongo
Availability Zone 2
![Page 10: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/10.jpg)
Horizontal scalability
Horizontal scaling requires stateless nodes
HTML5 clients need less server side state
Any state should go to some kind of store
![Page 11: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/11.jpg)
Auto scaling
Considerable higher loads during school hours
Enoughcapacity
Without paying for idle servers
at night...
![Page 12: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/12.jpg)
Load Balancer small node
Cluster per school
Always use a load balancer because we don’t want downtime during scaling
![Page 13: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/13.jpg)
Load Balancer small node
large node
large node
Early morning...
![Page 14: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/14.jpg)
End of the day...
Load Balancer small node
large node
large node
![Page 15: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/15.jpg)
But how do we install
our software on a node?
![Page 16: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/16.jpg)
Provisioning servers
![Page 17: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/17.jpg)
ace screenshot
User Interface
REST API Shell scripting
![Page 18: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/18.jpg)
Node
AWS Auto Scaling
1. Start Load Balancer
Apache ACE
2. register
3. register
4. provisiondeployment
package
Provisioning servers
![Page 19: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/19.jpg)
./as-‐create-‐launch-‐config demo -‐-‐image-‐id ami-‐0ee8e07a -‐-‐instance-‐type m1.small -‐-‐region eu-‐west-‐1 -‐-‐group sg-‐ce1420ba -‐-‐user-‐data-‐file userdata.txt
./as-‐create-‐auto-‐scaling-‐group demo -‐-‐launch-‐configuration demo -‐-‐min-‐size 1 -‐-‐max-‐size 1 -‐-‐availability-‐zones eu-‐west-‐1a -‐-‐load-‐balancers demo -‐-‐tag "k=Name,v=demo,p=true"
![Page 20: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/20.jpg)
Build releases
CI server
Release OBR
-runbundles: org.apache.felix.http.jetty;version=2.2.0,\ org.apache.felix.gogo.command;version=0.10.0,\ org.apache.felix.gogo.runtime;version=0.10.0,\ org.apache.felix.gogo.shell;version=0.10.0,\ osgi.cmpn;version='(4.2.0,4.2.1]',\ org.amdatu.security.tokenprovider.api;version=latest,\ org.amdatu.security.tokenprovider.impl;version=latest,\ org.amdatu.web.rest.doc,\ com.lopexs.arl.opensocial.config;version=latest,\ jackson-core-asl,\
Apache ACE2. Baseline 3. Publish
4. Distribution
Targets
1. GIT tag
5. Provision
![Page 21: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/21.jpg)
How do we know which
bundles to update?
![Page 22: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/22.jpg)
Semantic versioning
is usually about
package versions
Export-Package: org.apache.felix.dm;version="3.1.0"
![Page 23: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/23.jpg)
Semantic versioning
demystified
major.minor.micro(.qualifier)
Major: Backward incompatible change
Minor: Backward compatible change
Micro: Implementation change
Qualifier: Label, e.g. build number
1.0.0.x
![Page 24: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/24.jpg)
Semantic versioning
on bundles
Bundle-Version: 1.0.0
Important for deployment!
![Page 25: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/25.jpg)
Baselining
Bytecode analysisCompare build with latest releaseCheck if version numbers should be bumped
![Page 26: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/26.jpg)
Continuous deployment
CI server
Release OBR
-runbundles: org.apache.felix.http.jetty;version=2.2.0,\ org.apache.felix.gogo.command;version=0.10.0,\ org.apache.felix.gogo.runtime;version=0.10.0,\ org.apache.felix.gogo.shell;version=0.10.0,\ osgi.cmpn;version='(4.2.0,4.2.1]',\ org.amdatu.security.tokenprovider.api;version=latest,\ org.amdatu.security.tokenprovider.impl;version=latest,\ org.amdatu.web.rest.doc,\ com.lopexs.arl.opensocial.config;version=latest,\ jackson-core-asl,\
Apache ACE2. Baseline 3. Publish
4. Distribution
CD server
1. GIT tag
5. Provision
Snapshot OBR
![Page 27: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/27.jpg)
ACE scripting with GoGotargetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"
sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)
echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo
echo "Opening client workspace"workspace = (ace:cw)
echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}
echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}
echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}
echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }
echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }
echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }
echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }
echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }
echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"
echo "Committing workspace"$workspace commit
![Page 28: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/28.jpg)
ACE scripting with GoGo
targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"
sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)
echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo
echo "Opening client workspace"workspace = (ace:cw)
echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}
echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}
echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}
echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }
echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }
echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }
echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }
echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }
echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"
echo "Committing workspace"$workspace commit
![Page 29: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/29.jpg)
ACE scripting with GoGo
targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"
sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)
echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo
echo "Opening client workspace"workspace = (ace:cw)
echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}
echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}
echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}
echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }
echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }
echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }
echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }
echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }
echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"
echo "Committing workspace"$workspace commit
![Page 30: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/30.jpg)
ACE scripting with GoGo
targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"
sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)
echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo
echo "Opening client workspace"workspace = (ace:cw)
echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}
echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}
echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}
echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }
echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }
echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }
echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }
echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }
echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"
echo "Committing workspace"$workspace commit
![Page 31: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/31.jpg)
Cluster
OSGi Target
Load Balancer
MongoDB
MongoDB
MongoDB
OSGi Target
Infrastructure Management
Component Provisioning
Cluster Management
System Monitoring
Automatic Scaling
Starts/stops nodes
Deploys and updates
software components
Monit
ors Q
uality
of S
ervic
e
aspe
cts
Adds/removes nodes from
the cluster
Instantiates one or more template based clusters
Cluster TemplateCluster
TemplateCluster Template
Artifacts, Features &
Deployments
5
Cluster Configuration
1
Requests nodes based on provided requirements
2 3Assi
gns s
oftware
distrib
ution
s to t
argets
4
Gets configured with artifacts for deployments
Provides usage data for scaling algorithms
6
7
89
![Page 32: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/32.jpg)
Tooling and frameworks are
now mature enough
How does this fit other applications?
The architecture fits most
modern web applications
Works in the cloud
(but not a requirement)
![Page 33: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/33.jpg)
amdatu.org
![Page 34: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/34.jpg)
Recommended reading ;)
Available NOW!
Book signing Tue 14.30
@ O’Reilly Booth (JavaOne Pavilion)
Free books for the early birds...
![Page 35: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/35.jpg)
There is more...
Building Modular Cloud Applications in Java:
Lessons LearnedTue 10am - 11am
Modular JavaScriptWed 3pm - 4pm
![Page 36: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/36.jpg)
Cloud provisioning
http://ace.apache.org/Cloud OSGi services
http://www.amdatu.org/
Eclipse OSGi pluginhttp://bndtools.org/
That’s ushttp://luminis.eu/
There is more...
![Page 37: Modularity in the cloud a case study](https://reader033.fdocuments.net/reader033/viewer/2022060109/5552f581b4c90587048b4d6d/html5/thumbnails/37.jpg)
Takk
Grazie
Thankyou
Obrigado
MahaloDankeDank U
Merci
Gracias