Automated Scaling of Microservice Stacks for JavaEE Applications
-
Upload
jelastic -
Category
Technology
-
view
135 -
download
0
Transcript of Automated Scaling of Microservice Stacks for JavaEE Applications
![Page 1: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/1.jpg)
Automated Scaling of Microservice Stacks
for JavaEE Applications
![Page 2: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/2.jpg)
Why I am here?
I want to discuss with you:
• General autoscaling concept for JavaEE applications
• Real issues and Tricky things
My goals are:
• Save your time if you decide you need to scale your app
• Share our own experience
![Page 3: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/3.jpg)
I am sure you’ve heard those words many times …
I just deploy my application to Docker, set a number of replicas and
… have it running … bla bla bla … super easy! Done!
![Page 4: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/4.jpg)
Yes, pretty easy …
If you need a super scalable and clustered Hello World application …
![Page 5: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/5.jpg)
In real life with JavaEE application
we have architecture similar to this
and its autoscaling becomes not a very trivial task…
In real life with JavaEE application we have architecture
similar to this
![Page 6: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/6.jpg)
Theoretical Part
![Page 7: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/7.jpg)
At least
applications scalable
At least 3 key things to make your
applications scalable
![Page 8: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/8.jpg)
Adapted runtime(Container + JVM + JavaEE Server)
micro
FIRST:
![Page 9: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/9.jpg)
Preferably, thin versions of application servers
NO! YES!
![Page 10: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/10.jpg)
There are various versions adapted
for lightweight containers
micro
GlassFish
There are various versions adapted for lightweight
containers
![Page 11: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/11.jpg)
Any specific reason to use VMs for microservices?
NO!
![Page 12: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/12.jpg)
Well…it might be, but …Well…it might be, but …
![Page 13: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/13.jpg)
One container - one microservice instance
NO!YES!
![Page 14: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/14.jpg)
What should we use instead?What should we use instead?
LXD
YES!
![Page 15: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/15.jpg)
APP
MSMS
MS MS
Applicationprepared for microservices
SECOND:
![Page 16: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/16.jpg)
SORRY,
Applicationprepared for SCALABLE microservices
![Page 17: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/17.jpg)
Next few slides can be a bit boring for those
who know…
BUT
Still important!!!!!!
Next few slides can be a bit boring for those who know
![Page 18: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/18.jpg)
horizontal
ve
rtic
al
Scaling vectors
![Page 19: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/19.jpg)
Decomposition into microservice
At least
Decomposition into microservice
At least 2 key approaches
![Page 20: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/20.jpg)
Logical division
![Page 21: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/21.jpg)
1. Analyze metrics (NewRelic /
JavaVisualVM)
2. Find a weak point (!)
3. Move this part to microservice
4. Repeat
Splitting bigger parts into smaller
![Page 22: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/22.jpg)
Orchestration
software
THIRD:
![Page 23: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/23.jpg)
CONTAINERS
PROVISIONING
HEALTHCHECKS
METRICS
GATHERING
APPLICATION
LIFECYCLE
MANAGEMENT
AUTOMATIC
LOAD BALANCING
SERVICE
DISCOVERY &
CONNECTIVITY
SCALING
CUSTOM
SCRIPTING*
What it typically does?
![Page 24: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/24.jpg)
CPU
HDD
(disk I/O)
SERVICE
FAILURE
RAM
NETWORK
SCHEDULER
Typical triggers that are used for scaling
![Page 25: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/25.jpg)
Practical PartTricky things. Typical issues. Examples
![Page 26: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/26.jpg)
horizontal
ve
rtic
al
In which direction to scale?
Tricky thing #1
![Page 27: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/27.jpg)
TOPOLOGY
Java EE Server
Business Tier
Web Tier
EJB App1 EJB App2
JSP App1
Java EE
Micro
Instance
JSP App1
Java EE
Micro
Instance
JSP App2
JSP App1
Java EE
Micro
Instance
JSP App1
Java EE
Micro
Instance
Scaling type
Java EE
Micro Instance
Java EE
Micro Instance
Example #1
JSP App2 JSP App2JSP App1
JSP App1
Java EE
Micro
Instance
JSP App1
Java EE
Micro
Instance
JSP App1
Java EE
Micro Instance
JSP App1
Java EE
Micro
Instance
JSP App1
Java EE
Micro
InstanceJava EE
Micro Instance
JSP App1 JSP App2
Legacy JavaEE application
![Page 28: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/28.jpg)
Example #2
CPU NO
YES
YES
NO
NO
RAM
HDD (disk I/O)
NETWORK
SCHEDULER
Scaling type VERTICAL
Databases Triggers configuration
![Page 29: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/29.jpg)
(dedicatee)
Takes much time to sync data
Temporary locks
Temporary performance degradation
RESUME
Hard to use auto horizontal scaling
Automatic vertical scaling can be used just fine
It’s better to keep spare instances underloaded
(dedicated storage per instance)
Typical issues while scaling of database instances
![Page 30: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/30.jpg)
Events subscription
Usually, just spinning up an extra instance is NOT enough
Tricky thing #2
![Page 31: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/31.jpg)
onAfterScaleOut[nodeGroup:ejb-app1] {
“call”: “registerNodeInMonitoringSystem”,
“call”: “addMemberToHazelcast”,
“call”: “call3rdPartyApiService”
}
onAfterScaleIn[nodeGroup:jsp-app2] {
“call”: “removeNodeFromMonitoringSystem”,
“call”: “removeMemberFromHazelcast”
}
Example #1
Typical ScaleIn/ScaleOut events and cases
THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE:
https://github.com/jelastic-jps/payara/blob/master/payara-micro-cluster/manifest.jps
![Page 32: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/32.jpg)
1. Decrypt encrypted volume, that was attached on-fly
onAfterVolumeAttached[nodeGroup:instance] {{
“call”: “decryptVolume”
}
2. Get auth keys by instance before joining DAS
onBeforeStartService[nodeGroup:instance] {
“cmd”: “/root/scripts/pullAccessKeys.sh ${token}”
}
3. Deprovision node from DAS if cluster was shrinked
onAfterScaleOut[nodeGroup:das] {
“cmd”: “./asadmin_proxy remove ${instance[@last].ip}”
}
Example #2
THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE:
https://github.com/jelastic-jps/glassfish/blob/master/manifest.jps
![Page 33: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/33.jpg)
Proper triggers configuration
Tricky thing #3
This tuning is a long process, so be patient
![Page 34: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/34.jpg)
Typical problem #1
Changing the value in one place can be reflected in other places
![Page 35: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/35.jpg)
Load Testing
Detect deviations in testing loop
How to deal with this?
![Page 36: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/36.jpg)
The triggers can become outdated with new code
Typical problem #2
![Page 37: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/37.jpg)
COMMIT BUILD TESTINGPERFORMANCE
ANALYSISDELIVERY
The fix is easy here
Inject metrics collection as a part of your CI/CD step
![Page 38: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/38.jpg)
Getting rid of spare things
Tricky thing #4
![Page 39: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/39.jpg)
Some of JavaEE Servers are designed for VMs, not for containers
On example of Weblogic
Typical problem #1
![Page 40: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/40.jpg)
Just remove everything spare
What is the right way to go?
![Page 41: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/41.jpg)
Network is full of useless traffic
Total bandwidthUseful traffic
Typical problem #2
Multicast | Heartbeats
![Page 42: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/42.jpg)
Avoid multicast for members detection
Instance 1
Orchestrator
Instance 2
Instance 3
What we can do?
Use events and unicast
![Page 43: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/43.jpg)
Real use case
On-Demand Scalability and Easy Management of Java EE Project: Hybrid Cloud for Miele
• Migrated from VMs to Containers
• Migrated from GlassFish to WildFly
• Improved scaling and HA
• Implemented Hybrid Cloud
![Page 44: Automated Scaling of Microservice Stacks for JavaEE Applications](https://reader031.fdocuments.net/reader031/viewer/2022030318/5a6d276b7f8b9ae5418b4e1f/html5/thumbnails/44.jpg)
Java in Containers
https://jelastic.com/java-cloud-hosting/