Integration in the Age of DevOps
-
Upload
brian-ashburn -
Category
Technology
-
view
101 -
download
0
Transcript of Integration in the Age of DevOps
![Page 1: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/1.jpg)
Integration in the Age of DevOps
![Page 2: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/2.jpg)
Cloud Native Architectures
Who Am I?
● Brian Ashburn○ Senior Solutions Architect
● 6 years at Red Hat● Work with customers in the Southeast● Focus on cloud, integration, and messaging
![Page 3: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/3.jpg)
Cloud Native Architectures
![Page 4: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/4.jpg)
If change is happening on the outside faster than on the inside the end is in sight.
Jack Welch, former CEO, GE
Cloud Native Architectures
S&P company life expectancy
![Page 5: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/5.jpg)
Cloud Native Architectures
Source: Dave Gray, The Connected Company
![Page 6: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/6.jpg)
Source: Dave Gray, The Connected Company
![Page 7: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/7.jpg)
• Trying to incorporate new technology?• Trying to copy what others (Netflix, Amazon) are
doing?• Tactical automation?• Created a “DevOps” team?• Exploring cloud services?• Build/deploy automation?• OpenSource?• Piecemeal integration?
How are you keeping up with change?
Cloud Native Architectures
![Page 8: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/8.jpg)
Cloud Native Architectures
• Faster software delivery• Own database (data)• Faster innovation• Scalability• Right technology for the
problem• Test individual services• Isolation• Individual deployments
![Page 9: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/9.jpg)
Microservices helps solve the problem of “how do we decouple our services and teams to move quickly at scale to deliver business value”
Cloud Native Architectures
![Page 10: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/10.jpg)
Microservices Envy: http://www.thoughtworks.com/radar/techniques/microservice-envy
You’re not going to do microserviceshttp://christianposta.com/blog/not-gonna-happen
![Page 11: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/11.jpg)
• If my services are isolated at the process level, I’m doing #microservices
I’m doing microservices if…
• If I use REST/Thrift/ProtoBuf instead of SOAP, I’m doing #microservices
• If I use JSON, I’m doing #microservices
• If I use Docker / SpringBoot / Dropwizard / embedded Jetty, I’m doing #microservices
![Page 12: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/12.jpg)
People try to copy Netflix, but they can only copy what they see. They copy the results, not the process.
Adrian Cockcroft, former Chief Cloud Architect, Netflix
![Page 13: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/13.jpg)
![Page 14: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/14.jpg)
Cloud Native Architectures
Fallacies of distributed computing• Reliable networking• Latency is zero• Bandwidth is infinite• Network is secure• Topology doesn’t change• Single administrator• Transport cost is zero• Network is homogenous
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
![Page 15: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/15.jpg)
Cloud Native Architectures
If we have to take into account the network, do we still need integration?
![Page 16: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/16.jpg)
Cloud Native Architectures
We’ll just do reactive, event-driven distributed systems… still need integration?
![Page 17: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/17.jpg)
Cloud Native Architectures
Yes; we need reliable integration!• REST, RPC• Messaging (ActiveMQ, JMS, AMQP, STOMP, Kafka, etc)• Legacy (SOAP, mainframe, file processing, proprietary)• Managed file processing• Streaming• Message transformation• EIPs
![Page 18: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/18.jpg)
Heavy Lifting: Apache Camel for Microservices
![Page 19: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/19.jpg)
Real developers ride Camels!
![Page 20: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/20.jpg)
Cloud Native Architectures
Apache Camel to the rescue!• Small Java library• Distributed-system swiss-army knife!• Powerful EIPs• Declarative DSL• Embeddable into any JVM (EAP, Karaf, Tomcat, Spring
Boot, Dropwizard, Wildfly Swarm, no container, etc)• Very popular (200+ components for “dumb pipes”)
![Page 21: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/21.jpg)
Cloud Native Architectures
Apache Camel to the rescue!
• Automatic retries, backoff algorithms• Dynamic routing• Powerful testing/mocking framework• Circuit breakers• Backpressure mechanisms• Beautiful REST DSL with built in Swagger support
![Page 22: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/22.jpg)
• “Smart endpoints, dumb pipes”• Endpoint does one thing well • Metadata used for further routing• Really “dynamic” with rules engine (eg,
Drools/BRMS)
Dynamic Routing
![Page 23: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/23.jpg)
REST DSLpublic class OrderProcessorRouteBuilder extends RouteBuilder {
@Override public void configure() throws Exception {
rest().post(“/order/socks”).description(“New Order for pair of socks”).consumes(“application/json”).route()
.to(“activemq:topic:newOrder”) .log(“received new order ${body.orderId}”)
.to(“ibatis:storeOrder?statementType=Insert”); }}
![Page 24: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/24.jpg)
• Throttle EIP• http://camel.apache.org/throttler.html
• Blocking SEDA Queue• from(“seda:name?size=100&blockWhenFull=true)
• Configure jetty/netty to use blocking acceptor queues• https://wiki.eclipse.org/Jetty/Howto/High_Load
• Using Exception handling/retry and DLQ logic when getting flow controlled• http://camel.apache.org/error-handling-in-camel.html
Backpressure with Camel
![Page 25: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/25.jpg)
Circuit breakerpublic class OrderProcessorRouteBuilder extends RouteBuilder {
@Override public void configure() throws Exception {
from(“direct:someinterface”) .loadbalance()
.circuitBreaker(3, 20000L, MyException.class)
.to(“ibatis:storeOrder?statementType=Insert”); }}
![Page 26: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/26.jpg)
Cloud Native Architectures
![Page 27: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/27.jpg)
Cloud Native Architectures
Typical problems developing microservices• How to run them all locally?• How to package them (dependency management)• How to test?• Vagrant? VirtualBox? VMs?• Specify configuration• Process isolation• Service discovery• Multiple versions?
![Page 28: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/28.jpg)
Cloud Native Architectures
Shared infrastructure platforms headaches
• Different teams• Different rates of change• VM sprawl• Configuration drift• Isolation / multi-tenancy
• Performance• Real-time vs batch• Compliance• Security• Technology choices
![Page 29: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/29.jpg)
![Page 30: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/30.jpg)
Cloud Native Architectures
Immutable infrastructure/deploys• “we’ll just put it back in Ansible”
• Avoid chucking binaries / configs together and hope!
• Cattle vs Pets
• Don’t change it; replace it
• System created fully from automation; avoid drift
• Eliminate manual configuration/intervention
![Page 31: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/31.jpg)
Docker / Linux Containers
![Page 32: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/32.jpg)
Kubernetes
![Page 33: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/33.jpg)
• Developer focused workflow• Enterprise ready• Higher level abstraction above containers for
delivering technology and business value• Build/deployment triggers• Software Defined Networking (SDN)• Docker native format/packaging• CLI/Web based tooling
OpenShift
![Page 34: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/34.jpg)
Red Hat OpenShift
Public PaaSService
On-premise or Private PaaSSoftware
Open Source PaaSProject
![Page 35: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/35.jpg)
Cloud Native Architectures
Fuse Integration Services for OpenShift• Set of tools for integration developers
• Build/package your Fuse/Camel services as Docker
images
• Run locally on CDK
• Deploy on top of OpenShift
• Plugs-in to your existing build/release ecosystem
(Jenkins/Maven/Nexus/Gitlab,etc)
![Page 36: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/36.jpg)
Cloud Native Architectures
![Page 37: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/37.jpg)
RED HAT JBOSS FUSE
Development and tooling
Develop, test, debug, refine, deploy
JBoss Developer Studio
Web services frameworkWeb services standards, SOAP,
XML/HTTP, RESTful HTTP
Integration frameworkTransformation, mediation, enterprise
integration patterns
Management and monitoring
System and web services metrics, automated discovery, container
status, automatic updates
JBoss Operations Network
+ JBoss Fabric Management
Console (hawtio)
Apache CXF Apache Camel
Reliable MessagingJMS/STOMP/NMS/MQTT, publishing-subscribe/point-2-point, store and forward
Apache ActiveMQ
ContainerLife cycle management, resource management, dynamic deployment,
security and provisioning
Apache Karaf + Fuse Fabric
RED HAT ENTERPRISE LINUXWindows, UNIX, and other Linux
![Page 38: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/38.jpg)
Cloud Native Architectures
Fuse Integration Services for OpenShift• Manage them with Kubernetes/OpenShift
• Flat class loader JVMs
• Take advantage of existing investment into Karaf with
additional options like “just enough app server”
deployments
• Supports Spring, CDI, Blueprint
![Page 39: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/39.jpg)
Cloud Native Architectures
Using FIS on Red Hat CDK• Small VM run locally by developers
• Full access to Docker, Kubernetes, OpenShift
• Deploy your suite of microservices with ease!
• Uses Vagrant/VirtualBox
• Getting Started on Windows! http://bit.ly/1U5xU4z
![Page 40: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/40.jpg)
Cloud Native Architectures
Typical problems developing microservices• How to run them all locally?• How to package them • How to test?• Vagrant? VirtualBox? VMs?• Specify configuration• Process isolation• Service discovery• Multiple versions?
![Page 41: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/41.jpg)
Cloud Native Architectures
![Page 42: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/42.jpg)
• Trying to incorporate new technology?• Trying to copy what others (Netflix, Amazon) are
doing?• Tactical automation?• Created a “DevOps” team?• Exploring cloud services?• Build/deploy automation?• OpenSource?• Piecemeal integration?
How are you keeping up with change?
Cloud Native Architectures
![Page 43: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/43.jpg)
• 100% open source, ASL 2.0• Technology agnostic (java, nodejs, python,
golang, etc)• Built upon decades of industry practices• 1-click automation• Cloud native (on premise, public cloud, hybrid)• Complex build/deploy pipelines (human
workflows, approvals, chatops, etc)• Comprehensive integration inside/outside the
platform
What if you could do all of this right now with an open-source platform?
![Page 44: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/44.jpg)
![Page 45: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/45.jpg)
![Page 46: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/46.jpg)
• Docker native, built on top of Kubernetes API• Out of the box CI/CD, management UI• Logging, Metrics• ChatOps• API Management• iPaaS/Integration• Chaos Monkey• Lots and lots of tooling/libraries to make
developing cloud-native applications easier
http://fabric8.io
![Page 47: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/47.jpg)
![Page 48: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/48.jpg)
![Page 49: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/49.jpg)
![Page 50: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/50.jpg)
![Page 51: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/51.jpg)
![Page 52: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/52.jpg)
![Page 53: Integration in the Age of DevOps](https://reader031.fdocuments.net/reader031/viewer/2022030313/58eca04c1a28abc40c8b45c7/html5/thumbnails/53.jpg)