InterSystems Caché Database: Spinning up hundreds of nodes in seconds

© 2016 InterSystems Corporation. All rights reserved.© 2016 InterSystems Corporation. All rights reserved.

Spinning up 100s of Nodes in SecondsDefine, Provision and Deploy a Cloud Infrastructure

© 2016 InterSystems Corporation. All rights reserved.

The Context

2


The issue

3


Cloud Architecture & Scaling

4


Abc.

The Answer

5


Cloud-Native Architecture: Pet or Cattle?

6


• Cloud-Native Architecture• “Infinite” Horizontal Scalability

• Microservices• DevOps• Continuous Delivery (CI/CD)• Open Source software

Big-Data Architecture Landscape

7

0

5

10

15

20

2007 2008 2009 2010 2011 2012 2013 2014

Billions ReqSource:2015 State of the Software Supply Chain Report – from Sonatype


82% of components used by automotive manufacturers are now sourced from suppliers

The Supply Chain

8


Automation

9


• System Administrators knows• 6 Servers

• 2 x Intel Xeon E5-2699 v4

• 128GB RAM

• 800GB SSD auto-tiering

• 10TB HDDs for backup, snapshots

• Storage SAN discussion• Data heat, auto-tiering

• SSD for production

• HDD for snapshots, backup etc.

Not So Cloud-Ready Architecture

10


5MB HDD being delivered (circa 1956)

11


• Cloud Service Models• SaaS• PaaS• IaaS

• We’ll need a design and awareness• Networking needs to be understood & configured• Security configuration verified• Dependencies checked• All infrastructure concerns brought forward

Cloud-Native Infrastructure

12


• Time for Interdisciplinary Teams• SysAdmin + Network + DBA + Developer:

• Developers & Operation +

• Microservices =

• Functional App Service Team

Cloud-Native Infrastructure How-To

13


DevOps

14


"Organizations which design systemsare constrained to produce systems

which are copies of the communication structures of these organizations.”

Q: What is the communication structure like in our organisations?

Conway’s Law

15


16

• Grassroots in 2008• Mainstream on CIO & CTO agenda now• Agile Process: from Dev to Ops• It’s all about ALM (release management) or

• Application Lifecycle Management• CD tools, the most influential• DevOps, for rapid changes and CD, drive

the adoption of microservices architecture (MSA)

DevOps


• Let’s Admit it, we haven’t learnt the lesson yet:• A live production environment is never finished• Are we building a software architecture that is designed for changes?• Do we adhere and have we signed up to the Reactive Manifesto?• What about the 12 factor app?

Cloud-Native Architecture: MSA & DevOps

17


• Image Artifact Provisioning• VMware, KVM, Xen, Vagrant• Docker, CoreOS, LXD• Packer• Veewee• CloudFoundry

• Configuration Management tool• Ansible• Puppet• Chef• Salt• etc.

DevOps & Tooling

18

• Infrastructure Provisioning• Amazon AWS CloudFormation• Google GCP • Microsoft Azure Automation and Azure Resource

Manager (ARM)• Cobbler• Apache Provisionr• Apache Brooklyn• Canonical Juju• Dell Crowbar• HPE Stackato• OpenStack Heat• Capistrano• Boto• Fog• Terraform• custom solutions…


Provisioning Artifacts & Infrastructures

19

Caché ActorsActors

A

VMware, KVM, Xen, VagrantDocker, CoreOS, LXDPacker Veewee CF

AnsiblePuppet ChefSalt

CloudFormation GCP ARM Cobbler ProvisionrBrooklyn Juju CrowbarStackato HeatCapistrano BotoFog Terraform


The Strategy

20

InterSystems

Actors

A


• Terraform• Infrastructure Provisioning & Configuration• Simple Declarative Configuration Files

• You configure resources (security groups, TCP rules, NAT, compute nodes, storage, etc.)

• Cloud agnostic• Idempotent• Adaptive• Supports Multiple Cloud Providers• Supports Variables and• Variable Interpolation• Simple Math operators• Functions• Templates• Infrastructure State awareness

Infrastructure as Code

21


Terraform Declaration

22


• Terraform and other tools can perform initial Deployment • Can they all handle more subtle and custom needs?• What happens when we need to run an Update and an Upgrade?• How do you perform the above in the Cloud?

• Issues• Network configuration• 1000s of instances• Security Settings• Service Discovery• What is your SLA?• Safeguarding the Data

Deployment

23

solid Lego bricks


What if…• I was able to package my app in such a way that compilers, interpreters, OS lib dependencies and other

libs, npm, .so, jvm, jars, RESTful APIs etc. are never going to be a problem?• I was able to have full portability• I was able to secure it• I was able to version control the full “service” image I’m spinning up and it has a:

• Lightweight footprint• Minimal overhead• Fast boot time• Simplified maintenance and• Packaging is supported by all Cloud providers?

Containers

24


• Why do tools like Ansible, Puppet, Chef and Salt exist?• to take care of consistency across the infrastructure for updates…

• What if our infrastructure was un-touchable so that you would not need CM tools and the risk?• How do you update 1000s of nodes?• How do you guarantee consistency?• Can you offer zero-downtime?• How long does it take you to perform that update?• What about DB schema migration?• etc.

Immutable Infrastructure

25


• Egenera• RightScale• Scalr• CliQr• Flexiant• Cloudbolt• Cloudfoundry• VMware vCloud Air• Kubernetes

• Google sponsored• Mesosphere

• Apache Mesos• Docker UCP (Universal Control Pane)

• Panamax• Rancher• Shipyard• Vamp• Kitematic

Cloud Management Platforms (CMP)

26


A Long List…

27


• Mesos & Mesospshere• The DCOS • Init for the DC

• Kubernetes• OSS for automating

• Deployment

• Operations and

• Scaling of containerized apps

• Docker Swarm• Host clustering and• Container Scheduling

• Rancher• Enterprise-level access control• Container orchestration & scheduling

Cloud Cluster Resource Orchestrators

28


1. One single Trend makes you trendy A Confluence of trends determines a cultural shift for major changes

2. Cloud-Native architecture is here The truth is in the code Infrastructure-as-code Immutable infrastructure

3. The future will feature Automation Autonomic Computing & Promise Theory

Summary

29


30


Questions

31

https://community.intersystems.com/


• http://www.appian.com/blog/bpm/avoiding-the-two-biggest-mistakes-in-software-tco-analysi

• http://kief.com/configuration-drift.html

• http://martinfowler.com/bliki/ImmutableServer.html

• http://martinfowler.com/articles/microservices.html

• https://www.ibm.com/developerworks/community/blogs/BizTechAlign/entry/devops-ard-mgr5?lang=en

• https://www-01.ibm.com/marketing/iwm/iwm/web/signup.do?source=swg-rtl-sd-calc&S_PKG=ov26141&S_TACT=C25600PW&CT=ISM0056

• http://www.infoq.com/articles/microservices-intro

• http://highscalability.com/amazon-architecture

• http://techblog.netflix.com/

• http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

• http://pivotal.io/platform/migrating-to-cloud-native-application-architectures-ebook

• “The Phoenix Project” by Jim Kim (DevOps), IT Revolution Press, ISBN: 978-0988262591

• “Release it!” by Mike Nygard, Pragmatic Bookshelf, ISBN: 978-0978739218

• “Deliver better, safer software even faster with sonatype software supply chain solutions”, PDF by Sonatype

• https://www.youtube.com/watch?v=GaHzdqFithc (microservices & Docker @GILT)

• https://www.youtube.com/watch?v=HYiLzji7MuY (Stefan Tilkov on architecture)

• http://www.reactivemanifesto.org/

• http://12factor.net/

References

32

http://www.appian.com/blog/bpm/avoiding-the-two-biggest-mistakes-in-software-tco-analysi

http://kief.com/configuration-drift.html

http://martinfowler.com/bliki/ImmutableServer.html

http://martinfowler.com/articles/microservices.html

https://www.ibm.com/developerworks/community/blogs/BizTechAlign/entry/devops-ard-mgr5?lang=en

https://www-01.ibm.com/marketing/iwm/iwm/web/signup.do?source=swg-rtl-sd-calc&S_PKG=ov26141&S_TACT=C25600PW&CT=ISM0056

http://www.infoq.com/articles/microservices-intro

http://highscalability.com/amazon-architecture

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://pivotal.io/platform/migrating-to-cloud-native-application-architectures-ebook

https://www.youtube.com/watch?v=GaHzdqFithc

https://www.youtube.com/watch?v=HYiLzji7MuY

http://www.reactivemanifesto.org/

http://12factor.net/

InterSystems Caché Database: Spinning up hundreds of nodes in seconds

Software

Transcript of InterSystems Caché Database: Spinning up hundreds of nodes in seconds