EMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
-
Upload
david-vonthenen -
Category
Technology
-
view
15 -
download
0
Transcript of EMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
1© Copyright 2016 EMC Corporation. All rights reserved.
2© Copyright 2016 EMC Corporation. All rights reserved. 2© Copyright 2016 EMC Corporation. All rights reserved.
DEEP DIVE WITH MESOS & PERSISTENT STORAGE FOR APPLICATIONSDAVID VONTHENEN, EMC {code}
3© Copyright 2016 EMC Corporation. All rights reserved.
• David vonThenen, Developer Advocate - EMC {code}– Twitter: @dvonthenen– Blog: http://dvonthenen.com– Github: https://github.com/dvonthenen
PRESENTER
4© Copyright 2016 EMC Corporation. All rights reserved.
• Quick Review of Mesos• Frameworks and Two Layer Scheduling• Production Applications• Demo• Q&A
AGENDA
5© Copyright 2016 EMC Corporation. All rights reserved. 5© Copyright 2016 EMC Corporation. All rights reserved.
QUICK REVIEW OF MESOS
6© Copyright 2016 EMC Corporation. All rights reserved.
• From a resource perspective, it’s a cluster manager:– Pools resources to be centrally managed as a single unit
• From an application perspective, it’s a scheduler:– Dispatches workloads to consume pooled resources
• Often described as a Data Center Operating System (DCOS)– Static versus Dynamic Partition of Resource– Efficient utilization of CPU, Memory, Disk
• Proven resource scheduler for production workloads
QUICK REVIEW OF MESOS
7© Copyright 2016 EMC Corporation. All rights reserved.
SCHEDULERS ARE DIFFICULT TO BUILD
8© Copyright 2016 EMC Corporation. All rights reserved.
• Master– Manages Mesos Agents– Enables fine-grained sharing of resources (CPU, RAM, …)
across frameworks in the form of resource offers.• Zookeeper
– Enables highly reliable distributed coordination (cluster state)
• Agent (aka Slave)– The entity that will be running the task
MESOS COMPONENTS
9© Copyright 2016 EMC Corporation. All rights reserved.
MESOS ARCHITECTURE
10© Copyright 2016 EMC Corporation. All rights reserved.
• Generic– Anything that can be run on a standard Linux shell– Provides a mechanism for deploying applications packages– Supports Docker workloads
• Framework– A plug-in to Mesos that allows you to implement a sub-
scheduler– specialize a workload by providing custom placement and
resource allocation
APPLICATION SUPPORT
11© Copyright 2016 EMC Corporation. All rights reserved.
GENERIC APPLICATIONS
12© Copyright 2016 EMC Corporation. All rights reserved. 12© Copyright 2016 EMC Corporation. All rights reserved.
FRAMEWORKS ANDTWO LAYER SCHEDULING
13© Copyright 2016 EMC Corporation. All rights reserved.
• On the surface…• Allows specialization of
workload by providing custom placement and resource allocation• Provides monitoring of
these workloads to protect and react when things fail
MESOS FRAMEWORK
14© Copyright 2016 EMC Corporation. All rights reserved.
• Scheduler– a scheduler that registers with the master to be offered
resources• Executor
– an executor process that is launched on slave nodes to run the framework’s tasks
FRAMEWORK COMPONENTS
15© Copyright 2016 EMC Corporation. All rights reserved.
FRAMEWORK ARCHITECTURE
16© Copyright 2016 EMC Corporation. All rights reserved.
FRAMEWORK ARCHITECTURE
17© Copyright 2016 EMC Corporation. All rights reserved.
• Odds are you have been using at least 1 framework without realizing it…
AS IT TURNS OUT…
• Marathon is a framework– A framework for performing container
orchestration for long running tasks• Chronos is a framework
– A framework for performing batch or scheduled tasks
18© Copyright 2016 EMC Corporation. All rights reserved.
• Zooming into the “Marathon” box…• Marathon is a Scheduler• Launching a Marathon task, the “generic”
applications are created inside a “generic” (or default) executor• Dogfooding!
MARATHON DISSECTED
19© Copyright 2016 EMC Corporation. All rights reserved.
MESOS FRAMEWORKS
20© Copyright 2016 EMC Corporation. All rights reserved.
• 2 layer scheduling or sub-scheduling of resources• Manages the lifecycle of your
workload through application specific monitoring• What does this mean
specifically?
DIGGING DEEPER
21© Copyright 2016 EMC Corporation. All rights reserved.
BUILDING HADOOP CLUSTERS
22© Copyright 2016 EMC Corporation. All rights reserved.
MANAGING HADOOP CLUSTERS
23© Copyright 2016 EMC Corporation. All rights reserved.
APPLICATIONS USING HADOOP
24© Copyright 2016 EMC Corporation. All rights reserved.
THIS SEEMS SILLY
25© Copyright 2016 EMC Corporation. All rights reserved.
• Allows you to subsume the default scheduler such that:• You can install and configure your application• Resources can be allocated in a way that optimizes
your applications• Allows operation real-time automated improvements
for your applications– Scaling up for load balancing peak times– Scaling back to free up resources
2 LAYER SCHEDULING
26© Copyright 2016 EMC Corporation. All rights reserved.
• The details of your applications are hidden from the end user!– Deployment– Operations– Growth– Disaster recovery– Performance
• Just a consumer of the application as a service!
THE NON-TECHNICAL VERSION
27© Copyright 2016 EMC Corporation. All rights reserved.
• CouchBase Framework (doesn’t exist, let’s imagine it does)– CouchBase is a NoSQL DB
• Monitor the CouchBase instances, when nearing full– Dynamically add an node using “server-add”– Then call “rebalance”
• Reactive and Proactive but fully automated• I think we had all seen this movie…
FOR EXAMPLE
28© Copyright 2016 EMC Corporation. All rights reserved.
SKYNET
29© Copyright 2016 EMC Corporation. All rights reserved. 29© Copyright 2016 EMC Corporation. All rights reserved.
PRODUCTION APPLICATIONS
30© Copyright 2016 EMC Corporation. All rights reserved.
• Containers are ephemeral therefore stateless• Quickly found the need for state
– Configuration– Data– Logging / Auditing
• Containerizers like Mesos and Docker introduced exporting to local storage• Failure of a node? hardware? A better way?
EVOLUTION OF STATEFUL
31© Copyright 2016 EMC Corporation. All rights reserved.
• Persistent– Containers can come and go,
but continue where they left off– Application Data, Logs, Audit
Trail, etc• External
– Storage lives outside the Mesos Agent node
– High Availability– Tolerates node failure
PERSISTENT EXTERNAL STORAGE
32© Copyright 2016 EMC Corporation. All rights reserved.
• REX-Ray– Provides a vendor agnostic storage orchestration engine– AWS, GCE, ScaleIO, XtremIO, many many more– Github: https://github.com/emccode/rexray
• mesos-module-dvdi– Provides a hook to Mesos agent nodes to provision, attach,
and detach external storage– Github: https://github.com/emccode/mesos-module-dvdi
HOW WE ACHIEVE THIS?
33© Copyright 2016 EMC Corporation. All rights reserved.
• We can run “real world” applications• Persistence – DBs• Highly Available, Fault Tolerant• Audit trail – Transaction Logs• Not worry about the “2am call”
STATELESS TO STATEFUL
34© Copyright 2016 EMC Corporation. All rights reserved.
LET’S FACE IT…
35© Copyright 2016 EMC Corporation. All rights reserved.
• Trouble is… this still represents the “old way” of thinking• PREVENT THE CRASH OR FAILURE!• Problem is you can’t• What is the alternative?
STATEFUL 2.0?
36© Copyright 2016 EMC Corporation. All rights reserved.
• Mesos frameworks can proactively monitor for thing going sideways• Leveraging external persistent storage and
frameworks fail fast and recover quicker!• Skynet!
FRAMEWORKS + STATEFUL 2.0
37© Copyright 2016 EMC Corporation. All rights reserved. 37© Copyright 2016 EMC Corporation. All rights reserved.
DEMO
38© Copyright 2016 EMC Corporation. All rights reserved.
• Running on Amazon EC2– Mesos– Marathon– Mesos DNS
• Production HA Cluster– 3 x Master + Zookeeper Nodes (t2.micros)– 5 x Slave Nodes with Docker Support (t2.medium)
DEMO CONFIGURATION
39© Copyright 2016 EMC Corporation. All rights reserved.
• Elastic Search Framework– Create and mount external volume– Pour some data into Elastic Search– Github: https://github.com/mesos/elasticsearch
• Elastic Search Instances backed by Persistent External Storage– Using REX-Ray– Using mesos-module-dvdi
DEMO: FRAMEWORK + EXTERNAL STORAGE
40© Copyright 2016 EMC Corporation. All rights reserved.
• Deploy Elastic Search Framework using REST API– Observe Scheduler creation– Using browser, log into the Elastic Search Framework UI– Observe Executor nodes get created and attaching to EBS
volumes• Simulate Failure
– Kill EC2 Instance– Watch executor recreation and reattach to EBS volumes
DEMO: FRAMEWORK + EXTERNAL STORAGE
41© Copyright 2016 EMC Corporation. All rights reserved.
DEMO: AUTO DEPLOY ELASTIC SEARCH
42© Copyright 2016 EMC Corporation. All rights reserved.
Data Persistence in the New Container WorldWednesday 3PM
Joshua BernsteinVP of Technology for ETD
Tobi Knaup CEO & Co-Founder of Mesosphere
Guru Session
43© Copyright 2016 EMC Corporation. All rights reserved.
@EMCcode@dvonthenen
emccode.comcommunity.emccode.com
Come visit us at Booth #1044 or in the vLab
Questions?
44© Copyright 2016 EMC Corporation. All rights reserved. 44© Copyright 2016 EMC Corporation. All rights reserved.