Provisioning and Management of Storage in the Docker · PDF file contributions from Docker,...

Click here to load reader

  • date post

    24-Jun-2020
  • Category

    Documents

  • view

    5
  • download

    0

Embed Size (px)

Transcript of Provisioning and Management of Storage in the Docker · PDF file contributions from Docker,...

  • Anusha Ragunathan

    Provisioning and Management

    of Storage in the Docker

    platform

    Sr Software Engineer, Docker

  • Agenda

    Docker Volume Plugins (DVP) on Engine &

    Swarm

    Docker Enterprise & Kubernetes

    Storage Ecosystem

  • Storage Ops is Hard A key demand enterprises have is that developers should be able to store data

    in clusters without having to worry about how persistent storage is working

    under the hood.

    Provision

    Volumes

    Deploy

    Apps HA DR

    Understand

    Your

    use-case

    Map

    use-case to

    plugins

  • Life of a volume: Single node

    CreateVolume

    UnMountVolume (from

    container)

    MountVolume (to container

    and format if necessary)

    DeleteVolume (after use)

  • A way to extend the docker engine's volume subsystem on a

    single node. An out-of-tree implementation for:

    ● Create/Remove volumes

    ● Mount/Unmount volumes

    ● Get Volume Capabilities

    ● List volumes

    Examples: DVP include NetApp, VMware vSphere,

    PureStorage plugins.

    Docker Volume Plugins: Engine

  • Life of a volume: Cluster aware CreateVolume

    AttachVolume (to Node)

    Format and partition volume, if necessary

    MountVolume (to container)

    UnMountVolume (from container)

    DetachVolume (from Node)

    DeleteVolume (after use)

  • Docker Swarm does not orchestrate volume creation across a

    cluster.

    • Docker Swarm makes use of volume names instead of

    volume ID as unique volume identifier

    • Volume requests are simultaneously send to each node in a

    Swarm cluster.

    • Volume Plugins must run independently on each node in a

    Swarm cluster.

    These results in issues like race conditions that can result in

    the creation of a large number of volumes for a single request

    without a clear winner.

    Docker Volume Plugins: Swarm

  • ● Around the same time that Docker Swarm was considered

    getting re-architected to be volume aware, a new industry

    standard was evolving in the container orchestration

    community.

    ● Container Storage Interface (CSI) started and matured by

    contributions from Docker, Kubernetes and Mesos.

    ● Today, CSI specification has reached 1.0. Container

    Orchestrators such as Kubernetes and Mesos have

    supported for CSI.

    ● Docker Enterprise is a strong integrator for Kubernetes and

    works well with CSI and storage providers.

    Container Storage Interface

  • Docker Enterprise + Kubernetes

    ● A storage claim made by a

    user

    ● Just like how Pods consume

    Compute resources, PVC

    consume Volume resources.

    ● Just like how Pods can

    request specific levels CPU &

    memory, PVCs can request

    specific sizes and access

    modes.

    ● Pods reference PVC

    Persistent Volume

    Claim (PVC)

    Persistent Volume (PV)

    ● Storage resources in a cluster

    ● Lifecycle independent of a Pod

    Pods

    ● A set of running containers

    representing a workload

  • ● Provisioning is the creation/allocation of Persistent Volumes

    ● Static and Dynamic Provisioning

    ○ Static: pre creation of PV

    ○ Dynamic: automatic creation of PV based on size,

    permissions requested.

    ● Dynamic Provisioning through Storage Classes

    ○ Provides a way for Admins to describe “classes” of

    storage available. Eg, different performance SLAs,

    value-add features such as replication, backup, etc

    ○ Backed by a provisioner

    PV Provisioning

  • ● Provisioning is the creation/allocation of Persistent Volumes

    ● Static and Dynamic Provisioning

    ○ Static: pre creation of PV

    ○ Dynamic: automatic creation of PV based on size,

    permissions requested.

    ● Dynamic Provisioning through Storage Classes

    ○ Provides a way for Admins to describe “classes” of

    storage available. Eg, different performance SLAs,

    value-add features such as replication, backup, etc

    ○ Backed by a provisioner

    PV Provisioning

  • Provisioning Workflow (static)

    1. Cluster Admin

    pre-provisions

    volumes and

    registers PVs

    2. Developer

    claims a PV

    from the pool

    4. Developer

    references the

    claim in a Pod

    3. Controller BINDS

    PV to PVC

    Pool of Persistent Volumes

    NFS PV iSCSI PV EBS PV

    Claim

    `Claim

    ClaimClaim

    Pod

    Claim

    5. Controller inspects

    Claim and MOUNTs

    the PV into the Pod.

  • Provisioning Workflow (dynamic)

    1. Cluster Admin

    registers

    Storage Classes

    2. Developer defines a

    claim by referring a

    storage class

    5. Developer

    references the

    claim in a Pod

    4. Controller BINDS

    PV to PVC

    6. Controller

    inspects Claim

    and MOUNTs the

    PV into the Pod.

    SSD

    Pod

    Claim

    Claim

    Persistent

    Volumes Storage

    Classes

    Slow

    Fast

    3. Controller

    provisions

    volumes

    on-demand

  • Demo 1: PV Provisioning using iSCSI

    Worker Worker

    Master

    ISCSI Target Server

  • High Availability (HA) ● HA involves handling Node failures and Storage failures.

    ● Node failures:

    ○ Node drain: Maintenance

    ○ Node failures: Node lost/ kubelet crash

    ● Kubernetes provides built-in support for node failures. The

    attach-detach controller on the master is in charge of

    interacting with the volume plugin and moving PVs across

    nodes.

    ● If delays/instability with attach/detach is not desired, use a

    Software Defined Storage (SDS) solution.

  • Plugin Type Pros Cons Future

    InTree No extra installation

    necessary

    Release cycles tied to

    k8s releases Superseded by CSI

    External

    Provisioner

    Code maintained

    independently

    Limited customization for

    attach and mount ops Superseded by CSI

    FlexVolume Highly customizable code

    maintained independently

    Host based exec model

    reduces portability

    Deprecated for Linux

    Will be used for Windows

    CSI

    Highly customizable code

    maintained independently

    and based on standard

    Requires installation and

    configuration

    The future of storage

    plugins

    Kubernetes Storage Plugin Ecosystem

  • In-tree Plugin Scenarios

    Cloud Protocol Ephemeral SDS Extensions

    AWS EBS NFS local Portworx FlexVolume

    GCE PD iSCSI hostpath StorageOS CSI

    Azure Disk/File Fibre Channel config_map ScaleIO

    Openstack Cinder secret

    vSphereVolume

  • In tree Plugin Architecture

    Worker Worker

    Master

    Storage Service

    Kubelet Kubelet

    AttachVolume

    DetachVolume

    CreateVolume

    DeleteVolume

    Kube Core Components

    MountVolume

    UnmountVolume

    AttachVolume

    DetachVolume

    CreateVolume

    Delete Volume

    In-tree Plugins

    PV Claim

    API Server

  • External Provisioner Scenarios

    In-tree PV source Implementations [not certified with UCP yet]

    iSCSI Netapp-Trident, Dell/EMC-Isilon, HPE-Nimble, HPE-3PAR, Nutanix,

    OpenEBS-iscsi

    NFS Netapp-Trident, AWS-EFS, Dell-EMC Isilon

  • External Provisioner Architecture

    Worker Worker

    Master

    Storage Service

    Kubelet Kubelet

    External Provisioner (StatefulSet/Deployment Pod)

    Kube Core Components External components

    API Server

    MountVolume UnmountVolume

    AttachVolume DetachVolume

    CreateVolume

    Delete Volume

    In-tree Plugins

    PV Claim

    PV

  • CSI Scenarios

    Cloud Local/ephemeral

    On-prem SAN/NAS SDS

  • CSI Plugins

    Worker Worker

    Storage Service

    Kubelet Kubelet

    External

    Provisioner (Deployment/

    StatefulSet)

    CreateVolume

    DeleteVolume

    External

    Attacher (Deployment/

    StatefulSet)

    CSI Controller (Deployment/

    StatefulSet)

    CSI Node (DaemonSet)

    Kube Core Components Kube Sidecar containers CSI plugin components

    ControllerPublish

    ControllerUnPublish

    Master

    API Server

    NodeStage/NodeUnstage NodePublish/NodeUnpublish

    PV

    PV Claim

    volume attachment

    CSI In-

    Tree

  • Future enterprise features in CSI

    ❖Backup

    ➢Snapshot/restore

    ➢Application consistent/triggered snapshots

    ❖Volume resizing

    ❖Cloning and Replication

  • Demo 2: PV Provisioning using CSI AWS EBS

    Worker Worker

    Master