3-2-1 Action! Running OpenStack Shared File System Service in Production

download 3-2-1 Action! Running OpenStack Shared File System Service in Production

of 38

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of 3-2-1 Action! Running OpenStack Shared File System Service in Production

  • 3-2-1, Action Running OpenStack Shared File System Service

    in Production

    OpenStack Summit Barcelona Oct 25, 2016

  • Todays Directors

    Anika Suri

    Tom Barron

    Sean Cohen

    Technical Alliance Manager NetApp


    Senior Software Engineer Red Hat


    OpenStack Product Manager Red Hat


  • Scenes to be shot today

    Share of Deployments of Openstack in Production Market trends Meet our main character-Manila Action scenes Production-grade Deployments of Manila Shares The road to Ocata Some production considerations Supporting work for Manila in other projects Key takeaways Q&A

  • from Share of Deployments of OpenStack in Production

    to Production-grade Deployments of Manila Shares

  • OpenStack Foundation October 2016 Survey Share of deployments in production is 20% greater than a year ago

  • Strong OpenStack Deployment in Production and Pilots

    Red Hat has over 350+ private cloud customers and ~40 POCs underway at any given time, which aligns with this data.

    Over 40% are in production and/or are undergoing a proof of concept with OpenStack, while 20% are preparing to deploy OpenStack

    Source: OpenStack & Cloud Customer Insights Research, September 2016, Red Hat 7

  • OpenStack Used for Variety of Legacy & New Workloads

    OpenStack is not used only for net-new, cloud-native workloads but for a variety of existing and new workloads

    Over 60% are using OpenStack for existing virtualized workloads as well as new cloud-optimized workloads

    Source: OpenStack & Cloud Customer Insights Research, September 2016, Red Hat 8

  • 9 NetApp Insight 2016 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

    Of all storage sold is for file-based use cases * Per IDC

  • OpenStack Foundation 2016 Survey Project adoption for emerging projects forecasts growth (October 2016)

  • NetApp is the founder of, leading contributor to, and most widely deployed option for Manila

    OpenStack Foundation 2016 Survey

    11 NetApp Insight 2016 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

    Shared File System Drivers in Use (April 2016 )

  • The Age of Cloud Share Services File Share Service Market Trends

    OpenStack Manila: June 2013 Microsoft Azure Files: May 2014 Amazon Web Services Elastic File Services: April 2015

  • Meet our Main Character

  • Meet OpenStack Shared File System Service Multi-tenant, secure file share as a service

    An Open, Standard API for File

    System Provisioning and Management that is designed to provide a multi-tenant, secure file share as a service across OpenStack Compute instances.

    Vendor neutral API for provisioning and attaching filesystem-based storage such as NFS, CIFS, CephFS, HDFS and other network filesystems.

  • Action scenes

  • Manila Use Cases Support traditional enterprise applications Provide Elastic and Scale file shares Support cloud-native workloads, such DBaaS Big Data via Manilas HDFS native driver plugin DevOps / On-Demand development and build environments Integration with existing automation frameworks through REST

    API or CLI Hybrid Cloud shares (external consumption of shares / migration

    of workloads to the cloud from on-premise file shares) NFV/Telco - use case example: store emails headers and bodies

    in Manila shares and the the attachment in object storage. Persistent Shared storage for Containers

  • Use Case I Move traditional workloads without rewriting them

    Manila service makes it easier to

    transition traditional virtualized enterprise applications workloads into the cloud infrastructure without requiring to rewrite the applications stack, while gaining lower cost and more heterogeneous infrastructure.

    Key requirement: Shared resources High Availability Disaster Recovery

  • Use Case II Storage as a Service (Private or Public Cloud)

    Extend your Storage as a Services with your own shared file systems (Similar to AWS EFS)

    Manila delivers a durable, high-throughput file system for content-management systems that store and serve information for a range of applications, such as websites, online publications, email attachments and archives.

  • Use Case III DevOps

    Challenge: Speed up Dev and Test refresh

    Solution: Run test in parallel by leveraging

    containers and cloned Manila shares. Automate database deployment by

    leveraging both custom and Trove-based services.

    Infrastructure self-service

  • Production-grade Deployments of Manila Shares

  • Manila in Red Hat OpenStack Platform 10

    Production-grade automated deployment of Manila service with OSP director with High Availability

    Horizon dashboard support (via Manila UI plugin)

  • Integration with NetApp Manila Driver

    With OSP 10 release, the director can now deploy the Shared File System with a NetApp on the overcloud.

    NetApp unified driver Certified backend Allows the Shared File System service to use NetApp storage controllers

    (running Data ONTAP) as a back end.

  • Integration with NetApp Manila Driver In addition to all the standard features all Manila drivers support, the NetApp driver supports: Expand/shrink shares (All drivers can expand but shrink is optional) Manage/unmanage shares (for importing and exporting data to/from Manila) Snapshot and clone from snapshot are both supported and space/time

    efficient Share replication (to address DR use cases) Share servers (in addition to no-share-servers) for extra tenant isolation

  • The road to Ocata

    PIc credit: K. Basil

  • Manila features in Newton

  • Manila Today -

    Number of Drivers: 23 Blueprints Completed: 14 Specs* new with Newton:

    13 specs proposed

    5 accepted Share Migration Improvements Store auth secrets in access map

    Share networks span subnets Share revert-to-snapshot Hierarchical Port Binding

    * https://review.openstack.org/#/q/project:openstack/manila-specs

  • New in Newton Snapshot instance admin APIs Extend share and extend snapshot APIs enhancements Scheduler improvements for thin provisioning, driver filter and

    Goodness weigher Access list api enhanced to return access key Network gateway & MTU DB support; port-binding support Many share migration and data service fixes and improvements* Share replication improvements*

    * Experimental Features


  • Some Production Considerations... Dont use Reference Implementation drivers in production! Generic driver is unstable and not cut for scale

    Core vs. Experimental and Microversioned APIs Experimental Features: Consistency Groups Consistency Group Snapshots Share Migration and Data Services Share Replication

    Experimental APIs may be changed at will, even removed. Microversions allow new implementations to break backwards compatibility while maintaining support for earlier versions of the APIs.

  • Newton share driver work

    New drivers: Docker, Tegile IntelliFlash, Hitachi HSP, NexentaStor

    NetApp cDOT driver hybrid aggregate support NetApp cDOT driver snapshot manage/unmanage support CephFS Native Driver (Read-only share support) CephFS Native Driver returns access key Huawei added share replication support EMC Unity driver added NFS/CIFS plugin Hitachi HNAS CIFS support GlusterFS new volume XML schema support GPFS driver NFS Ganesha support ZFSonLinux manage-snapshot support Oracle ZFSSA driver supports manage/unmanage shares Various fixes and enhancements to HPE 3PAR, Hitachi HSP, Huawei, Native

    CephFS, NetApp cDOT, ZFSonLinux, and Windows drivers.

  • Supporting work for Manila in other projects TripleO heat templates and puppet manifests using new

    composable/custom roles infrastructure to support overcloud deployment with NetApp and native CephFS backends.

    Ganesha work on dynamic update of exports via d-bus Lack of this capability was holding up use of Ganesha by Docker driver Glusterfs and GPFS drivers use today CephFS driver will use Ganesha for NFS gateway

    Tempest library stable interfaces to improve CI stability Security Vulnerability Management (when CVEs hit), Barbican example Monitoring, Troubleshooting and Telemetry Kolla have added the support for deploying Manila (for running Manila

    services containerized) Fuxi Docker Manila use for shared persistent volume

  • New High Availability architecture


    Pacemaker systemd HAProxy / VIPs A/A OpenStack Services

    RabbitMQ Memcached

    Galera manila-api

    manila-share manila-scheduler


    Default for new deployments, upgrades to OSP 10 and higher Fencing enabled for Pacemaker managed services when/where required

    (testing to determine this)

    TripleO Manila Service HA Architecture

  • Ocata focus and beyond New spec process has been proposed to provide more focus

    Have fewer features in flight, more finished Begin journey to scalable, active-active services

    consistent model for avoiding races How to scale out services across multiple nodes while still avoiding races Exploit composable role TripleO infra

    High Availability & Disaster Recovery Replication,