Cloud Computing, Docker, Mesos, DCOS, Container, Big Data, Paas

Click here to load reader

  • date post

    16-Apr-2017
  • Category

    Technology

  • view

    746
  • download

    0

Embed Size (px)

Transcript of Cloud Computing, Docker, Mesos, DCOS, Container, Big Data, Paas

  • 2016XavientInformationSystems

    CloudComputingandDCOS

  • 2016XavientInformationSystems

    Agenda

    IaaS

    PaaS

    TechnologyOverview

  • 2016XavientInformationSystems

    IaaS

  • 2016XavientInformationSystems

    InfrastructureasaService isaformofcloudcomputingthatprovidesvirtualizedcomputingresources.

    PublicCloud

    AWS Azure GCE

    PrivateCloud

    VMWare OpenStack ApacheCloudStack

    IaaS

  • 2016XavientInformationSystems

    PaaS/DCOS

  • 2016XavientInformationSystems

    Platformasaservice (PaaS)isacategoryofcloudcomputingservicesthatprovidesaplatformallowingcustomerstodevelop,run,andmanageapplicationswithoutthecomplexityofbuildingandmaintainingtheinfrastructuretypicallyassociatedwithdevelopingandlaunchinganapp

    DCOS isanewkindofoperatingsystemthatorganizesallofyourmachines,VMs,andcloudinstancesintoasinglepoolofintelligentlyanddynamicallysharedresources.ItrunsontopofandenhancesanymodernversionofLinux

    Paas andDCOS

  • 2016XavientInformationSystems

    NotoStaticPartitioning

    Underutilize Doesnotscale Hardtodealwithfailures

  • 2016XavientInformationSystems

    MesosEcosystem

  • 2016XavientInformationSystems

    Architecture

  • 2016XavientInformationSystems

    Clusterconfigurationexample

  • 2016XavientInformationSystems

    SoftwareonMesos

    Aurora isaserviceschedulerthatrunsontopofMesos,enablingyoutorunlong-runningservicesthattakeadvantageofMesos'scalability,fault-tolerance,andresource

    isolation.

    Marathon isaprivatePaaSbuiltonMesos.Itautomaticallyhandleshardwareorsoftwarefailuresandensuresthatanappisalwayson.

    Hadoop RunningHadooponMesosdistributesMapReducejobsefficientlyacrossanentirecluster.

    Spark isafastandgeneral-purposeclustercomputingsystemwhichmakesparalleljobseasytowrite.

    Chronos isadistributedjobschedulerthatsupportscomplexjobtopologies.Itcanbeusedasamorefault-tolerantreplacementforCron.Jenkins isacontinuousintegrationserver.Themesos-jenkinspluginallowsittodynamicallylaunchworkersonaMesosclusterdependingontheworkload.

    Cassandra isaperformantandhighlyavailabledistributeddatabase.Linearscalabilityandprovenfault-toleranceoncommodityhardwareorcloudinfrastructuremakeitthe

    perfectplatformformission-criticaldata.

    ElasticSearch isadistributedsearchengine.Mesosmakesiteasytorunandscale.

  • 2016XavientInformationSystems

    MesosArchitectureinHAmode

    Behindthescenes,ApacheMesosusesApacheZooKeeper,adistributedconfigurationstoretopersistitsstate.The"state"ofMesosincludeseverythingfromconfigurationtothestatusandloadofindividualMesosnodes,aswellasframeworkslikeCassandra-Mesos.

  • 2016XavientInformationSystems

    PaaSonMesos

  • 2016XavientInformationSystems

    Marathonuse

    Constraintscontrolwhereappsruntoallowoptimizingforeitherfaulttolerance(byspreadingataskoutonmultiplenodes)orlocality(byrunningallofanapplicationstasksonthesamenode).Thereare

    theredifferentpartsa)fieldb)operatorc)optional

    "constraints":[["rack_id","CLUSTER","rack-1"]],"constraints":[["hostname","CLUSTER","a.specific.node.com"]]"constraints":[["rack_id","GROUP_BY"]]

    Example:https://github.com/mesosphere/marathon/tree/master/examples

    curl-i -H'Content-Type:application/json'-d@localhost:8080/v2/apps

  • 2016XavientInformationSystems

    Reservationproblem DataPersistenceproblem Whereisthesecurity?AuthenticationandAuthorization Monitoring FrameworkRateLimiting Vendorsupport Managedservices

    Letsgetpractical

  • 2016XavientInformationSystems

    Resourcesarereservedforrolesandrolesusedtoreserveresources StaticReservationmesos-slave\ --master=:\ --resources="cpus:4;mem:2048;cpus(ads):8;mem(ads):40968CPUsand4096MBofRAMreservedforadsonthisslave.

    DynamicreservationinMesos>=0.23.0 CreatethroughFrameworkandOperatorAPI

    Framework Operator

    Reservation-mechanisms to reserve resourcesinspecificslaves

  • 2016XavientInformationSystems

    Amechanismtocreateapersistentvolumefromdiskresources PersistentvolumesenablestatefulservicessuchasHDFS andCassandra tostoretheirdata

    withinMesos Authorizationismusttocreatepersistentvolumes Persistentvolumescanonlybecreatedfrom reserveddiskresources,whetheritbe

    staticallyreservedordynamicallyreserved- Important CreatethroughFrameworkandOperatorAPI(SameasDynamicReservation)

    https://mesosphere.github.io/marathon/docs/persistent-volumes.html

    https://mesosphere.github.io/marathon/docs/external-volumes.htmlAnexternalstorageserviceenablesyourappstobemorefault-tolerant.Ifahostfails,Marathonreschedulesyourapponanotherhost,alongwithitsassociateddata,withoutuserintervention.https://github.com/emccode/rexray

    PersistentVolumes

  • 2016XavientInformationSystems

    Persistentvolume

  • 2016XavientInformationSystems

    Authenticationisdisabledbydefault.Whenauthenticationisenabled,operatorscanconfigureMesostoeitherusethedefaultauthenticationmoduleortousea customauthenticationmodule.

    AuthenticationpermitsonlytrustedentitiestointeractwithaMesoscluster Authenticationcanbeusedatframework,agentandoperatorlevel Configureauthenticationduringthemesosmasterandslavestart Authorization,localandcustom Configuretheactionsthatcertainprincipalsareallowedtoperform

    http://mesos.apache.org/documentation/latest/authentication/http://mesos.apache.org/documentation/latest/authorization/

    *ThedefaultMesosauthenticationmoduleusesthe CyrusSASLlibrary.SASLisaflexibleframeworkthatallowstwoendpointstoauthenticatewitheachotherusingavarietyofmethods.Bydefault,MesosusesCRAM-MD5authentication.

    Security

  • 2016XavientInformationSystems

    Whatsgoingon?

  • 2016XavientInformationSystems

    MesosandContainers

    "If a Docker application is a Lego brick, Kubernetes would be like a kit for building the Millennium Falcon and the Mesos cluster would be like a whole Star Wars universe made of Legos." ~ Solomon

  • 2016XavientInformationSystems

    Mesos+Docker+Kubernetes

    MesosclusterscanbedeployedonnearlyeveryIaaScloudproviderinfrastructureorinyourown

    physicaldatacenter.KubernetesonMesosrunson-topofthatandthereforeallowsyoutoeasilymoveKubernetesworkloadsfromoneoftheseenvironmentstotheother.http://kubernetes.io/docs/getting-started-guides/mesos/

  • 2016XavientInformationSystems

    CassandraandMesos/Marathon

    https://mesosphere.github.io/cassandra-mesos/

    https://github.com/mesosphere/dcos-cassandra-service

    https://docs.mesosphere.com/1.7/usage/services/cassandra/install-and-customize/

  • 2016XavientInformationSystems

    https://mesosphere.com/blog/2015/11/05/jenkins-dcos-mesos/

    Jenkins

  • 2016XavientInformationSystems

    Auroravs.Marathonhttp://code.hootsuite.com/marathon-vs-aurora-a-high-level-comparison/

    MesosandKuberneteshttp://stackoverflow.com/questions/26705201/whats-the-difference-between-apaches-mesos-and-googles-kubernetes

    JenkinsandMesoshttp://www.slideshare.net/carlossg/ci-and-cd-at-scale-scaling-jenkins-with-docker-and-apache-mesos

    BloombergandMonitoringhttp://schd.ws/hosted_files/mesosconna2016/98/SeamlessMonitoringForMesosClusters.pdf

    UberandCassandrahttp://schd.ws/hosted_files/mesosconna2016/60/mesoscon-uber.pdf Mesoscon 2016

    https://www.youtube.com/playlist?list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC Jenkinsalternatesolutionhttps://github.com/HubSpot/Blazar Mesosoversubscriptionhttp://mesos.apache.org/documentation/latest/oversubscription/ https://support.mesosphere.com/hc/en-us/categories/200383389-Technical

    Read

  • 2016XavientInformationSystems

  • 2016XavientInformationSystems

    CloudFoundry

    CloudFoundryisanopen-sourceplatformasaservice(PaaS)thatprovidesyouwitha

    choiceofclouds,developerframeworks,andapplicationservices

  • 2016XavientInformationSystems

    Architecture- PCF

  • 2016XavientInformationSystems

    RedHat OpenShift (onRadar)

  • 2016XavientInformationSystems

    Q&A

  • 2016XavientInformationSystems

  • 2016XavientInformationSystems

    Xavients Capabilities: BigDataandAnalytics

  • 2016XavientInformationSystems

    Agenda

    XavientIntroduction

    CompanySnapshot GlobalPresence OurDifferentiators

    XavientBigData&BICapabilities

    CaseStudies

  • 2016XavientInformationSystems

    CompanySnapshot

    WhoWeAre

    Founded in US in 2002 | Global Scale |Exponential Growth

    Fortune 200 clientele | Partner of choicefor4out of Top5CSPs inUS

    Comprehensive Technology Coverage &PersonalizedService

    WhatWeDo

    Enable Enterprise Class Solutions onWeb, Mobility, Open Source, Cloud,

    BigData

    Focus on Telecom, Retail, Healthcare&Banking

    OptimizeBAUSDLC Practices

    HowWeDeliver

    Customer First Approach - Long termongoing engagements are testimony toour success

    Talent Farming Thought leadership |Bootcamps | Embedded Factory

    Maturity & Agility TechnologyInnovation | Reusable Best Practices |Early Adopters

  • 2016XavientInformationSystems

    GlobalPresence

    Tampa, FL

    Atlanta,GA

    Herndon,VA

    Sterling, VA

    Bethpage, NY

    Frisco,TX

    Englewood, CO

    Denver, CO

    St.Louis,MO

    Bellevue,WA

    SimiValley,CA

    (Headquarters)

    Noida

    Bengaluru

    OnshoreDeliveryCenter/

    ClientLocation

    OffshoreDeliveryCenter

    Mississauga, Canada

  • 2016XavientInformationSystems

    Xavient EnablingDigitalTransformation

    Mobility

    OpenSource

    BigData

    IoT

    UI/UX

    Dataingestionandanalys