How to Manage Your Cloud by Drupal (DrupalCon CPH 2010)

PROPEOPLE26. aug 16:00

How to ManageYour Cloud by Drupal

Yas Naoi

The Purpose of This Presentation

•Learn about Cloud Computing

•Introduce Virtual Infra Manager

•Share Business Value

•Feedback from YOU!

Biz ValueDemoVirtual InfraCloud

Agenda

•Cloud Computing Overview

•Virtual Infra Manager

•Demo

•Business Value

•Q&A

Cloud Computing Overview

Centralized to the Internet

UsersUsers Service Service ProvidersProviders

Server

Networks

DevelopmentEnvironment

UIContents

Database

Application

Operations

Develop

Services

ProvideServices

DevelopersCommunity

UsersUsers Service Service ProvidersProviders

Server

Networks

DevelopmentEnvironment

UIContents

Database

Application

Operations

Low PricePCs

ServerVirtualizationBroadband

Network-centricServices

Develop

Services

ProvideServices

DevelopersCommunity

UsersUsers Cloud Service Service ProvidersProviders

Low PricePCs

ServerVirtualizationBroadband

Network-centricServices

Develop

Services

ProvideServices

Cloud Computing Features

•On-Demand Resources•Cost Reduction by

Pay-as-you-go Billing / Server Consolidation

•Easy Provisioning•Auto-scaling•Higher Productivity

Cloud Computing Architecture

Cloud Application ： Software as a Service

Photo VirtualDesktopBackupMailSearch

Cloud Infrastructure ： Infrastructure as a Service

Cloud Management Infrastructure ： Platform as a ServicePaaSPaaS

SaaSSaaS

IaaSIaaS

Log AnalysisMovie …Maps Chat

Virtual InfraManagement Security Auto-scalingLoad

Balancing …

PersistentStorage

MessageQueue

DistributedFile System

VirtualServer …

Virtual Infra Manager

Why Cloud for Service Providers?

•Problems

•Manage Tons of Servers

•Manage Peta-byte Class Data

•Requirements

•Downsize and Reduce the Cost

•Flexible Capacity Planning

•Problems

•Manage Tons of Servers

•Manage Peta-byte Class Data

•Requirements

•Cost Reduction

•Flexible Resource Management

•Cost Reduction

•Open Source

•Browser-based Easy Operations

•Flexible Resource Management

•Multiple Cloud Access

•Private

•Public

Private Cloud Requirements

•Browser-based UI

•Unified Access for Multiple Cloud

•Browser-based UI

•Management

•Linux & Windows

•Virtual Machine Image

•Server Template

•Network

•Audit Log and Reports

•Backup

•Security

•User Authentication

•Permissions

•Browser-based UI

•Management

•Linux & Windows

•Server Template

•Network

•Backup

•Security

•Permissions

•Monitoring

•Physical Servers / Cloud Nodes

•Virtual Machines

•Applications

•Browser-based UI

•Management

•Linux & Windows

•Server Template

•Network

•Backup

•Security

•Permissions

•Monitoring

•Physical Servers / Cloud Nodes

•Virtual Machines

•Applications

•Open Source

•Browser-based UI

•Management

•Linux & Windows

•Server Template

•Network

•Backup

The Goal ofVirtual Infra Manager

Multiple Public Cloud

C1- Amazon- Google-Windows

Azure- etc.Multiple Private Cloud

• Unified Management Console

• Flexible Resource Management

The Goal ofVirtual Infra Manager

Multiple Public Cloud

HybridHybridCloud PortalCloud Portal C2

C1- Amazon- Google-Windows

Azure- etc.Multiple Private Cloud

• Unified Management Console

• Flexible Resource Management

• Monitoring• Security• Backup• Failover, etc.

Architecture OverviewHome

Control Life Logs DataAnalysisHealthcare Location

Service…

Private Cloud

EucalyptusXCP

Public Cloud

Amazon

UserAuthN Permissions User

Management Access LogPortal UI JobQueue

Virtual InfraManagement

ServerCloning

LoadBalancing DatastoreFailover

StorageSnapshot

User ActivityAuditing

Batch JobExecution

Provisioning

Monitoring ServerImage Backup

Billing

PersistentStorage

MessageQueue

DistributedFile System memcached

SaaSPaaSIaaS

Why Drupal?Biz ValueDemoVirtual InfraCloud

Drupal + Custom Modules

Enomaly(Until Feb. 2010)

oVirt(As of Mar. 2010)

OpenXenManager

Developer - Enomaly Red Hat CitrixLicense GPL2 AGPL3 GPL2 GPL2Browser-based Yes Yes Yes Yes (Recently)Programming Language PHP Python Ruby on Rails

Python Python

Database MySQL MySQL PostgreSQL N/AHypervisor Support (Any) KVM / Xen / etc

(thru libvirt)KVM / Xen /etc

(thru libvirt) Xen Only

Install & Setup Easy Difficult Difficult EasyFunctionality (Yes) Yes w/ Limitations (Yes) Yes

API Drupal / REST REST REST(Not Exposed) XAPI / XML-RPC

Perspective Users (and Admin) Admin (and Users) Admin and Users Admin

Stability D6 Core Stable SometimesDatabase Corrupt

ImplementationImmature

SometimesHang-up

OpenXenManager

Python Python

SometimesHang-up

OpenXenManager

Python Python

SometimesHang-up

Why Drupal?

•Web Application Framework

• Highly-Modularized Architecture

• User Authentication & Management

• Permissions

• Key-value Pair Programming Style (Array)

• Queuing System Support (D7)

• NoSQL Support (D7)

Why Drupal?

• Permissions

Why Drupal?

• Permissions

Service…

Private Cloud

EucalyptusXCP

Public Cloud

Amazon

ServerCloning

StorageSnapshot

Batch JobExecution

Provisioning

Billing

PersistentStorage

MessageQueue

SaaSPaaSIaaS

Service…

Private Cloud

EucalyptusXCP

Public Cloud

Amazon

ServerCloning

StorageSnapshot

Batch JobExecution

Provisioning

Billing

PersistentStorage

MessageQueue

SaaSPaaSIaaS Drupal Core

Virtual Infra Manager•Works with Multiple Cloud

•Amazon EC2

•Eucalyptus

•XCP (Xen Cloud Platform)

Architecture

Development Module

Drupal Core Drupal Modules

Eucalyptus

Amazon EC2

Drupal

• Drupal Core as Web Application Framework

Hybrid Cloud Portal

Public

Private

UI Module

Existing Component

Architecture

Development Module

Eucalyptus

Amazon EC2AWS

Drupal

• Drupal Core as Web Application Framework• Drupal Custom Modules for Each Cloud, Templates, etc.

Hybrid Cloud Portal

Public

Private

Eucalyptus

XCPUI Module

Existing Component

Architecture

Development Module

Eucalyptus

Amazon EC2AWS

Drupal

Existing Component

• Drupal Core as Web Application Framework• Drupal Custom Modules for Each Cloud, Templates, etc.• Failover by Monitoring Virtual Machines

Hybrid Cloud Portal

Public

Private

Monitoring

Backup

Failover

Eucalyptus

XCPUI Module

DeploymentPrivate Cloud

• User AuthN• Permissions• User Activity Logs• VM Image Management• Provisioning• Server Template• Snapshot / Backup• Monitoring

Virtual InfraManager

RESTREST

XML-RPCXML-RPC

ActiveDirectory

LDAPLDAP

HTTPSHTTPS

Eucalyptus Cloud

Cloud Controller + Cluster Controller

Node Controller VM VM VM VMNode Controller VM VM VM VM

XCP Cloud

Pool Master VM VM VM

VM VM VM VM

Slave VM VM VM

VM VM VM VMSlave VM VM VM

VM VM VM VM

Storage

VMImage

• Provisioning• Monitoring• Alert

PhysicalServer

MaintenanceSystem

Systems Requirements

•Drupal 6.19

•Ubuntu 10.04

•Apache 2.2

•MySQL 5.1

•PHP 5.3

•Collectd 4.8

•RRDTool 1.3

• Nagios 3.2

ModulesCategory Module Name Description

Cloud Management

Common •System Common LibraryAmazon •Manage Amazon EC2Eucalyptus •Manage Eucalyptus CloudXCP •Manage XCP Cloud

Cloud Management

Provisioning

Registration •Manage Virtual Server ImagesServer Templates •Manage Server TemplatesCluster •Manage Server Template CollectionScripting •Manage Scripts for AutomationInput Parameters •Input Parameters for Running Virtual Server

Cloud Management

Provisioning

ReportingAudit •User Activity LogUsage Report •Cost Estimation Depending on Hours of UsageAlerts •Provides Alert Functionality

Cloud Management

Provisioning

ReportingAudit •User Activity LogUsage Report •Cost Estimation Depending on Hours of UsageAlerts •Provides Alert Functionality

ReliabilityFailover •Provide Failover FunctionalityBackup •Provide Backup Functionality

Problem in-the-CloudBiz ValueDemoVirtual InfraCloud

Problem in-the-Cloud•Async Operations for VMs on Virtual Infra

•Boot

•Shutdown

•Suspend / Resume

•VM Backup

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VM

Virtual Infra Operation

BootXCP Module

Drupal Core

Browser

Slave-2 VM VM VM

VM VM VM VM

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VM

Shutdown

XCP Module

Drupal Core

Browser

Slave-2 VM VM VM

VM VM VM VM

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VMSlave-2 VM VM VM

VM VM VM VM

Shutdown

Backup

XCP Module

Drupal Core

Browser

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VM

XCP Module

Drupal Core Boot

Shutdown

Backup

Browser

Slave-2 VM VM VM

VM VM VM VM

Shutdown

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VM

VM VM VM

VM VM VM VM

Bottleneck

XCP Module

Drupal Core Boot

Shutdown

Backup

Browser

Slave-2 VM VM VM

VM VM VM VM

Shutdown

Virtual Infra OperationUsing Drupal Queue

XCP Module

Browser

Drupal Core

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VM

Dequeue

Drupal QueueCron

Enqueue

Drupal QueueBootBoot

ShutdownShutdown

SuspendSuspend

BootBoot

Virtual Infra OperationUsing Drupal Queue

XCP Module

Browser

Drupal Core

XCP Cloud

Master VM VM VM

VM VM VM VM

Slave-1 VM VM VM

VM VM VM VM

Dequeue

Drupal QueueCron

Enqueue

Drupal QueueBootBoot

ShutdownShutdown

SuspendSuspend

BootBoot

VM Image BackupUsing Drupal Queue

XCP Module

Drupal Core

Enqueue

Dequeue

Drupal Queue

Drupal QueueCron

Persistent Storage

Image RepositoryUpload Im

3 – 20 min+ 3 – 10 min+

1. Shutdown VM2. Bundle Image3. Upload Image

Browser

Wn …Worker Process

XCP Module

Drupal Core

Enqueue

Dequeue

Drupal Queue

Drupal QueueCron

Persistent Storage

3 – 20 min+ 3 – 10 min+

Browser

XCP Module

Drupal Core

Enqueue

Dequeue

Drupal Queue

Drupal QueueCron

Persistent Storage

3 – 20 min+ 3 – 10 min+

Browser

Business Value byVirtual Infra Manager

Cost ReductionBiz ValueDemoVirtual InfraCloud

Cost ReductionTraditional Traditional

Cost Reduction

Install Windows

Traditional Traditional

Cost Reduction

Windows Update

Install Windows

Cost Reduction

Software Installation

Windows Update

Install Windows

Cost Reduction

Printer Setup

Windows Update

Install Windows

Cost Reduction

Printer Setup

Network Setup

Windows Update

Install Windows

Cost Reduction

Network Setup

Just Copya Pre-configured VM Image

Virtual Infra Virtual Infra

Printer Setup

Network Setup

Windows Update

Install Windows

Business Value (1/2)•Virtual Infra Manager•No Promising OSS•Reference Model and Implementation for

Cloud Management•Multiple Cloud Support •Security•Reliability•Scalability

Business Value (2/2)•Virtual Infra Manager•Effective Cost Reduction of Cloud

Management

•Unified Access to Multiple Cloud•Easy Provisioning•Easy Operations•Open Source

Future DirectionMail Maps DocsSearch Location

Service…

Drupal 7 Core

User & AuthNManagement Permissions Report Cache

Custom Modules

ServerCloning

Provisioning

StorageSnapshot

Automation

Failover

Billing

Portal UI

Batch JobExecution

Private Cloud

EucalyptusXCP MySQL Mem

cachedVarnish

Public Cloud

Amazon Google MongoDBPersistentStorage

SaaSPaaSIaaS Custom ExistingD7 Core

Future DirectionMail Maps DocsSearch Location

Service…

Drupal 7 Core

User & AuthNManagement Permissions Report Cache

Custom Modules

ServerCloning

Provisioning

StorageSnapshot

Automation

Failover

Billing

Portal UI

Batch JobExecution

Private Cloud

EucalyptusXCP MySQL Mem

cachedVarnish

Public Cloud

Amazon Google MongoDBPersistentStorage

SaaSPaaSIaaS

Virtual Infra ManagerMore Scalable and Reliable

Cloud Management Platformon D7

Custom ExistingD7 Core

http://cph2010.drupal.org/node/14728

Biz ValueVirtual InfraCloudBackground

Yas Naoi

naoi@docomolabs-usa.com

Thank You!“Coming together is a beginning, keeping together is a progress; working together is a success.”- Henry Ford

Biz ValueVirtual InfraCloudBackground

Yas Naoi

naoi@docomolabs-usa.com

Backup

Hybrid (Public / Private) Cloud ManagementHybrid (Public / Private) Cloud Management

Instance MonitoringInstance Monitoring

Supports Windows and Linux VMs

Report – Usage Estimate

How to Manage Your Cloud by Drupal (DrupalCon CPH 2010)

Technology

Transcript of How to Manage Your Cloud by Drupal (DrupalCon CPH 2010)

Drupal at ASU - Drupalcon 2010

State of Drupal keynote, DrupalCon New Orleans

DrupalCon Europe Dublin Sponsor... · 2015-12-07 · DrupalCon Europe 26 30 September 2016 The Convention Centre Dublin Dublin, Ireland DrupalCon is the heartbeat of the Drupal community.

Drupalcon Sydney - Show Me the Tests! Writing Automated Tests for Drupal

From Photoshop to Drupal Theme (DrupalCon San Francisco 2010)

DRUPALCON NEW ORLEANS€¦ · DRUPALCON NEW ORLEANS May 1012, 2016 ANNUAL SUPPORTING PROGRAMS In addition to sponsoring DrupalCon, your company can help fund the Drupal Association’s

Putting Drupal in the Cloud with Red Hat's OpenShift PaaS #DrupalCon/Prague

State of Drupal keynote, DrupalCon Prague

Multilingual Content in Drupal 7 & 8 at DrupalCon Munich

DrupalCon Chicago Practical MongoDB and Drupal

DrupalCon Prague 2013: AUTOMATE DRUPAL DEPLOYMENTS WITH LINUX CONTAINERS, DOCKER AND VAGRANT

Understanding the Semantic Web and Drupal - DrupalCon Chicago

DRUPALCON NEW ORLEANS › files...DRUPALCON NEW ORLEANS May 10 12, 2016 ANNUAL SUPPORTING PROGRAMS In addition to sponsoring DrupalCon, your company can help fund the Drupal Association’s

Drupal architectures for flexible content - Drupalcon Barcelona

DrupalCon Amsterdam 2014 - Between structure and flexibility: Drupal for Marketeers

@moRtenDk #drUpaltWIG - DrupalCon | Be Human, Think … · tHemIng DRUpALCoN BaRCElONA 2015 DRUpAl 8. @moRtenDk #drUpaltWIG SLIdES aRE oNLInE In 60 MInUTEs sLidEs. @moRtenDk ... data-drupal-foo

VoIP Drupal Framework DrupalCon Prague

Voip Drupal presentation at DrupalCon SF 2010

DrupalCon Sydney - Selling Drupal to Large Enterprises

How To Contribute To Drupal Drupal - DrupalCon Paris