MongoDB on Windows Azure

MongoDB on Windows Azure

WIP Report

MongoBoston September 2010

Brief Survey

• Business Decision Makers?

• IaaS (ex. AWS or RackSpace) users

Management of Resources: What you manage

Applications

Development &

Runtime Kernels

Databases

Security, Management,

Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

PRIVATE(On-Premise)

Applications

Development &

Runtime Kernels

Databases


Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

Infrastructure

as a Service (IaaS)

Applications

Development &

Runtime Kernels

Databases


Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

Platform as a

Service (PaaS)

Yo

u M

an

ag

e Yo

u M

an

ag

e

Yo

u M

an

ag

e

Ma

na

ge

d b

y V

en

do

r

Ma

na

ge

d b

y V

en

do

r

Applications

Development &

Runtime Kernels

Databases


Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

Software as a

Service (SaaS)

Yo

u C

on

fig

ure

Ma

na

ge

d b

y V

en

do

r

Windows Azure is an internet-scale cloud services platform hosted in Microsoft data centers around the

world, proving a simple, reliable and powerful platform for the creation of web applications and

services.

The Windows Azure Platform

Defining the Windows Azure Architecture

GOAL: Massive Scalability

Scale out – not up – by replicating

worker instances as needed.

Allow applications to scale

user and compute processing

independently.

Two role types: Web Role & Worker RoleWindows Azure applications are built with web roles, worker roles, or a combination of both deployed to a number of instances.

Each instance runs on its own VM (virtual machine), replicated as needed

Compute Services in Windows Azure

BLOBS: Provide a simple

interface for storing

named files along with

metadata for the file.

TABLES:

Provide structured

storage. A table is

a set of entities

which contain a set

of properties.

QUEUES:

Provide reliable

storage and delivery

of messages for

an application.

Blobs, Tables, Queues and Drives

DRIVES:

A durable NTFS file

system volume,

sharable across

instances.

Running even one Mongo instance

• Azure runs Windows 2008 server VMs– .NET 4 included. All else must be bundled.

– XCOPY = GOOD. Registry and MSI = BAD

– MongoDB = XCOPY = GOOD!!!

• Mongo needs disk storage.– Choose: non-durable or durable

– Local disk: non-durable. Blob: durable!

• Mongo needs connection port– Multiple instances can’t talk to each other

– Client can’t choose server instance

– Can’t scale – multiple instances are independent!

– Azure uses random ports – Mongo needs to use this port• Mongod won’t allow mapping of http server port!

Single-Instance Solution

• Single worker role instance

• Local (non-durable) or blob (durable)

storage

• Single port mapped to mongod.exe

– Server only; no web server access

• Mongod instance as spawned process

– Not as service

– Must specify mapped port, data path, no http

• If server exits, recycle instance

Here’s How It’s Done

var storage = RoleEnvironment.GetLocalResource("MongoDBStorage");

varstoragePath = storage.RootPath;

string mongoRoot = Path.Combine(Environment.GetEnvironmentVariable("RoleRoot") +

@"\", @"approot\MongoDB");

intport = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["MongoIn"].IPEndpoint.Port;

varcmdline = String.Format( "--nohttpinterface --port {0} --dbpath {1} ", port, storagePath);

MongoProcess= new Process()

{

StartInfo= new ProcessStartInfo(Path.Combine(mongoRoot, @"mongod.exe"), cmdline)

{

UseShellExecute= false,

WorkingDirectory= mongoRoot

}

};

MongoProcess.Start();

Managing Instances

• MongoD.exe provides status

– http (28017), mongod shell

• Azure Load Balancer hides instances!

– Single IP/Port inbound from client apps

– Multiple port round-robin internally

• Azure-hosted apps can access instances!

– Must treat all instances equally

– Assuming specific instance access = bad

Client works against single IP/Port

No way for client app to access individual server

No shared storage

Mongo

MongoD servers

Client-side app

Single IP

Multiple ports

Multiple servers accessible via Azure server

Mainly for management purposes

Mongo

MongoD servers

End user

Single IP

Multiple ports

Replica Sets Challenges

• No shell access

– how to configure?

• Single IP

– How to access?

– How to monitor?

• DB access

– How to access Master in set?

• What about storage?

Replica Set Solution

• ReplicaSet worker role– Runs mongod with –replSet

• Management role (either worker or part of web)– Enumerates all ReplicaSet role instances

– Builds configuration document

– Connects to one node; uploads configwhich initializes replica set

• Client application– Connects to replica set via compatible driver

• Storage– Either local storage or blob storage.

– If blob storage, each replica set node has its own blob

Areas to be Explored

• Replica set per deployment

• Self configuring replica sets and shards

• Configuration data held in WA Storage

– Instance and Replica Set information pushed

• Mongo HTTP port configuration

Coming Soon to a Cloud Near

You• RDP access to instances

• Mongo Sharding will be simpler to

implement than replication due to its

cloud friendly architecture

MongoDB on Windows Azure

Technology

Transcript of MongoDB on Windows Azure