MongoDB 3.0 migration - MongoDB Days Munich

Better, Faster, Stronger! Migration to 3.0

© 2015 A

madeus IT G

roup SA

Upgrade MongoDB in the enterprise environment

Attila Tozser, Amadeus Data Processing GmbH, MongoDB Days

Germany, 20/10/2015

Amadeus IT Group1

© 2015 A

madeus IT G

roup SA

Amadeus is a technology company dedicated to the

global travel industry.

We are present in 195 countries with a worldwide team of more than 13,200 people.

Our solutions help improve the

business performanceof travel agencies, corporations, airlines,

airports, hotels, railways and more.

_ Introduction1Amadeus in a few words

© 2015 A

madeus IT G

roup SA

Our solutions facilitate journeys and enrich the travel

experience for hundreds of millions of people every year.

to shape the future of travel

_ Introduction1

We are working together with

our customers and partners

We have a clear purpose

© 2015 A

madeus IT G

roup SA

A history of shaping the future of travel _ Amadeus history2

1987Amadeus is founded

1995World leader in number of travel agency locations

19981 MILLIONbookings made on a single day for the first time

2010Amadeus diversifiesinto IT for hotel, railand airport

1992First booking made through Amadeus

1996e-Commerce division launched

2000Partnership with BA and Qantas to launch Amadeus Altéa – our core Airline IT offering

2014Contracts with Ryanair and Southwest Airlinesand a strategic technology relationship with IHG

© 2015 A

madeus IT G

roup SA

_ Amadeus today3Robust global operations

© 2015 A

madeus IT G

roup SA

1.6+billiondata requests processed per day

526milliontravel agencybookings processed in 2014

695millionPassengers Boarded (PBs) in 2014

95%of the world’s scheduled network airline seats

Close to our customers

© 2015 A

madeus IT G

roup SA

A global team of 13,200+ professionals

_ 118 nationalities

_ Cultural diversity

_ Clear focus on recruiting and developing talent

_ Values based culture

MongoDB at Amadeus2

© 2015 A

madeus IT G

roup SA

Business needsGathered from customers

user-friendly way to query any datain our main operational databases

_ Unpredictable ad-hoc search_ Many advanced reporting requirements

A way to store orders of magnitude more different data from various sources, but still maintain low latency

Sustain Organic growth of data and throughput (time-to-time 2X or more a year) without degradation

Runs on standard x86 architecture

C++, Python & Java drivers

Enterprise-grade security_ SSL encryption

_ Kerberos authentication

_ Data-at-rest encryption

MongoDB is perfect match to our needs

IntegrationIn the Amadeus standards

New features in MongoDB 3.02

© 2015 A

madeus IT G

roup SA

© 2015 A

madeus IT G

roup SA

© 2015 A

madeus IT G

roup SA

WiredTiger our observations• Compression ratio 3x-10x• IO characteristic tunable

• Better latency• Enhanced security

© 2015 A

madeus IT G

roup SA

In-memory our observations• On OOM instant crash

• Sustained 150K/4KB IOPSOn a single box

• 1-6ms latency

Upgrade MongoDB in the enterprise environment

2

© 2015 A

madeus IT G

roup SA

© 2015 A

madeus IT G

roup SA

AmadeusClients

© 2015 A

madeus IT G

roup SA

SS

S

S

S

S

Application servers MongoDB

© 2015 A

madeus IT G

roup SA

S

S

S

S

S

S

Database sever:Operating SystemMongoDB binariesMisc. processes:

Ops manager agentsMonitoring codeAutomation code

© 2015 A

madeus IT G

roup SA

S

S

S

S

S

S

This appserver consist of• Operating System• Application code• MongoDB driver• Mongos binary

© 2015 A

madeus IT G

roup SA

MongoDB Cluster

MMS/Ops Manager

© 2015 A

madeus IT G

roup SA

MongoDB Cluster

MMS/Ops Manager

_Always Sharded and replicated

_Over 10 test levels

_Monitoring

_Backup_(automation)

Mongodb 3.0 needs Ops Manager > 1.6.0

Upgrading MMS3

© 2015 A

madeus IT G

roup SA

MMS application database

BackupDatabase

BackupDatabase

BackupDatabase

MMS App Server MMS App Server MMS App Server

Backup Daemon

Backup Daemon

Backup Daemon

Backup agents

Monitoringagents

Automationagents

Upgrading MMS3

© 2015 A

madeus IT G

roup SA

MMS application database

BackupDatabase

BackupDatabase

BackupDatabase

MMS App Server MMS App Server MMS App Server

Backup Daemon

Backup Daemon

Backup Daemon

Backup agents

Monitoringagents

Automationagents

Upgrade separately as any other MongoDB cluster3.0 only from MMS 1.6

The MMS server process contains the new versions

Should be on the same version: Upgrade together

Upgrading MMS3

© 2015 A

madeus IT G

roup SA

MMS App Server

REST API

Corporate Monitoring

API callsMetrics

Corporate Automation

API callsConfig options

Upgrading MMS3

© 2015 A

madeus IT G

roup SA

MMS App Server

REST API

Corporate Monitoring

API callsMetrics

Corporate Automation

API callsConfig options

Test the changes of the API to avoid bad surprises

© 2015 A

madeus IT G

roup SA

SS

S

S

S

S

http://docs.mongodb.org/manual/release-notes/3.0-upgrade/

_Upgrade MongoDB

_Be prepared for fallback funnies_Backup, MONGODB-CR auth

_Lucky case: Give a try to Ops Manager automation

_Less lucky case: Do it by hand_Start from a 2.6 installation

Upgrading MMS3

© 2015 A

madeus IT G

roup SA

S

Upgrade the Mongos processes

Config 1 Config 2 Config 3

S, 3.xUpgrade cluster metadata

3

© 2015 A

madeus IT G

roup SA

PrimaryMMAP v1

2.6

SecondaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

Install binaries

Install binaries

Install binaries

SecondaryMMAPv1

2.6

3

© 2015 A

madeus IT G

roup SA

PrimaryMMAP v1

2.6

SecondaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

SecondaryWT 3.0

3

© 2015 A

madeus IT G

roup SA

PrimaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

SecondaryWT 3.0

SecondaryMMAP v1

2.6

3

© 2015 A

madeus IT G

roup SA

PrimaryMMAP v1

2.6

SecondaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

SecondaryWT 3.0PrimaryWT 3.0

SecondaryWT 3.0

SecondaryWT 3.0

3

© 2015 A

madeus IT G

roup SA

PrimaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

SecondaryWT 3.0

SecondaryMMAPv1

2.6

PrimaryWT 3.0

3

© 2015 A

madeus IT G

roup SA

PrimaryMMAP v1

2.6

SecondaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

SecondaryWT 3.0PrimaryWT 3.0

SecondaryWT 3.0

SecondaryMMAPv1

2.6

SecondaryWT 3.0

3

© 2015 A

madeus IT G

roup SA

SecondaryMMAP v1

2.6

SecondaryMMAP v1

2.6

Upgrade the Shards

SecondaryWT 3.0

PrimaryWT 3.0

SecondaryWT 3.0

3

© 2015 A

madeus IT G

roup SA

MY MongoDB wishlist

_ Cross cluster replication

_ Filtered / delayed / batched

_Multimaster (AP operation)

_Multithreaded queries

_ Transactions

_On multi update

MongoDB 3.0 migration - MongoDB Days Munich

Technology

Transcript of MongoDB 3.0 migration - MongoDB Days Munich