NoSQL@Mobiliar How we started · 2017. 9. 11. · Oracle, SQL Server, MariaDB, PostgreSQL, MongoDB,...

NoSQL@Mobiliar – How we started

Christian Hählen

Content

1. Background

1.1 Short introduction

1.2 Why did we start with NoSQL?

2. NoSQL@Mobiliar

3.1 Self-developped telemetry framework on classic hardware

3.2 Toolbox / Templates for usage in Docker / Kubernetes

3. Looking back

11.09.2017NoSQL@Mobiliar 2

Background


About the company…

Swiss Mobiliar Insurance

• Own applications as well as external applications

• Operate Windows and Linux,

directly and on Docker/Kubernetes

• Observe and anticipate development

on new technologies

Mobiliar and databases

• Oracle, SQL Server, DB2

• Linux, Windows, z/OS


… and the speaker

Christian Hählen

• 9 years working experience

• on SQL Server as data architect & database administrator

• as a database interface developer (.NET)

• 4 years working experience as System Administrator /

Head of Infrastructure and Operations

• Working at Mobiliar as database specialist / DBA for one year

• Member of PASS and co-organizer Swiss Chapter Events in Bern

• Interested in the future of

• SQL Server

• NoSQL

• Information system technologies


History – Why did we start with NoSQL?

• NoSQL «everywhere»

• NoSQL already in the house

• With upcoming technologies like docker

RDBMS have some struggles in adapting

How to scale?

Multi-replica write

For some applications RDBMS are not the right tool

Images (BLOBs), Objects, Graphs, JSON

Transient workflow data


• Being a DBA…

different opinions about DBA in NoSQL and agile world

developers ask if they may use NoSQL system XYZ

be ready to answer the first developer’s question

either with

"No, we don't want this in our company because …”

or with

"yes, we made a Pilot / PoC, leading to this toolbox

and templates"

History – Why did we start with NoSQL?


NoSQL@Mobiliar


NoSQL@Mobiliar

Performance Monitoring

• Self-developped telemetry framework (ElasticSearch, spoud)

• Operational data from applications (Prometheus)


NoSQL@Mobiliar


• Self-developed telemetry framework

Linked in application

Runs on every (~800) employee laptop to monitor

performance of busines application

v1: Oracle

v2: ElasticSearch - performance

v3: spoud - flexibility


NoSQL@Mobiliar


• Self-developed telemetry framework

Transmits data about every step, network, I/O …

90 to 100 Millions of documents per day

Last 500 days stored

For performance: index per day

Analysation / visualisation done with Kibana


NoSQL@Mobiliar


• Operational data from applications

Applications are developed exposing data for

Prometheus

Docker containers

currently only showing overall data

in future should expose internal data (processes etc.)


NoSQL@Mobiliar

Usage in Docker and Kubernetes

• Change application to use MongoDB!

Why MongoDB? – We heard about it

Does data management already have a tool of choice?

• Let’s do an evaluation of NoSQL systems

Let’s define a set of systems to be supported

Prepare toolbox and guidelines

• But we need it now…


«If you don’t have

the time to do something

right, where are you

going to find the time

to fix it?»

Stephen King

NoSQL@Mobiliar

Toolbox / Templates for usage in Docker

• Evaluation:

RDBMS vs. NoSQL, classic infrastructure

vs. VMWare vs. Container

Oracle, SQL Server, MariaDB, PostgreSQL, MongoDB,

Couchbase, Cockroach, ArangoDB

Focus on scalability, community, container size / startup

time, UI, schema / typing

Interviews with stakeholders

• Result:

Vision of the future

recommendation of RDBMS and NoSQL systems


NoSQL@Mobiliar


• Pilot / PoC:

Rocket.Chat

Not off the shelf

scalable NoSQL system

automated backup

performance monitoring

deployment «as one» on our platform

DevOps Project:

Developer

Container Operator

Database specialist


• Pilot targets:

Ready-to-use template for Docker /

Kubernetes Application

AMW & Kubernetes & Docker Deployment

Using Helm chart

Persistent storage claim

Automated backup external storage

Kubernetes Job

Covered by company backup solution

Integration in our Prometheus

health / performance of DB

GUI for DevOps to have «a feeling»

Collectable by Data Management team

NoSQL@Mobiliar



• Pilot state:

Ready-to-use template for Docker /

Kubernetes Application

AMW & Kubernetes & Docker Deployment

Using Helm chart

Persistent storage claim

Automated backup external storage

Kubernetes Job

Covered by company backup solution

Integration in our Prometheus

health / performance of DB

GUI for DevOps to have «a feeling»

Collectable by Data Management team

NoSQL@Mobiliar



Looking back


Conclusion


• NoSQL

Might look scary to a DBA

Hot topic

Not yet on it’s peak

Conclusion

• NoSQL


Hot topic


You can’t hide / ignore

If you dive into it, you feel how…


Conclusion

• NoSQL


Hot topic



If you dive into it, you feel how

interesting

powerful

tempting


Conclusion

• NoSQL


Hot topic



If you dive into it, you feel how

interesting

powerful

tempting

• Implementation @ Mobiliar

Not everybody was enthusiastic at first

Toolset and guidelines are important

Pilot ist still ongoing

More to do than we first thought

Very interesting work


Questions

Thanks for listening …

Want to learn more about our future experiences?

Follow our new blog at www.mobiliar.ch/db-blog


http://www.mobiliar.ch/db-blog

NoSQL@Mobiliar How we started · 2017. 9. 11. · Oracle, SQL Server, MariaDB, PostgreSQL, MongoDB,...

Documents

Transcript of NoSQL@Mobiliar How we started · 2017. 9. 11. · Oracle, SQL Server, MariaDB, PostgreSQL, MongoDB,...