RAVENDB EMBEDDED AT MASSIVE SCALES

ABOUT MERodrigo Rosauro, Software Architect Manager at RDI

• Technology passionate• Developer• 14 years of experience in software development• Working with RavenDB since 2013

ABOUT RDI

• We make restaurant automation software for QSR

• Our software runs on more than 36K restaurants

• Estimated number of individual machines: around half a million

• Processing almost USD $50K per second of customer payments

“ONE IN A MILLION IS NEXT TUESDAY”

- Gordon Letwin, Architect for DOS 4

HOW WE USED TO PERSIST DATA ON POS... and still do (on legacy modules)

LEGACY PERSISTENCY

• Flat files• Custom data format• Event sourcing persistence• Rebuild in-memory state at

every restart

PROBLEMS WITH THAT APPROACH

•No querying – at all•High complexity to store new data•No standard data format•Lack of management tools

WHY WE DECIDED TO USE A DBMS?

WHY A DBMS?

• We started a full re-architecture of the POS platform

• We are definitely not database/persistence specialists

• We wanted to remove complexity to persist data

THE SEEK FOR A DBMS

GOALS / GUIDELINES

• The new architecture is based on plug-ins, so we wanted each individual plug-in to have its own, isolated, database• Zero administration• Easy schema upgrades• Transparent replication• Unit tests

WHY WE CHOSE RAVENDB?

WHY RAVENDB?

• .NET• Embedded operation mode• In-memory mode for unit tests• Multi-tenant• Good transparent replication options

WHAT RAVENDB ALLOWED US TO DO

RAVENDB AT RDI

• Create unit tests without mocking the database• Have advanced statistics about the data persistence• Both system-wide and per plug-in

• Transparent replication• Hot failover, data distribution & real-time backups

• Transparent encryption of data at rest

THE CHALLENGES... and how we faced them

CHALLENGE #1 – MEMORY CONSTRAINTS

SPECIALIZED, OLD HARDWAREOld hardware (sometimes 10 years old) means that we have very little memory and processing power.

MEMORY CONSTRAINTS

• Many iterations with the RavenDB support team to fine-tune its memory usage

• We learned that under these constraints, caching can be evil

• In the end, the solution was to completely disable caching on RavenDB and do some cache at the application side, only for the “hot” data.

CHALLENGE #2 – ESENT

WE ARE STILL USING RAVENDB 2.5

• Our software must support Windows XPe

• .NET 4.0

• 32 bits machines

• … this means that ESENT is our only option for now

FACT: WE DON’T LIKE ESENT(at least not for our usage scenarios)

WHY?

• Any unclean shutdown may cause a completely undetermined result• Power outages / Application crashes / Task kill

• The possible results are not easy to find out during regular testing• Crappy hardware doesn’t help

• We still face sporadic full database losses with ESENT

• Copying data between OS versions is awful

• Recovering from most unclean shutdowns requires using ESENTUTL.EXE

ESENTUTL.EXE

• We also hate don’t like ESENTUTL.EXE

• Many different commands to attempt to recover from different kinds of unclean shutdowns. Some have different syntax on different OS versions

• We had to automate all that (zero maintenance, remember?)• Almost 500 lines of code

“SUCCESS IS STUMBLING FROM FAILURE TO FAILURE WITH NO LOSS OF

ENTHUSIASM”― Winston S. Churchill, Ex-prime minister of UK

THE FUTURE

THE FUTURE

• Working with Hibernating Rhinos to improve Voron to our use case• 32 bits support• Better reliability on unclean shutdowns

• We may jump straight to RavenDB 4.0 & CoreCLR• Long-term target: Around 250K individual POS nodes running

RavenDB embedded

THANK YOU VERY MUCH