October 2013 NYC Meetup

25
Copyright © 2013 NuoDB Keeping Pace with a Distributed Database Seth Proctor CTO

description

These slides were used as part of an interactive talk/demo in New York. The video fills in the gaps: https://www.youtube.com/watch?v=cO197XEkcVo. Abstract: The move from scale-up to scale-out architectures has challenged developers to re-think database design. It's been a great forcing function, resulting in a number of systems that combine traditional database functionality with modern scaling capability. What is sometimes lost in that discussion, however, is how difficult it is to manage a distributed system. In this talk I'll introduce you to the newly released NuoDB 2.0, a NewSQL database that can be distributed across data centers, and demonstrate why the user experience should be considered part of the criteria for judging scale.

Transcript of October 2013 NYC Meetup

Page 1: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Keeping Pace with a Distributed Database

!Seth Proctor

CTO

Page 2: October 2013 NYC Meetup

Copyright © 2013 NuoDB !2

Page 3: October 2013 NYC Meetup

Copyright © 2013 NuoDB

NuoDB is a Distributed Database

!3

Page 4: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Simple; Composable

!4

Page 5: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Peers play one of two roles

!5

Host 1

Transaction Engine

Host 2

Storage Manager

SQL Clients

Durable Store (KV)

Page 6: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Scaled independently

!6

Host 1

Transaction Engine

Host 2

Storage Manager

Host 3

Transaction Engine

Host 4

Transaction Engine

Host 5

Storage Manager

Page 7: October 2013 NYC Meetup

Copyright © 2013 NuoDB

On-demand

!7

Page 8: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Adjust transactional throughput‣ Add, move or remove TEs to ‣ Adjust for demand

‣ Migrate or upgrade active databases

‣ Re-adjust resources

!8

Page 9: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Add durability points

!9

TE (Running)

TE (Running)

SM (Running)

SM (Syncing)

Archive Archive

TE (Running)

TE (Running)

SM (Running)

SM (Running)

Archive Archive

Active Database: add SM

Active Database: redundant archives

TE (Running)

TE (Running)

SM (Running)

Archive

Active Database: one SM running

Page 10: October 2013 NYC Meetup

Copyright © 2013 NuoDB

React to failure

!10

TE (Running)

TE (Joining)

SM (Running)

Archive

Active Database: add TE

TE (Running) TE (Fails)

SM (Running)

Archive

Active Database: one TE fails

TE (Running)

TE (Running)

SM (Running)

Archive

Active Database: two TEs

Page 11: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Multi-tenant; Managed

!11

Page 12: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Installation == Provisioning

!12

‣ Every host has a management agent ‣ Makes the host available

‣ Keeps track of local activity

‣ Some are “Connection Brokers” ‣ Load-balancer for SQL clients

‣ Single management point

Page 13: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Independent databases

!13

Host 1 Host 2 Host 3 Host 6

EBS

Host 5

TE SM SM

TE

TE TE

SM

SM

S3

EBS

EBS

Host 4

TE

SM

HDFS

TE

Logically addressable Database

Complete, independent archive

Provisioned host

Page 14: October 2013 NYC Meetup

Copyright © 2013 NuoDB

http://dev.nuodb.com/techblog/tags/moonshot

!14

Page 15: October 2013 NYC Meetup

Copyright © 2013 NuoDB

And now, a few quick demos..

!15

Page 16: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Templates

!16

Page 17: October 2013 NYC Meetup

Copyright © 2013 NuoDB

One host

!17

{ "name" : "Single Host", "summary" : "...", "requirements" : [{ "processes" : [{ "type" : "SM", "host" : "${HOST}" },{ "type" : "TE", "host" : "${HOST}" }] }] }

Page 18: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Many hosts, one region

!18

{ "name" : "Multi Host", "summary" : "...", "requirements" : [{ "region" : "${REGION}", "processes" : [{ "type" : "SM", "min" : 1, "max" : 2 },{ "type" : "TE", "min" : 1, }] }] }

Page 19: October 2013 NYC Meetup

Copyright © 2013 NuoDB

GET /api/1/templates

!19

Page 20: October 2013 NYC Meetup

Copyright © 2013 NuoDB

GET /api/1/databases

!20

Page 21: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Statistics

!21

Page 22: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Java Stored Procedures

!22

Page 23: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Flexible Schema

!23

Page 24: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Want to learn more?‣ http://dev.nuodb.com ‣ Techblog, documentation & forums

‣ http://githib.com/nuodb ‣ Community projects & APIs

[email protected]

!24

Page 25: October 2013 NYC Meetup

Copyright © 2013 NuoDB

Thank You.