How companies-use-no sql-and-couchbase-10152013
-
Upload
dipti-borkar -
Category
Technology
-
view
797 -
download
0
Transcript of How companies-use-no sql-and-couchbase-10152013
![Page 1: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/1.jpg)
How Companies use NoSQL and Couchbase
Dipti Borkar Director, Product Management
Anil KumarProduct Management
![Page 2: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/2.jpg)
Two kinds of Database Management System
OLTP / OLTP like
Operational Stores
Data warehouse orAnalytics system
![Page 3: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/3.jpg)
NoSQL + Big Data
Map-reduce against huge datasets to analyze and find insights and answers
Operational database for web and mobile apps with high performance at scale
![Page 4: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/4.jpg)
Common Use CasesSocial Gaming
• Couchbase stores player and game data
• Examples customers include: Zynga
• Tapjoy, Ubisoft, Tencent
Mobile Apps
• Couchbase stores user info and app content
• Examples customers include: Kobo, Playtika
Ad Targeting
• Couchbase stores user information for fast access
• Examples customers include: AOL, Mediamind, Convertro
Session store
• Couchbase Server as a key-value store
• Examples customers include: Concur, Sabre
User Profile Store
• Couchbase Server as a key-value store
• Examples customers include: Tunewiki
High availability cache
• Couchbase Server used as a cache tier replacement
• Examples customers include: Orbitz
Content & Metadata Store
• Couchbase document store with Elastic Search
• Examples customers include: McGraw Hill
3rd party data aggregation
• Couchbase stores social media and data feeds
• Examples customers include: LivePerson
![Page 5: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/5.jpg)
Use Case: High-Availability Caching
High availability caching
RDBMS
Application Layer
User Requests
Cache Misses and Write Requests
Read-Write Requests
Couchbase Distributed Cache
![Page 6: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/6.jpg)
• Application objects
• Popular search query results
• Session information
• Heavily accessed web landing pages
Use Case: High-Availability Caching
Data Cached in Couchbase?
• Speed up RDBMS
• Consistently low response times for document / key lookups
• High-availability 24x7x365
• Replacement for entire caching tier
Application characteristic
![Page 7: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/7.jpg)
Use Case: High-Availability Caching
• Low latency in sub-milliseconds with consistently high read / write throughput using built-in cache
• Always-on operations even for database upgrades and maintenance with zero down time
Why NoSQL?
![Page 8: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/8.jpg)
Use Case: Session Store
Session Store
![Page 9: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/9.jpg)
Use Case: Session Store
Data stored in Couchbase?
• Extremely fast access to session data using unique session ID
• Easy scalability to handle fast growing number of users and user-generated data
• Always-on functionality for global user base
Application characteristic
• Session values or Cookies (stored as key-value pairs)
• Examples include: items in a shopping cart, flights selected, search results, etc.
![Page 10: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/10.jpg)
Use Case: Session Store
• Low latency in sub-milliseconds with consistently high read / write throughput for session data via the built-in object-level cache
• Linear throughput scalability to grow the database as user and data volume grow
• Always-on operations even particularly high availability using Couchbase replication and failover
• Intra cluster and cross cluster (XDCR) replication for globally distributed active-active platform
Why NoSQL?
![Page 11: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/11.jpg)
Use Case: Globally Distributed User Profile Store
User ID / Profile Store
![Page 12: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/12.jpg)
Use Case: Globally Distributed User Profile Store
Data stored in Couchbase?
• Extremely fast access to individual profiles
• Always online system as multiple applications access user profiles
• Flexibility to add and update user attributes
• Easy scalability to handle fast growing number of users
Application characteristic
• User profile with unique ID
• User setting / preferences
• User’s network
• User application state
![Page 13: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/13.jpg)
Use Case: Globally Distributed User Profile Store
• Low latency and high throughput for very quick lookups for millions of concurrent users using built-in cache
• Intra cluster and cross cluster (XDCR) replication for high availability and disaster recovery
• Active-active geo-distributed system to handle globally distributed user base
• Online admin operations eliminate system downtime
Why NoSQL?
![Page 14: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/14.jpg)
Use Case: Data Aggregation
Data Aggregation
![Page 15: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/15.jpg)
Use Case: Data Aggregation
Data stored in Couchbase?
• Flexibility to store any kind of content
• Flexibility to handle schema changes
• Full-text Search across data set
• High speed data ingestion
• Scales horizontally as more content gets added to the system
Application characteristic
• Social media feeds: Twitter, Facebook, LinkedIn
• Blogs, news, press articles
• Data service feeds: Hoovers, Reuters
• Data form other systems
![Page 16: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/16.jpg)
Use Case: Data Aggregation
• JSON provides schema flexibility to store all types of content and metadata
• Fast access to individual documents via built-in cache, high write throughput
• Indexing and querying provides real-time analytics capabilities across dataset
• Integration with ElasticSearch for full-text search
• Ease of scalability ensures that the data cluster can be grown seamlessly as the amount of user and ad data grows
Why NoSQL?
![Page 17: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/17.jpg)
Use Case: Content and Metadata Store
Data Aggregation
![Page 18: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/18.jpg)
Use Case: Content and Metadata Store
Data stored in Couchbase?
• Flexibility to store any kind of content
• Fast access to content metadata (most accessed objects) and content
• Full-text Search across data set
• Scales horizontally as more content gets added to the system
Application characteristic
• Content metadata
• Content: Articles, text
• Landing pages for website
• Digital content: eBooks, magazine, research material
![Page 19: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/19.jpg)
Use Case: Content and Metadata Store
• Fast access to metadata and content via object-managed cache
• JSON provides schema flexibility to store all types of content and metadata
• Indexing and querying provides real-time analytics capabilities across dataset
• Integration with ElasticSearch for full-text search
• Ease of scalability ensures that the data cluster can be grown seamlessly as the amount of user and ad data grows
Why NoSQL?
![Page 20: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/20.jpg)
McGraw Hill Education Labs Learning portal
![Page 21: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/21.jpg)
Use Case: Content and metadata store
Building a self-
adapting, interactive learning
portal with Couchbase
![Page 22: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/22.jpg)
As learning move online in great numbers
Growing need to build interactive learning environments that
Scale!
Scale to millions of learners
Serve MHE as well as third-party content
Including open content
Support learning apps
010100100111010101010101001010101010
Self-adapt via usage data
The Problem
![Page 23: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/23.jpg)
• Allow for elastic scaling under spike periods
• Ability to catalog & deliver content from many
sources
• Consistent low-latency for metadata and stats access
• Require full-text search support for content
discovery
• Offer tunable content ranking & recommendation
functions
Backend is an Interactive Content Delivery Cloud that must:
XML Databases
SQL/MR Engines
In-memory Data Grids
Enterprise Search Servers
Experimented with a combination of:
The Challenge
![Page 24: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/24.jpg)
![Page 25: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/25.jpg)
Architecture
![Page 26: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/26.jpg)
LivePerson – Real time visitor engagement
![Page 27: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/27.jpg)
Use Case: 3rd party data aggregation with analytics
Real time Analytics for
LivePerson's customers
LiveEngage DASHBOARD
![Page 28: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/28.jpg)
LivePerson: Leading customer engagement platform
![Page 29: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/29.jpg)
Requirements Requirements Requirements
• High throughput, really fast
• Linear scale
• Searchable (Views and M/R)
• Supports both K/V & Document store
• Cross data center replication
• “Always on”, Resilience solution
The Problem
13 TBper month ~1 PB
In total 1.8 BVisits per month
VOLUME
![Page 30: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/30.jpg)
Couchbase Java SDK
Application serverTomcat
M/R views
cluster
M/R views
cluster
XDCR
REST API
Couchbase Java SDK
Storm Topology
Couchbase Java SDK
Storm Topology
Architecture
![Page 31: How companies-use-no sql-and-couchbase-10152013](https://reader033.fdocuments.net/reader033/viewer/2022042816/559b13e81a28ab04248b457b/html5/thumbnails/31.jpg)
Questions?