Post on 27-Nov-2014
description
Ops 101
Asya KamskyPrincipal Community Advocate
MongoDB
Operational Database Landscape
RDBMS
Agility
MongoDB
{
_id : ObjectId("4c4ba5e5e8aabf3"),
employee_name: "Dunham, Justin",
department : "Marketing",
title : "Product Manager, Web",
report_up: "Neray, Graham",
pay_band: “C",
benefits : [
{ type : "Health",
plan : "PPO Plus" },
{ type : "Dental",
plan : "Standard" }
]
}
Document Data Model
Relational MongoDB
{ first_name: ‘Paul’, surname: ‘Miller’, city: ‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } }}
Document Model Benefits
• Agility and flexibility– Data models can evolve easily– Companies can adapt to changes quickly
• Intuitive, natural data representation– Developers are more productive– Many types of applications are a good fit
• Reduces the need for joins, disk seeks– Programming is more simple– Performance can be delivered at scale
ShellCommand-line shell for interacting directly with database
Shell and Drivers
DriversDrivers for most popular programming languages and frameworks
> db.collection.insert({company:“10gen”, product:“MongoDB”})> > db.collection.findOne(){
“_id” : ObjectId(“5106c1c2fc629bfe52792e86”),
“company” : “10gen”“product” : “MongoDB”
}
Haskell
Scalability
Automatic Sharding
• Increase or decrease capacity as you go
• Automatic balancing
• Three types of sharding:
hash-based
range-based
tag-aware
Query Routing
• Multiple query optimization models
• Many sharding options appropriate for different apps
High Availability
• High Availability – Ensure application availability during many types of failures
• Disaster Recovery – Address the RTO and RPO goals for business continuity
• Maintenance – Perform upgrades and other maintenance operations with no application downtime
Availability Considerations
Replica Sets
• Replica Set – two or more copies
• “Self-healing” shard
• Addresses many concerns:
- High Availability
- Disaster Recovery
- Maintenance
Replica Set Benefits
Business Needs Replica Set Benefits
High Availability Automated failover
Disaster Recovery Hot backups offsite
Maintenance Rolling upgrades
Low Latency Locate data near users
Workload Isolation Read from designated nodes
Data Consistency Tunable Consistency
Performance
Better Data Locality
Performance
In-Memory Caching In-Place Updates
Performance at Scale
• Entertainment Company: 1,400 servers• Craigslist: 5B documents• Carfax: 11B documents• Tier 1 Bank: 30K ops/sec• Major Retailer: 50K ops/sec• Fed Agency: 500K ops/sec• Wordnik: 20B documents, 35,000 ops/sec
MongoDB Performance*
Top 5 Marketing Firm Government Agency Top 5 Investment Bank
Data Key/value 10+ fields, arrays, nested documents
20+ fields, arrays, nested documents
Queries Key-based1 – 100 docs/query80/20 read/write
Compound queriesRange queriesMapReduce20/80 read/write
Compound queriesRange queries50/50 read/write
Servers ~250 ~50 ~40
Ops/sec 1,200,000 500,000 30,000
* These figures are provided as examples. Your application governs your performance.
Key Deployment Considerations
Capacity Planning
• Requirements
• Testing
• Monitoring
Key Performance Considerations
Capacity Planning
• Requirements
• Testing
• Monitoring
Performance Tuning
• Understanding
• Adjusting
• Monitoring
Monitoring
Monitoring
• CLI and internal status commands• mongostat; mongotop; db.serverStatus()
• Plug-ins for munin, Nagios, cacti, etc.
• Integration via SNMP to other tools
• MMS
MongoDB Management ServiceCloud-based suite of services for managing MongoDB deployments
MongoDB Management Service
• Charts, custom dashboards and automated alerting
• Tracks 100+ metrics – performance, resource utilization, availability and response times
• 15,000+ users
Cloud-based suite of services for managing MongoDB deployments
• Backup and restore with
– point-in-time recovery,
– support for sharded clusters
Cloud-based suite of services for managing MongoDB deployments
• MMS On-Prem included with MongoDB Enterprise (backup coming soon)
MongoDB Management Service
A Picture Speaks a Thousand Words
Symptoms
High Use CPU Similar Query Pattern
Monitoring Best Practices
• Monitor Logs– Alert, escalate– Correlate
• Disk– Monitor
• Instrument/Monitor App (including logs!)• Know your application and application (write) characteristics