Couchbase Manual 1.8

Couchbase Server Manual 1.8


Abstract

This manual documents the Couchbase Server 1.8 series, including installation, administration, and associated tools.

Couchbase Server 1.8 is a re-branded release of Membase Server 1.7 with some additional fixes and changes.

Last document update: 01 Feb 2012 16:36; Document built: 01 Feb 2012 16:41.

Documentation Availability and Formats. This documentation is available online: HTML Online . For other documentation fromCouchbase, see Couchbase Documentation Library

Contact: [email protected] or couchbase.com

Copyright © 2010, 2011 Couchbase, Inc. Contact [email protected].

For documentation license information, see Section D.1, “Documentation License”. For all license information, see Appendix D, Licenses.

http://www.couchbase.com/docs/couchbase-manual-1.8

http://www.couchbase.com/docs

mailto:[email protected]

http://www.couchbase.com

mailto:[email protected]

iii

Table of ContentsPreface .................................................................................................................................................... ix1. Introduction to Couchbase Server .............................................................................................................. 1

1.1. Couchbase Server Basics ............................................................................................................... 11.2. Couchbase Server and NoSQL ........................................................................................................ 21.3. Architecture and Concepts ............................................................................................................. 2

1.3.1. Nodes and Clusters ............................................................................................................ 31.3.2. Cluster Manager ................................................................................................................ 31.3.3. Memory Quotas ................................................................................................................. 31.3.4. Buckets ............................................................................................................................ 41.3.5. vBuckets .......................................................................................................................... 61.3.6. Disk Storage ..................................................................................................................... 61.3.7. Failover ............................................................................................................................ 71.3.8. Client Interface .................................................................................................................. 71.3.9. Administration Tools .......................................................................................................... 81.3.10. Statistics and Monitoring ................................................................................................... 8

2. Getting Started ..................................................................................................................................... 102.1. Preparation ................................................................................................................................ 10

2.1.1. Supported Platforms ......................................................................................................... 102.1.2. Hardware Requirements .................................................................................................... 102.1.3. Storage Requirements ....................................................................................................... 102.1.4. Web Browser (for administration) ....................................................................................... 112.1.5. Network Ports ................................................................................................................. 11

2.2. Installing Couchbase Server .......................................................................................................... 112.2.1. Red Hat Linux Installation ................................................................................................. 112.2.2. Ubuntu Linux Installation .................................................................................................. 122.2.3. Microsoft Windows Installation .......................................................................................... 132.2.4. Mac OS X Installation ...................................................................................................... 17

2.3. Upgrading to Couchbase Server 1.8 ............................................................................................... 182.3.1. Manually Controlled Upgrade Options ................................................................................. 192.3.2. Using cbupgrade Manually ............................................................................................... 20

2.4. Setting up Couchbase Server ........................................................................................................ 212.5. Testing Couchbase Server ............................................................................................................ 292.6. Next Steps ................................................................................................................................. 30

3. Administration Basics ............................................................................................................................ 313.1. Working with Couchbase data files ................................................................................................ 313.2. Startup and Shutdown of Couchbase Server ..................................................................................... 32

3.2.1. Startup and Shutdown on Linux .......................................................................................... 323.2.2. Startup and Shutdown on Windows ..................................................................................... 323.2.3. Startup and Shutdown on Mac OS X ................................................................................... 32

4. Best Practices ....................................................................................................................................... 354.1. Cluster Design Considerations ...................................................................................................... 354.2. Sizing Guidelines ........................................................................................................................ 35

4.2.1. RAM Sizing .................................................................................................................... 364.2.2. Disk Throughput and Sizing ............................................................................................... 384.2.3. Network Bandwidth .......................................................................................................... 384.2.4. Data Safety ..................................................................................................................... 39

4.3. Deployment Considerations .......................................................................................................... 404.4. Ongoing Monitoring and Maintenance ............................................................................................ 414.5. Couchbase Behind a 2nd Firewall .................................................................................................. 424.6. Using Couchbase in the Cloud ...................................................................................................... 42

4.6.1. Local Storage .................................................................................................................. 43


iv

4.6.2. IP addresses .................................................................................................................... 434.6.3. Security groups/firewall settings .......................................................................................... 444.6.4. Swap Space ..................................................................................................................... 45

4.7. Deployment Strategies ................................................................................................................. 454.7.1. Using Couchbase Embedded Proxy ..................................................................................... 454.7.2. Standalone Proxy ............................................................................................................. 454.7.3. Using a vBucket aware Client ............................................................................................ 46

5. Administration Tasks ............................................................................................................................. 475.1. Failover with Couchbase .............................................................................................................. 47

5.1.1. Automatic Failover ........................................................................................................... 475.1.2. Automated failover considerations ....................................................................................... 485.1.3. Monitored failover ............................................................................................................ 48

5.2. Backup and Restore with Couchbase .............................................................................................. 495.2.1. Backup ........................................................................................................................... 495.2.2. Restoring a Backup .......................................................................................................... 50

5.3. Expanding and Shrinking your Cluster (Rebalancing) ........................................................................ 525.3.1. Growing your cluster ........................................................................................................ 525.3.2. Shrinking your cluster ....................................................................................................... 525.3.3. Rebalancing .................................................................................................................... 53

6. Administration Web Console ................................................................................................................... 546.1. Cluster Overview ........................................................................................................................ 54

6.1.1. Cluster ........................................................................................................................... 556.1.2. Buckets .......................................................................................................................... 566.1.3. Servers ........................................................................................................................... 56

6.2. Web Console Statistics ................................................................................................................ 576.3. Data Buckets ............................................................................................................................. 57

6.3.1. Creating and Editing Data Buckets ...................................................................................... 586.3.2. Bucket Information ........................................................................................................... 616.3.3. Individual Bucket Monitoring ............................................................................................. 61

6.4. Server Nodes ............................................................................................................................. 686.5. Couchbase Server States .............................................................................................................. 716.6. Settings ..................................................................................................................................... 72

6.6.1. Update Notification Settings ............................................................................................... 726.6.2. Auto-Failover Settings ...................................................................................................... 736.6.3. Alerts ............................................................................................................................. 73

6.7. Log .......................................................................................................................................... 756.7.1. Diagnostic Report ............................................................................................................. 76

6.8. Update Notifications .................................................................................................................... 776.8.1. Notifications .................................................................................................................... 776.8.2. Viewing Available Updates ................................................................................................ 77

6.9. Warnings and Alerts ................................................................................................................... 777. Couchbase Command-line Interface .......................................................................................................... 79

7.1. couchbase-cli Tool ..................................................................................................................... 797.2. cbstats Tool .............................................................................................................................. 827.3. cbflushctl Tool .......................................................................................................................... 827.4. tap.py Tool ............................................................................................................................... 837.5. cbvbucketctl Tool ...................................................................................................................... 837.6. cbvbuckettool Tool .................................................................................................................... 837.7. cbdbmaint Tool ......................................................................................................................... 847.8. cbcollect_info Tool ..................................................................................................................... 84

8. Couchbase Management REST API ......................................................................................................... 858.1. Key Concepts ............................................................................................................................ 85

8.1.1. Resources ....................................................................................................................... 858.1.2. HTTP Request Headers ..................................................................................................... 86


v

8.1.3. HTTP Status Codes .......................................................................................................... 868.2. Operations ................................................................................................................................. 87

9. Developing with Couchbase .................................................................................................................. 1059.1. Use Cases ................................................................................................................................ 105

9.1.1. Session store .................................................................................................................. 1059.1.2. Social gaming ................................................................................................................ 1059.1.3. Ad, offer and content targeting ......................................................................................... 1059.1.4. Real-time logging and alerting .......................................................................................... 105

9.2. Best practices ........................................................................................................................... 1069.2.1. What should I store in an object ........................................................................................ 1069.2.2. How should I store an object? .......................................................................................... 1069.2.3. Objects that refer to other objects ...................................................................................... 1069.2.4. Nested Items .................................................................................................................. 1069.2.5. Simple Lists .................................................................................................................. 1079.2.6. Multi-GET .................................................................................................................... 1089.2.7. Locking ........................................................................................................................ 1089.2.8. Data partitioning with buckets .......................................................................................... 109

9.3. Couchbase for Memcached Users ................................................................................................ 10910. Developing Couchbase Clients ............................................................................................................. 110

10.1. REST/JSON ........................................................................................................................... 11010.1.1. Parsing the JSON .......................................................................................................... 11110.1.2. Encoding the vBucketId into each request ......................................................................... 11210.1.3. Client Libraries and Rebalancing ..................................................................................... 11210.1.4. Fast Forward Map ......................................................................................................... 11310.1.5. Redundancy & Availability ............................................................................................. 113

10.2. SASL Authentication Example .................................................................................................. 11310.2.1. List Mechanisms ........................................................................................................... 11410.2.2. Authentication request ................................................................................................... 115

11. Couchbase Architecture ...................................................................................................................... 11711.1. Cluster Design ........................................................................................................................ 11711.2. Persistence Design ................................................................................................................... 11711.3. Component Overview ............................................................................................................... 11711.4. Disk Storage (Growing Data Sets Beyond Memory) ...................................................................... 118

11.4.1. Design ........................................................................................................................ 11811.4.2. Consequences of Memory faster than Disk ........................................................................ 119

11.5. Couchbase APIs ...................................................................................................................... 12111.5.1. memcached protocol ..................................................................................................... 12111.5.2. Additional Protocol Operations ........................................................................................ 124

11.6. Buckets ................................................................................................................................. 12511.7. vBuckets ................................................................................................................................ 127

11.7.1. Couchbase Document ID-vBucket-Server Mapping Illustrated ............................................... 12711.7.2. vBuckets in a world of memcached clients ....................................................................... 128

12. Monitoring Couchbase ........................................................................................................................ 13012.1. Port numbers and accessing different buckets ............................................................................... 13012.2. Monitoring startup (warmup) ..................................................................................................... 13012.3. Disk Write Queue ................................................................................................................... 131

12.3.1. Handling Reads ............................................................................................................ 13112.3.2. Monitoring the Disk Write Queue .................................................................................... 132

12.4. Couchbase Server Statistics ....................................................................................................... 13212.4.1. REST Interface Statistics ................................................................................................ 13212.4.2. Couchbase Server Node Statistics .................................................................................... 132

12.5. Couchbase Server Moxi Statistics ............................................................................................... 13313. Troubleshooting ................................................................................................................................. 135

13.1. General Tips ........................................................................................................................... 135


vi

13.2. Responding to Specific Errors ................................................................................................... 13513.3. Log File Entries ...................................................................................................................... 13613.4. Linux Logs ............................................................................................................................ 13613.5. Windows Logs ........................................................................................................................ 13613.6. Common Errors ...................................................................................................................... 136

A. FAQs ................................................................................................................................................ 138B. Uninstalling Couchbase Server .............................................................................................................. 140

B.1. Uninstalling on a RedHat Linux System ....................................................................................... 140B.2. Uninstalling on an Debian/Ubuntu Linux System ........................................................................... 140B.3. Uninstalling on a Windows System ............................................................................................. 140B.4. Uninstalling on a Mac OS X System ........................................................................................... 140

C. Release Notes ..................................................................................................................................... 141C.1. Release Notes for Couchbase Server 1.8.0 GA (23 January 2012) ...................................................... 141

D. Licenses ............................................................................................................................................ 144D.1. Documentation License ............................................................................................................. 144D.2. Couchbase, Inc. Community Edition License Agreement ................................................................. 144D.3. Couchbase, Inc. Enterprise License Agreement: Free Edition ............................................................ 145

vii

List of Figures1.1. Multi-tenancy ...................................................................................................................................... 52.1. Windows Installation Welcome Screen ................................................................................................... 132.2. Windows Installation Location Screen .................................................................................................... 142.3. Windows Installation Ready Screen ....................................................................................................... 152.4. Windows Installation Progress Screen .................................................................................................... 162.5. Windows Installation Completion Screen ................................................................................................ 172.6. Couchbase Server Setup ....................................................................................................................... 222.7. Couchbase Server Setup Step 1 (New Cluster) ......................................................................................... 232.8. Couchbase Server Setup Step 1 (Existing Cluster) .................................................................................... 242.9. Couchbase Server Setup Step 2 ............................................................................................................. 252.10. Couchbase Server Setup Step 3 ........................................................................................................... 262.11. Couchbase Server Setup Step 4 ........................................................................................................... 282.12. Couchbase Server Setup Completed ..................................................................................................... 293.1. Couchbase Server on Mac OS X Menubar Item ....................................................................................... 334.1. Deployment Strategy: Using the Embedded Proxy .................................................................................... 454.2. Deployment Strategy: Standalone Proxy ................................................................................................. 464.3. Deployment Strategy: Using a vBucket Aware Client ................................................................................ 466.1. Couchbase Web Console - Cluster Overview ........................................................................................... 556.2. Couchbase Web Console - Data Buckets Overview ................................................................................... 586.3. Couchbase Web Console - Create Bucket ............................................................................................... 596.4. Couchbase Web Console - Bucket Information ........................................................................................ 616.5. Couchbase Web Console - Summary Statistics ......................................................................................... 626.6. Couchbase Web Console - vBucket Resources statistics ............................................................................. 636.7. Couchbase Web Console - Disk Queue Statistics ...................................................................................... 656.8. Couchbase Web Console - TAP Queue Statistics ...................................................................................... 666.9. Couchbase Web Console - Memcached Statistics ..................................................................................... 676.10. Couchbase Web Console - Server Nodes ............................................................................................... 696.11. Couchbase Web Console- Data Bucket/Server view ................................................................................ 706.12. Couchbase Web Console - Server specific view ..................................................................................... 716.13. Down Status .................................................................................................................................... 726.14. Pend Status ...................................................................................................................................... 726.15. Web Console Settings — Update Notifications ....................................................................................... 736.16. Web Console Settings — Auto-Failover ................................................................................................ 736.17. Web Console Settings — Alerts .......................................................................................................... 756.18. Web Console Log Viewer .................................................................................................................. 7611.1. Bucket Configuration ....................................................................................................................... 11911.2. Background Fetch Workflow ............................................................................................................. 12011.3. Couchbase Buckets .......................................................................................................................... 126

viii

List of Tables1.1. Bucket Types ...................................................................................................................................... 41.2. Couchbase Bucket Capabilities ............................................................................................................... 52.1. Network Ports used by Couchbase Server ............................................................................................... 114.1. Input Variables ................................................................................................................................... 364.2. Constants .......................................................................................................................................... 374.3. Input Variables ................................................................................................................................... 374.4. Constants .......................................................................................................................................... 374.5. Variable Calculations .......................................................................................................................... 387.1. Couchbase Server Command-line Tools .................................................................................................. 797.2. couchbase Tool Commands ................................................................................................................. 807.3. Standard couchbase Options ................................................................................................................ 808.1. Supported Request Headers .................................................................................................................. 868.2. HTTP Status Codes ............................................................................................................................. 868.3. Controller Functions ............................................................................................................................ 948.4. Node Attributes .................................................................................................................................. 948.5. Cluster Joining Arguments .................................................................................................................. 1038.6. Additional Arguments ........................................................................................................................ 10312.1. Stats .............................................................................................................................................. 13113.1. Responses to Specific Errors ............................................................................................................. 135

ix

PrefaceThis manual documents the Couchbase Server database system, for the 1.8 release series. For differences between individ-ual version within this release series, see the version-specific notes throughout the manual.

Couchbase Server 1.8 is a re-branded release of Membase Server 1.7 with some additional fixes and changes. See Appen-dix C, Release Notes for more details of the precise changes.

Couchbase Server 1.8 retains all the same functionality and capabilities of Membase Server 1.7, including:

• Memcached compatible

• Clone to grow with auto-sharding

• Zero-downtime maintenance

• Production-ready management and monitoring

• Reliable storage architecture

• Data replication with auto-failover

• Professional SDKs for wide variety of languages

For more help on specific areas of using Couchbase Server 1.8, see:

• Chapter 2, Getting Started

• Chapter 4, Best Practices

• Section 5.2, “Backup and Restore with Couchbase”

• Section 5.1, “Failover with Couchbase”

• Section 4.6, “Using Couchbase in the Cloud”

• Chapter 6, Administration Web Console

• Section 4.2, “Sizing Guidelines”

• Chapter 11, Couchbase Architecture

• Appendix A, FAQs

1

Chapter 1. Introduction to Couchbase ServerCouchbase Server is a distributed, document ("NoSQL") database management system, designed to store the informationfor web applications. Couchbase Server primarily uses RAM as the storage mechanism, enabling it to support very fastcreate, store, update and retrieval operations.

These features are designed to support web application development where the high-performance characteristics are re-quired to support low-latency and high throughput applications. Couchbase Server achieves this on a single server andprovides support for the load to be increased almost linearly by making use of the clustered functionality built into Couch-base Server.

The cluster component distributes data over multiple servers to share the data and I/O load, while incorporating intelli-gence into the server and client access libraries that enable clients to quickly access the right node within the cluster forthe information required. This intelligent distribution allows Couchbase Server to provide excellent scalability that can beextended simply by adding more servers as your load and application requirements increase.

For a more in-depth description of Couchbase Server, see the following sections:

• The guiding principles and design goals of Couchbase Server are covered inSection 1.1, “Couchbase Server Basics”.

• Couchbase Server is part of the NoSQL database movement. For background information on what NoSQL is, and howthis maps to Couchbase Server functionality, see Section 1.2, “Couchbase Server and NoSQL”.

• Information on the different components and systems in Couchbase Server, and how these map to the concepts and ar-chitecture required to understand the fundamentals of how it works are provided in Section 1.3, “Architecture and Con-cepts”.

1.1. Couchbase Server Basics

Couchbase Server is a database platform that combines the principles and components of Membase Server and ApacheCouchDB. From Membase Server, Couchbase Server builds on the high performance, memory-based, document storageinterface, and incorporates the core principles of being Simple,Fast, and Elastic.

• Simple

Couchbase Server is easy to install and manage, and through the document nature and memcached protocol interface,an easy to use database system. Because the database uses the document structure you do not need to create or managethe databases, tables and schemas. The simplified structure also means that the information can be distributed acrossnodes in a Couchbase Cluster automatically, without having to worry about normalizing or sharding your data to scaleout performance.

• Fast

Couchbase Server is fast, primarily because of the in-memory nature of the database. Furthermore, Couchbase Serv-er provides quasi-deterministic latency and throughput, meaning that you can predict and rely on the speed and perfor-mance of your database without having to prepare and cope for spikes in load and problems.

• Elastic

Couchbase Server was built from the core with the ability to expand and distribute the load across multiple servers. Thisis achieved through a combination of intelligence built into the server for distributing the stored data, and complimen-tary intelligence for clients accessing the data to be directed to the right machine. Data is automatically redistributedacross the cluster, and changing the capacity of the cluster is a case of adding or removing nodes and rebalancing thecluster.

Introduction to Couchbase Server

2

In tandem with the elastic nature of Couchbase Server, a Couchbase Cluster also takes advantage of the clustered archi-tecture to support high availability. All nodes in a cluster are identical, and the cluster automatically creates replicas ofinformation across the cluster. If a node fails, the stored data will be available on another node in the cluster.

• memcached Compatibility

memcached is an memory-based caching application that uses the notion of a document store to save important datathat are required by applications directly in RAM. Because the information is stored entirely in RAM, the latency forstoring and retrieving information is very low. As a caching solution in it's right,memcached is used by a wide range ofcompanies, including Google, Facebook, YouTube, FarmVille, Twitter and Wikipedia to help speed up their web-appli-cation performance by acting as a storage location for objects retrieved at comparative expense from a traditional SQLdatabase.

Couchbase Server supports the same client protocol used by memcached for creating, retrieving, updating and delet-ing information in the database. This enables Couchbase Server to be a drop-in replacement for memcached, and thismeans that applications already employing memcached can take advantage of the other functionality within CouchbaseServer, such as clustered and elastic distribution.

1.2. Couchbase Server and NoSQLNoSQL is a somewhat unfortunate term that has been widely used to describe a class of database management systems thatdon't employ a relational data model. The terminology keys off the SQL query language - a hallmark of relational databasemanagement systems. Unfortunately the query language is not the real differentiator; in fact, it is not necessarily a differ-entiator at all. Some NoSQL database management systems do, in fact, support the SQL query language! The fundamen-tal difference in these systems lies not in the query language, but in the non-relational data model they employ. While non-relational database would be a more technically accurate term, it would also be more broad than the term NoSQL intends.It is interesting to note that a backronym has emerged in which NoSQL is proposed to stand for Not Only SQL.While moreaccurate, it is even less descriptive.

NoSQL databases are characterized by their ability to store data without first requiring one to define a database schema.In Membase, data is stored as a distributed, associative array of document IDs and contents, where the value is a blob ofopaque binary data that doesn't conform to a rigid, pre-defined schema from the perspective of the database managementsystem itself. Additionally, and largely enabled by their schema-less nature, these systems tend to support a scale out ap-proach to growth, increasing data and I/O capacity by adding more servers to a cluster; and without requiring any changeto application software. In contrast, relational database management systems scale up by adding more capacity (CPU,Memory and Disk) to a single server to accommodate growth.

Relational databases store information in relations which must be defined, or modified, before data can be stored. A re-lation is simply a table of rows, where each row (also known as a tuple) in a given relation has a fixed set of attributes(columns). These columns are consistent across each row in a relation. Relations (tables) can be further connected throughcross-table references. One table, CITIZENS for example, could hold rows of all individual citizens residing in a town.Another table, PARENTS, could have rows consisting of PARENT, CHILD and RELATIONSHIP fields. The first twofields could be references to rows in the CITIZENS table while the third field describes the parental relationship betweenthe persons in the first two fields (father, mother).

1.3. Architecture and ConceptsIn order to understand the structure and layout of Couchbase Server, you first need to understand the different componentsand systems that make up both an individual Couchbase Server instance, and the components and systems that work to-gether to make up the Couchbase Cluster as a whole.

The following section provides key information and concepts that you need to understand the fast and elastic nature of theCouchbase Server database, and how some of the components work together to support a highly available and high perfor-mance database.


3

1.3.1. Nodes and Clusters

Couchbase Server can be used either in a standalone configuration, or in a cluster configuration where multiple CouchbaseServers are connected together to provide a single, distributed, data store.

In this description:

• Couchbase Server or Node

A single instance of the Couchbase Server software running on a machine, whether a physical machine, virtual ma-chine, EC2 instance or other environment.

All instances of Couchbase Server are identical, provide the same functionality, interfaces and systems, and consist ofthe same components.

• Cluster

A cluster is a collection of one ore more instances of Couchbase Server that are configured as a logical cluster. Allnodes within the cluster are identical and provide the same functionality and information. Information is shared acrossthe entire cluster.

Clusters operate in a completely horizontal fashion. To increase the size of a cluster, you add another node. There are noparent/child relationships or hierarchical structures involved. This means that Couchbase Server scales linearly, both interms of increasing the storage capacity and the performance and scalability.

1.3.2. Cluster Manager

Every node within a Couchbase Cluster includes the Cluster Manager component. The Cluster Manager is responsible forthe following within a cluster:

• Cluster management

• Node administration

• Node monitoring

• Statistics gathering and aggregation

• Run-time logging

• Multi-tenancy

• Security for administrative and client access

• Client proxy service to redirect requests

Access to the Cluster Manager is provided through the administration interface (see Section 1.3.9, “AdministrationTools”) on a dedicated network port , and through dedicated network ports for client access. Additional ports are config-ured for inter-node communication.

1.3.3. Memory Quotas

Couchbase Server manages the memory used across different components of the system:

• Managing Disk and Memory for Nodes in the Cluster

Couchbase Server automatically manages storing the working set between disk and memory resources for nodes in acluster. This allows an installation to have a working set that is larger than the available RAM in the nodes participating


4

in the cluster. To keep throughput high and latency low, Couchbase Server will always keep metadata about all items inmemory.

When configuring a Couchbase Server, a memory quota is set. Couchbase Server will automatically migrate items frommemory to disk when the configured memory quota is reached. If those items are later accessed, they will be movedback into system memory. For efficiency purposes, these operations are performed on a regular basis in the background.

At the moment, there is no ability define a quota for the on-disk persistent storage. It is up to the administrator to appro-priately monitor the disk utilization and take action (either deleting data from Couchbase or adding capacity by upgrad-ing the nodes themselves or adding more nodes).

Couchbase Server monitors and reports on statistics for managing disk and memory. As with any multi-tier cache, ifthe working set of data is greater than the available amount of the bucket RAM quota (the first level of caching), per-formance will drop due to disk access latencies being higher and disk throughput being lower than RAM latencies andthroughput. Acceptable performance of the system is application dependent. Statistics should be monitored in case tun-ing adjustments are required.

• Server Quotas

Each server node has a memory quota that defines the amount of system memory that is available to that server node onthe host system. The first node in a cluster sets a memory quota that is subsequently inherited by all servers joining thecluster. The maximum memory quota set on the first server node must be less than or equal to 80% of the total physicalRAM on that node. A server cannot join a cluster if it has less physical RAM than 1.25x the RAM quota (the same max-imum allocation of 80% of physical RAM to the cluster). If a server that was a standalone cluster joins another cluster,the memory quota is inherited from the cluster to which the node is added.

Server nodes do not have disk quotas. System administrators are responsible for monitoring free disk space on individ-ual server nodes. Each server node in a cluster has its own storage path - the location on disk where data will be stored.Storage paths do not need to be uniform across all server nodes in a cluster. If a server that was a standalone clusterjoins another cluster, the storage path for that server remains unchanged.

• Bucket Quotas

Memory quota allocation is also controlled on a bucket-by-bucket basis. A fixed amount of memory per node is allocat-ed for use by a bucket. Adding or removing nodes will change the size of the bucket.

1.3.4. Buckets

Couchbase Server provides data management services using named buckets. These are isolated virtual containers for da-ta. A bucket is a logical grouping of physical resources within a cluster of Couchbase Servers. They can be used by multi-ple client applications across a cluster. Buckets provide a secure mechanism for organizing, managing, and analyzing datastorage resources.

Couchbase Server provides the two core types of buckets that can be created and managed. Couchbase Server collects andreports on run-time statistics by bucket type.

Table 1.1. Bucket Types

Bucket Type Description

Couchbase Provides highly-available and dynamically reconfigurabledistributed data storage, providing persistence and replica-tion services. Couchbase buckets are 100% protocol com-patible with, and built in the spirit of, the memcached opensource, distributed document cache.

Memcached Provides a directly-addressed, distributed (scale-out), in-memory, document cache. Memcached buckets are de-


5

Bucket Type Description

signed to be used alongside relational database technolo-gy – caching frequently-used data, thereby reducing thenumber of queries a database server must perform for webservers delivering a web application.

The different bucket types support different capabilities. Couchbase-type buckets provide a highly-available and dynami-cally reconfigurable distributed data store. Couchbase-type buckets survive node failures and allow cluster reconfigurationwhile continuing to service requests. Couchbase-type buckets provide the following core capabilities.

Table 1.2. Couchbase Bucket Capabilities

Capability Description

Persistence Data objects can be persisted asynchronously to hard-diskresources from memory to provide protection from serverrestarts or minor failures. Persistence properties are set atthe bucket level.

Replication A configurable number of replica servers can receive copiesof all data objects in the Couchbase-type bucket. If the hostmachine fails, a replica server can be promoted to be thehost server, providing continuous (HA) cluster operationsvia fail-over. Replication is set at the bucket level.

Rebalancing Rebalancing enables load distribution across resources anddynamic addition or removal of buckets and servers in thecluster.

For more information on the bucket types, their configuration and accessibility, see Section 11.6, “Buckets”.

Couchbase Server leverages the memcached storage engine interface and the Couchbase Bucket Engine to enable isolatedbuckets that support multi-tenancy.

Figure 1.1. Multi-tenancy

Smart clients discover changes in the cluster using the Couchbase Management REST API. Buckets can be used to isolateindividual applications to provide multi-tenancy, or to isolate data types in the cache to enhance performance and visibili-


6

ty. Couchbase Server allows you to configure different ports to access different buckets, and gives you the option to accessisolated buckets using either the binary protocol with SASL authentication, or the ASCII protocol with no authentication

Couchbase Server allows you to use and mix different types of buckets (Couchbase and Memcached) as appropriate inyour environment. Buckets of different types still share the same resource pool and cluster resources. Quotas for RAM anddisk usage are configurable per bucket so that resource usage can be managed across the cluster. Quotas can be modifiedon a running cluster so that administrators can reallocate resources as usage patterns or priorities change over time.

1.3.5. vBuckets

A vBucket is defined as the owner of a subset of the key space of a Couchbase cluster. These vBuckets are used to allowinformation to be distributed effectively across the cluster. The vBucket system is used both for distributing data, and forsupporting replicas (copies of bucket data) on more than one node.

Note

vBuckets are not a user-accessible component, but they are a critical component of Couchbase Serverand are vital to the availability support and the elastic nature.

Every document ID belongs to a vBucket. A mapping function is used to calculate the vBucket in which a given documentbelongs. In Couchbase Server, that mapping function is a hashing function that takes a document ID as input and outputsa vBucket identifier. Once the vBucket identifier has been computed, a table is consulted to lookup the server that "hosts"that vBucket. The table contains one row per vBucket, pairing the vBucket to its hosting server. A server appearing in thistable can be (and usually is) responsible for multiple vBuckets.

The hashing function used by Couchbase Server to map document IDs to vBuckets is configurable - both the hashing algo-rithm and the output space (i.e. the total number of vBuckets output by the function). Naturally, if the number of vBucketsin the output space of the hash function is changed, then the table which maps vBuckets to Servers must be resized.

1.3.6. Disk Storage

For performance, Couchbase Server prefers to store and provide all the information to clients using RAM. However, thisis not always possible or desirable in an application. Instead, what is required is the 'working set' of information stored inRAM and immediately available for supporting low-latency responses.

Couchbase Server provides support for storing document on disk. This is provided for a number of reasons:

• Persistence. Couchbase can be loaded with data, shutdown, and started up again without the data having to be re-loadedfrom another source. In this way, Couchbase Server more than simply a caching layer on top of an existing database in-frastructure.

• Faster warm-up times. Other the lifetime of Couchbase Server node there may be soft or hard failures and regular ad-ministration tasks that need to take place. By storing information on disk, when you start-up your Couchbase Servernode the data can be loaded to 'warm-up' the database with the active dataset so that it can start serving requests for dataas quickly as possible.

• Disk storage enables you to store datasets larger than the physical RAM size. Couchbase automatically moves data be-tween RAM and disk (asynchronously in the background) in order to keep regularly used information in memory, andless frequently used data on disk.

The process of moving data from RAM to disk is called eviction, and is configured automatically through thresholds seton each configured bucket in your Couchbase Cluster.

The use of disk storage presents an issue in that a client request for a document ID must know whether the information ex-ists or not. Couchbase Server achieves this using metadata structures. The metadata holds information about each docu-


7

ments stored in the database and this information is held in RAM. This means that the server can always return a 'docu-ment ID not found' response for an invalid document ID, while waiting (and returning) the data for an item either in RAM(in which case it is returned immediately), or after the item has been read from disk (after a delay, or until a timeout hasbeen reached).

The process of moving information to and from disk is asynchronous. Data is evicted to disk from memory in the back-ground while the server continues to service active requests. Eviction requests are written to an eviction queue to be writ-ten to disk in the background. During sequences of high writes to the database, clients will be notified that the server istemporarily out of memory until enough items have been evicted from memory to disk.

Similarly, when the server identifies an item that needs to be loaded from disk because it is not in active memory, theprocess is handled by a background process that processes the load queue and reads the information back disk and intomemory. The client is made to wait until the data has been loaded back into memory before the information is returned.

The asynchronous nature and use of queues in this way enables reads and writes to be handled at a very fast rate, while re-moving the typical load and performance spikes that would otherwise cause a server running Couchbase Server to produceerratic performance.

1.3.7. Failover

Information is distributed around a cluster using a series of replicas. For Couchbase Buckets you can configure the num-ber of replicas(complete copies of the data stored in the bucket) that should be kept within the Couchbase Cluster, and thenumber of replicas should be less than the number of Couchbase Servers configured in your cluster.

In the event of a failure in a server (either due to transient failure, or for administrative purposes), you can use a techniquecalled failover to indicate that a node within the Couchbase Cluster is no longer available, and that the replica vBucketsfor the server are enabled.

The Failover process contacts each server that was acting as a replica and updates the internal table that maps client re-quests for documents to an available server.

Failover can be performed manually, or you can use the built-in automatic failover that reacts after a preset time when anode within the cluster becomes unavailable.

For more information, see Section 5.1, “Failover with Couchbase”.

1.3.8. Client Interface

Within Couchbase Server, the techniques and systems used to get information into and out of the database differ accordingto the level and volume of data that you want to access. The different methods can be identified according to the base op-erations of Create, Retrieve, Update and Delete:

• Create

Information is stored into the database using the memcached protocol interface to store a document against a specifieddocument ID. Bulk operations for setting the documents of a larger number of operations at the same time are available,and these are more efficient that multiple smaller requests.

The value stored can be any binary value, including structured and structured strings, serialized objects (from the nativeclient language), native binary data (for example, images or audio).

• Retrieve

To retrieve, you must know the document ID used to store a particular value, then you can use the memcached protocol(or an appropriate memcached compatible client-library) to retrieve the value stored against a specific document ID.You can also perform bulk operations


8

• Update

To update information in the database, you must use the memcached protocol interface. The memcached protocol in-cludes functions to directly update the entire contents, and also to perform simple operations, such as appending infor-mation to the existing record, or incrementing and decrementing integer values.

• Delete

To delete information from Couchbase Server you need to use the memcached protocol which includes an explicitdelete command to remove a document from the server.

However, Couchbase Server also allows information to be stored in the database with an expiry value. The expiry valuestates when a document should be automatically deleted from the database, and can either be specified as a relative time(for example, in 60 seconds), or absolute time (31st December 2012, 12:00pm).

The methods of creating, updating and retrieving information are critical to the way you work with storing data in Couch-base Server.

1.3.9. Administration Tools

Couchbase Server was designed to be as easy to use as possible, and does not require constant attention. Administration ishowever offered in a number of different tools and systems. For a list of the most common administration tasks, see Chap-ter 5, Administration Tasks.

Couchbase Server includes three solutions for managing and monitoring your Couchbase Server and cluster:

• Web Administration Console

Couchbase Server includes a built-in web-administration console that provides a complete interface for configuring,managing, and monitoring your Couchbase Server installation.

For more information, see Chapter 6, Administration Web Console.

• Administration REST API

In addition to the Web Administration console, Couchbase Server incorporates a management interface exposedthrough the standard HTTP REST protocol. This REST interface can be called from your own custom management andadministration scripts to support different operations.

Full details are provided in Chapter 8, Couchbase Management REST API

• Command Line Interface

Couchbase Server includes a suite of command-line tools that provide information and control over your CouchbaseServer and cluster installation. These can be used in combination with your own scripts and management proceduresto provide additional functionality, such as automated failover, backups and other procedures. The command-line toolsmake use of the REST API.

For information on the command-line tools available, see Chapter 8, Couchbase Management REST API.

1.3.10. Statistics and Monitoring

In order to understand what your cluster is doing and how it is performing, Couchbase Server incorporates a complete setof statistical and monitoring information. The statistics are provided through all of the administration interfaces. Withinthe Web Administration Console, a complete suite of statistics are provided, including built-in real-time graphing and per-formance data.


9

The statistics are divided into a number of groups, allowing you to identify different states and performance informationwithin your cluster:

• By Node

Node statistics show CPU, RAM and I/O numbers on each of the servers and across your cluster as a whole. This infor-mation can be used to help identify performance and loading issues on a single server.

• By vBucket

The vBucket statistics show the usage and performance numbers for the vBuckets used to store information in the clus-ter. These numbers are useful to determine whether you need to reconfigure your buckets or add servers to improve per-formance.

• By Disk Queues

These statistics monitor the queues used to read and write information to disk and between replicas. This informationcan be helpful in determining whether you should expand your cluster to reduce disk load.

• By TAP Queues

The TAP interface is used to monitor changes and updates to the database. TAP is used internally by Couchbase to pro-vide replication between Couchbase nodes, but can also be used by clients for change notifications.

In nearly all cases the statistics can be viewed both on a whole of cluster basis, so that you can monitor the overall RAMor disk usage for a given bucket, or an individual server basis so that you can identify issues within a single machine.

10

Chapter 2. Getting StartedTo start using Couchbase Server, you need to follow these steps:

1. Prepare your target system by ensuring that you meet the system requirements. See Section 2.1, “Preparation”.

2. Install Couchbase Server using one of the available binary distributions. See Section 2.2, “Installing Couchbase Serv-er”.

3. Test the installation by connecting to the Couchbase Server and storing some data using the native Memcached proto-col. See Section 2.5, “Testing Couchbase Server”.

4. Setup the new Couchbase Server system by completing the web-based setup instructions. See Section 2.4, “Setting upCouchbase Server”.

2.1. PreparationWarning

Heterogeneous or mixed deployments (deployments with both Linux and Windows server nodes) arenot supported at this time. It is recommended that when deploying to multiple systems, that system berunning the same operating system.

When running Couchbase Server your system should meet or exceed the following system requirements.

2.1.1. Supported Platforms

The following operating systems are supported:

• RedHat Enterprise Linux 5.2 (Deprecated). Requires additional third-party libraries and packages.

• RedHat Enterprise Linux 5.4 (32-bit and 64-bit)

• Ubuntu Linux 10.04 (32-bit and 64-bit)

• Ubuntu Linux 11.10 (32-bit and 64-bit) Developer Only

• Windows Server 2008 (32-bit and 64-bit)

• Mac OS X 10.5 or higher (minimum), 10.6 or higher preferred (64-bit only) Developer Only

2.1.2. Hardware Requirements

The following hardware requirements are recommended for installation:

• Quad-core, 64-bit CPU running at 3GHz

• 16GB RAM (physical)

A minimum specification machine should have the following characteristics:

• Dual-core CPU running at 2GHz

• 4GB RAM (physical)

2.1.3. Storage Requirements

For running Couchbase Server you must have the following storage available:

• 100MB for application logging

Getting Started

11

• Disk space to match your physical RAM requirements for persistence of information

2.1.4. Web Browser (for administration)

The Couchbase Server administration interface is supported using the following Web browsers, with Javascript supportenabled:

• Mozilla Firefox 3.6 or higher

To enable JavaScript, select the Enable JavaScript option within the Content panel of the application preferences.

• Safari 5 or higher

To enable JavaScript, use the checkbox on the security tab of the application preferences.

• Google Chrome 11 or higher

To enable JavaScript, use the Allow all sites to run JavaScript (recommended)option within the Content button of theUnder the Hood section of the application preferences.

• Internet Explorer 8 or higher

To enable JavaScript, by enabling Active Scripting within the Custom Level, section of the Security section of the Inter-net Options item of the Tools menu.

2.1.5. Network Ports

The following network ports are used by Couchbase Server and must be opened on servers, data clients and administrationclients as noted below. Firewall configurations should be updated accordingly to allow traffic through on these ports. Forthe Couchbase Server ports are used for communication with data clients, administration clients, and for inter-node datatransfer within the Couchbase cluster. You may choose to configure explicit host access for different ports, for exampleexplicitly allowing port 11211 between each Couchbase Server and data clients that use it:

Table 2.1. Network Ports used by Couchbase Server

Port Purpose Couchbase Server Couchbase Client AdministrationClient

8091 Web AdministrationPort

Yes No Yes

11211 Data Port Yes Yes Yes

11210 Internal Cluster Port Yes Yes No

4369 Erlang Port Mapper(epmd)

Yes No No

21100 to 21199 (inclu-sive)

Node data exchange Yes No No

2.2. Installing Couchbase ServerTo install Couchbase Server on your machine you must download the appropriate package for your chosen platform fromhttp://www.couchbase.com/downloads. For each platform, following the corresponding platform-specific instructions.

2.2.1. Red Hat Linux Installation

The RedHat installation uses the RPM package. Installation is supported on RedHat and RedHat based operating systemssuch as CentOS.

http://www.couchbase.com/downloads

Getting Started

12

To install, use the rpm command-line tool with the RPM package that you downloaded. You must be logged in as root(Superuser) to complete the installation:

root-shell> rpm --install couchbase-server version.rpm

Where version is the version number of the downloaded package.

Once the rpm command has been executed, the Couchbase Server starts automatically, and is configured to automaticallystart during boot under the 2, 3, 4, and 5 runlevels. Refer to the RedHat RPM documentation for more information aboutinstalling packages using RPM.

Once installation has completed, the installation process will display a message similar to that below:

Starting Couchbase server: [ OK ]

You have successfully installed Couchbase Server.Please browse to http://hostname:8091/ to configure your server.Please refer to http://couchbase.com/support foradditional resources.

Please note that you have to update your firewall configuration toallow connections to the following ports: 11211, 11210, 4369, 8091and from 21100 to 21199.

By using this software you agree to the End User License Agreement.See /opt/couchbase/LICENSE.txt.

Once installed, you can use the RedHat chkconfig command to manage the Couchbase Server service, including checkingthe current status and creating the links to enable and disable automatic startup. Refer to the RedHat documentation for in-structions.

To continue installation you must open a web browser and access the web administration interface. See Section 2.4, “Set-ting up Couchbase Server”.

2.2.2. Ubuntu Linux Installation

The Ubuntu installation uses the DEB package.

To install, use the dpkg command-line tool using the DEB file that you downloaded. The following example uses sudowhich will require root-access to allow installation:

shell> dpkg -i couchbase-server version.deb

Where version is the version number of the downloaded package.

Once the rpm command has been executed, the Couchbase Server starts automatically, and is configured to automatical-ly start during boot under the 2, 3, 4, and 5 runlevels. Refer to the Ubuntu documentation for more information about in-stalling packages using the Debian package manager.

Once installation has completed, the installation process will display a message similar to that below:

Selecting previously deselected package couchbase-server.(Reading database ... 218698 files and directories currently installed.)Unpacking couchbase-server (from couchbase-server-community_x86_64_beta.deb) ...Setting up couchbase-server (2-0~basestar) ... * Started Couchbase server

You have successfully installed Couchbase Server.Please browse to http://tellurium-internal:8091/ to configure your server.Please refer to http://couchbase.com for additional resources.

Please note that you have to update your firewall configuration toallow connections to the following ports: 11211, 11210, 4369, 8091and from 21100 to 21199.

By using this software you agree to the End User License Agreement.See /opt/couchbase/LICENSE.txt.

Getting Started

13

Once installed, you can use the service command to manage the Couchbase Server service, including checking the currentstatus. Refer to the Ubuntu documentation for instructions.

To continue installation you must open a web browser and access the web administration interface. See Section 2.4, “Set-ting up Couchbase Server”.

2.2.3. Microsoft Windows Installation

To install on Windows you must download the Windows installer package. This is supplied as an Windows executable.You can install the package either using the GUI installation process, or by using the unattended installation process.

2.2.3.1. GUI Installation

To use the GUI installer, double click on the downloaded executable file.

The installer will launch and prepare for installation. You can cancel this process at any time. Once completed, you will beprovided with the welcome screen.

Figure 2.1. Windows Installation Welcome Screen

Click Next to start the installation. You will be prompted with the Installation Location screen. You can change the loca-tion where the Couchbase Server application is located. Note that this does not configure the location of where the persis-

Getting Started

14

tent data will be stored, only the location of the application itself. To select the install location, click the Browse button toselect the folder. Click Next to continue the installation.

Figure 2.2. Windows Installation Location Screen

Configuration has now been completed. You will be prompted to confirm that you want to continue installation. ClickNext to confirm the installation and start the installation process.

Getting Started

15

Figure 2.3. Windows Installation Ready Screen

The install will copy over the necessary files to the system. During the installation process, the installer will also check toensure that the default administration port is not already in use by another application. If the default port is unavailable,the installer will prompt for a different port to be used for administration of the Couchbase Server.

Getting Started

16

Figure 2.4. Windows Installation Progress Screen

Once the installation process has been completed, you will be prompted with the completion screen. This indicates that theinstallation has been completed and your Couchbase Server is ready to be setup and configured. When you click Finish,the installer will quit and automatically open a web browser with the Couchbase Server setup window.

Getting Started

17

Figure 2.5. Windows Installation Completion Screen

To continue installation you should follow the server setup instructions. See Section 2.4, “Setting up Couchbase Server”.

2.2.3.2. Unattended Installation

The unattended installation process works by first recording your required installation settings using the GUI installationprocess outlined above which are saved to a file. You can then use the file created to act as the option input to future in-stallations.

To record your installation options, open a Command Terminal or Powershell and start the installation executable with the/r command-line option:

C:\Downloads> couchbase_server_version.exe /r

You will be prompted with the installation choices as outlined above, but the installation process will not actually be com-pleted. Instead, a file with your option choices will be recorded in the file C:\Windows\setup.iss.

To perform an installation using a previously recorded setup file, copy the setup.iss file into the same directory as theinstaller executable. Run the installer from the command-line, this time using the/s option.

C:\Downloads> couchbase_server_version.exe /s

You can repeat this process on multiple machines by copying the executable package and the setup.iss file to eachmachine.

2.2.4. Mac OS X Installation

The Mac OS X installation uses a Zip file which contains a standalone application that can be copied to the Applica-tions folder or to any other location you choose. The installation location does not affect the location of the Couchbasedata files.

To install:

Getting Started

18

1. Download the Mac OS X Zip file.

2. Double-click the downloaded Zip installation file to extract the contents. This will create a single file, theCouchbase.app application.

3. Drag and Drop the Couchbase.app to your chosen installation folder, such as the system Applications folder.

Once the application has been copied to your chosen location, you can double-click on the application to start it. The ap-plication itself has no user interface. Instead, the Couchbase application icon will appear in the menubar on the right-handside. If there is no active configuration for Couchbase, then the Couchbase Web Console will be opened and you will beasked to complete the Couchbase Server setup process. See Section 2.4, “Setting up Couchbase Server” for more details.

The Couchbase application runs as a background application. Clicking on the menubar gives you a list of operations thatcan be performed. For more information, see Section 3.2.3, “Startup and Shutdown on Mac OS X”.

2.3. Upgrading to Couchbase Server 1.8Note

In-Place and rolling upgrades to 1.8.0 are supported from 1.7.x versions only. For Membase Server1.6 and earlier you must upgrade to Membase Server 1.7 first.

In-Place Upgrades. It's important to ensure that your disk write queue (Section 12.3, “Disk Write Queue”) has beencompletely drained before shutting down the Membase Server service. This will ensure that all data has been persisted todisk and will be available after the upgrade. It is a best practice to turn off the application and allow the queue to drain pri-or to beginning the upgrade.

Before beginning any upgrade, a backup should be taken as a best practice, see Section 5.2, “Backup and Restore withCouchbase”.

Upgrade Steps

1. Download couchbase-server-edition_and_arch_1.8.0 and copy it to every node in your cluster.

2. Turn off your application, so that no requests are going to your Membase cluster.

3. Ensure that Disk Write Queue size reaches zero (all data has been persisted to disk).

4. Next, on each individual node (and this may be concurrent):

a. Backup each node's data using mbbackup

b. Backup each node's configuration files. While the upgrade script will perform a backup of the configuration and datafiles, it is our recommended best practice to take your own backup of the files located at:

• 1.7.0 Linux

/opt/membase/var/lib/membase/config/config.dat

• 1.7.0 Windows

C:\Program Files\Membase\Server\Config\config.dat

Important

If you have multiple version subdirectories in your /etc/opt/membase directory, you mustfirst cleanup the directory so only the last, most recent version remains.

5. Linux Upgrade Process

Getting Started

19

Linux package managers will prevent couchbase-server from being installed when there's already a membase-server in-stalled.

• Red Hat / CentOS

a. Uninstall the existing membase-server package — this will keep the user's db data and copies of their config-uration.

b. Install Couchbase Server 1.8 with special environment variable flags, which force an upgrade. The special env varis INSTALL_UPGRADE_CONFIG_DIR.

INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config rpm -i \ couchbase-server-edition_and_arch_1.8.0.rpm

• Ubuntu

a. Uninstall the existing membase-server package — this will keep the user's db data and copies of their config-uration.

b. Install Couchbase Server 1.8 with special environment variable flags, which forces an upgrade. The special envvar is INSTALL_UPGRADE_CONFIG_DIR

INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config dpkg -i \ couchbase-server-edition_and_arch_1.8.0.deb

• Windows Upgrade Process

Windows installer will upgrade your current membase server installation to Couchbase Server in the same loca-tion where Membase Server was already installed. If you have installed membase server in the default location, C:\Program Files\Membase\Server, Couchbase Server installer will copy the new files to the same location.Once the upgrade process is completed you will see Couchbase Server icon in the Desktop and under Start->Pro-grams replacing membase server.

After every node has been upgraded and restarted, and you can follow the steps below to monitor its progress of "warmingup" Turn your application back on.

2.3.1. Manually Controlled Upgrade Options

Note

This section is not applicable to Windows.

By using environment variable flags during installation you amy optionall take more control of the upgrade process andresults. The available environment variables are:

• INSTALL_UPGRADE_CONFIG_DIR

This variable sets the value of the directory of the previous versions config directory. When this environment variable isdefined, the rpm/dpkg scripts will upgrade configuration files and data records from Membase Server 1.7 to CouchbaseServer 1.8.

The data directory defined and used by your Membase Server 1.7 installation will continue to be used by your upgrad-ed Couchbase Server 1.8.0 instance. For example, if you had mounted/mapped special filesystems for use while runningMembase Server 1.7, those paths will continue to be used after upgrading to Couchbase Server 1.8.0.

• INSTALL_DONT_START_SERVER

When set to '1', the rpm/dpkg scripts will not automatically start the Couchbase Server as its last step.

Getting Started

20

• INSTALL_DONT_AUTO_UPGRADE

When set to '1', the rpm/dpkg scripts will not automatically invoke the cbupgrade script that's included inCouchbase Server 1.8.0, allowing you to manually invoke cbupgrade later. This may be useful in case youneed to perform more debugging. This should be used with the INSTALL_DONT_START_SERVER=1 andINSTALL_UPGRADE_CONFIG_DIR= PATH environment variables.

Example flag usage for RedHat / CentOS:

INSTALL_DONT_START_SERVER=1 INSTALL_DONT_AUTO_UPGRADE=1 \ INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config \ rpm -i couchbase-server-community_x86_64_1.8.0.rpm

For Ubuntu

INSTALL_DONT_START_SERVER=1 INSTALL_DONT_AUTO_UPGRADE=1 \ INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config \ dpkg -i couchbase-server-community_x86_64_1.8.0.deb

Example output when using flags, first uninstalling the existing Membase Server 1.7.x:

[root@localhost ~]# rpm -e membase-serverStopping membase-server[ OK ]warning: /opt/membase/var/lib/membase/config/config.dat saved as /opt/membase/var/lib/membase/config/config.dat.rpmsave[root@localhost ~]# INSTALL_DONT_START_SERVER=1 INSTALL_DONT_AUTO_UPGRADE=1INSTALL_UPGRADE_CONFIG_DIR=/opt/membase/var/lib/membase/config rpm -icouchbase-server-community_x86_64_1.8.0r-55-g80f24f2.rpmUpgrading couchbase-server .../opt/couchbase/bin/cbupgrade -c /opt/membase/var/lib/membase/config -a yesSkipping cbupgrade due to INSTALL_DONT_AUTO_UPGRADE ...Skipping server start due to INSTALL_DONT_START_SERVER ...You have successfully installed Couchbase Server.Please browse to http://localhost.localdomain:8091/ to configure your server.Please refer to http://couchbase.com for additional resources.Please note that you have to update your firewall configuration toallow connections to the following ports: 11211, 11210, 4369, 8091and from 21100 to 21299.By using this software you agree to the End User License Agreement.See /opt/couchbase/LICENSE.txt.[root@localhost ~]#

2.3.2. Using cbupgrade Manually

Note

This section is not applicable to Windows.

After using the INSTALL_DONT_AUTO_UPGRADE option, you can use the /opt/couchbase/bin/cbupgrade program lat-er to fully control the upgrade steps. It's command-line options include:

[root@localhost ~]# /opt/couchbase/bin/cbupgrade -hUsage: cbupgrade [-c path/to/previous/config/dir] [-a AUTO] [-d FILENAME] [-n] [-s FACTOR]-c <path/to/previous/config/dir>-- example: -c /etc/opt/membase/1.6.5.3.1-a <yes|no>-- automatic or non-interactive mode; default is 'no';'yes' to force automatic 'yes' answers to all questions-d <dbdir_output_file>-- retrieve db directory from config file and exit-n -- dry-run; don't actually change anything-s <free_disk_space_needed_factor>-- free disk space needed, as a factor of current bucket usage-- default value is 2.0-- example: -s 1.0

The cbupgrade program can be run using the -n flag, which tells cbupgrade to not modify any files, but just describe thechanges it would make. For example:

Getting Started

21

[root@localhost ~]# /opt/couchbase/bin/cbupgrade -c /opt/membase/var/lib/membase/config -nDry-runmode: no actual upgrade changes will be made.Upgrading your Couchbase Server to 1.8.0r-55-g80f24f2.The upgrade process might take awhile.Analysing...Previous config.dat file is /opt/membase/var/lib/membase/config/config.dat.rpmsaveTarget node: [email protected]/Couchbase should not be running.Please use: /etc/init.d/couchbase-server stopor: /etc/init.d/membase-server stopIs the Membase/Couchbase server already stopped? [yes|no]yesDatabase dir: /opt/membase/var/lib/membase/dataIs that the expected database directory to upgrade? [yes|no]yesBuckets to upgrade: defaultAre those the expected buckets to upgrade? [yes|no]yesChecking disk space available for buckets in directory:/opt/membase/var/lib/membase/dataFree disk bucket space wanted: 0.0Free disk bucket space available: 177790963712Free disk space factor: 2.0Ok.Analysis complete.Proceed with config & data upgrade steps? [yes|no]yesSKIPPED (dry-run): Copying /opt/membase/var/lib/membase/config/config.dat.rpmsaveSKIPPED (dry-run): cp /opt/membase/var/lib/membase/config/config.dat.rpmsave /opt/couchbase/var/lib/couchbase/config/config.datEnsuring bucket data directories.SKIPPED (dry-run): Ensuring bucket data directory: /opt/membase/var/lib/membase/data/default-dataSKIPPED (dry-run): mkdir -p /opt/membase/var/lib/membase/data/default-dataSKIPPED (dry-run): Ensuring dbdir owner/group: /opt/membase/var/lib/membase/dataSKIPPED (dry-run): chown -R couchbase:couchbase /opt/membase/var/lib/membase/dataSKIPPED (dry-run): Ensuring dbdir owner/group: /opt/membase/var/lib/membase/dataSKIPPED (dry-run): chown -R couchbase:couchbase /opt/membase/var/lib/membase/dataUpgrading buckets.Skipping already converted bucket: /opt/membase/var/lib/membase/data/default-dataSkipping already converted bucket: /opt/membase/var/lib/membase/data/test0-dataDone.

2.4. Setting up Couchbase Server

To setup the Couchbase Server you must use the web browser setup screens to configure the Couchbase Server installa-tion, including setting the memory settings, disk locations, and existing cluster configuration. You will also be asked tocreate a password to be used when logging in and administering your server.

Note

We recommend that you clear your browser cache before starting the setup process. You can findnotes and tips on how to do this on different browsers and platforms on this page.

To start the configuration and setup process, you should open the Couchbase Web Console. On Windows this is openedfor you automatically. On all platforms you can access the web console by connecting to the embedded web server onport 8091. For example, if your server can be identified on your network as servera, you can access the web console byopening http://servera:8091/. You can also use an IP address or, if you are on the same machine, http://lo-calhost:8091.

1. Once you have opened the web console for the first time immediately after installation you will be prompted with thescreen shown below.

http://www.wikihow.com/Clear-Your-Browser's-Cache

Getting Started

22

Figure 2.6. Couchbase Server Setup

Click the SETUP button to start the setup process.

2. First, you must set the disk storage and cluster configuration.

Getting Started

23

Figure 2.7. Couchbase Server Setup Step 1 (New Cluster)

Configure Disk Storage

The Configure Disk Storage option specifies the location of the persistent (on-disk) storage used by Couchbase Server.The setting affects only this server and sets the directory where all the data will be stored on disk.

Join Cluster/Start New Cluster

The Configure Server Memory section sets the amount of physical RAM that will be allocated by Couchbase Server forstorage.

Getting Started

24

If you are creating a new cluster, you specify the memory that will be allocated on each node within your Couchbasecluster. You must specify a value that will be supported on all the nodes in your cluster as this is a global setting.

If you want to join an existing cluster, select the radio button. This will change the display and prompt the IP address ofan existing node, and the username and password of an administrator with rights to access the cluster.

Figure 2.8. Couchbase Server Setup Step 1 (Existing Cluster)

Click Next to continue the installation process.

3. You must specify the default bucket for this server.

Getting Started

25

Figure 2.9. Couchbase Server Setup Step 2

The options are:

• Bucket Type

Specifies the type of the bucket to be created, either Memcached or Couchbase. See Section 1.3.4, “Buckets” formore information.

The remainder of the options differ based on your selection.

When selecting the Couchbase bucket type:

• Memory Size

This option specifies the amount of available RAM configured on this server which should be allocated to the de-fault bucket.

• Replication

For Couchbase buckets you can enable replication to support multiple replicas of the default bucket across theservers within the cluster. You can configure up to three replicas. Each replica receives copies of all the docu-ments that are managed by the bucket. If the host machine for a bucket fails, a replica can be promoted to take itsplace, providing continuous (high-availability) cluster operations in spite of machine failure.

Getting Started

26

You can disable replication by setting the number of replica copies to zero (0). This will configure the defaultbucket as local-only and therefore only accessible on this machine.

When selecting the Memcached bucket type:

• Memory Size

The bucket is configured with a per-node amount of memory. Total bucket memory will change as nodes areadded/removed.

For more information, see Section 1.3.3, “Memory Quotas”.

Click Next to continue the setup process.

4. You can optionally enable the notification system within the Couchbase Web Console.


If you select the Update Notifications option, the Web Console will communicate with Couchbase servers to confirmthe version number of your Couchbase installation. During this process, the client submits the following information tothe Couchbase Server:

Getting Started

27

• The current version of your Couchbase Server installation. When a new version of Couchbase Server becomes avail-able, you will be provided with notification of the new version and information on where you can download the newversion.

• Basic information about the size and configuration of your Couchbase cluster. This information will be used to helpus prioritize our development efforts.

Note

The process occurs within the browser accessing the web console, not within the server itself, andno further configuration or internet access is required on the server to enable this functionality.Providing the client accessing the Couchbase Server console has internet access, the informationcan be communicated to the Couchbase Servers.

The update notification process the information anonymously, and the data cannot be tracked. Theinformation is only used to provide you with update notification and to provide information thatwill help us improve the future development process for Couchbase Server and related products.

• Enterprise Edition

You can also register your product from within the setup process. On Enterprise Editions of Couchbase Server youcan optionally select that Couchbase, Inc. plants a tree as a thank you for completing the registration process. Formore information on the tree planting process, see Mokugift where you can find out about the tree planting sponser-ship programme, supported by the United Nations Encironment Programme.

To have your tree planted, please fill in your email address, name and company details, and tick the Yes, please plantmy tree! checkbox.

• Community Edition

Supplying your email address will add you to the Couchbase community mailing list, which will provide you withnews and update information about Couchbase and related products. You can unsubscribe from the mailing list atany time using the unsubscribe link provided in each email communication.

Click Next to continue the setup process.

5. The final step in the setup process is to configure the username and password for the administrator of the server. If youare create a new cluster then this information will be used to authenticate each new server into the cluster. The samecredentials are also used when using the Couchbase Management REST API. Enter a username and password. Thepassword must be at least six characters in length.

http://www.mokugift.com/

Getting Started

28


Click Next to continue the complete the process.

Once the setup process has been completed, you will be presented with the Couchbase Web Console showing the ClusterOverview page.

Getting Started

29

Figure 2.12. Couchbase Server Setup Completed

Your Couchbase Server is now running and ready to use.

2.5. Testing Couchbase Server

You can test your Couchbase Server installation by using Telnet to connect to the server and using the Memcached textprotocol. This is the simplest method for determining if your Couchbase Server is running.

Note

You will not need to use the Telnet method for communicating with your server within your applica-tion. Instead, use one of the Couchbase SDKs.

You will need to have telnet installed on your server to connect to Couchbase Server using this method. Telnet is suppliedas standard on most platforms, or may be available as a separate package that should be easily installable via your operat-ing systems standard package manager.

Getting Started

30

Connect to the server:

shell> telnet localhost 11211Trying 127.0.0.1...Connected to localhost.localdomain (127.0.0.1).Escape character is '^]'.

Make sure it's responding (stats is a great way to check basic health):

statsSTAT delete_misses 0STAT ep_io_num_write 0STAT rejected_conns 0...STAT time 1286678223...STAT curr_items_tot 0...STAT threads 4STAT pid 23871...END

Put a document in:

set test_key 0 0 1aSTORED

Retrieve the document:

get test_keyVALUE test_key 0 1aEND

Disconnect:

quitConnection closed by foreign host.shell>

All of the Memcached protocols commands will work through Telnet.

2.6. Next Steps

• For basic instructions on using your Couchbase Server installation, seeChapter 3, Administration Basics.

• For information on deploying and building your Couchbase Server cluster, seeSection 4.7, “Deployment Strategies”.

• For instructions on how to use the Couchbase Web Console to manage your Couchbase Server installation, see Chap-ter 6, Administration Web Console.

• If you already have an application that uses the Memcached protocol then you can start using your Couchbase Serverimmediately. If so, you can simply point your application to this server like you would any other memcached server.No code changes or special libraries are needed, and the application will behave exactly as it would against a standardmemcached server. Without the client knowing anything about it, the data is being replicated, persisted, and the clustercan be expanded or contracted completely transparently.

If you do not already have an application, then you should investigate one of the available Couchbase client libraries toconnect to your server and start storing and retrieving information. For more information, see Couchbase SDKs.

http://www.couchbase.com/develop

31

Chapter 3. Administration BasicsThis chapter covers everything on the Administration of a Couchbase Sever cluster. Administration is supported throughthree primary methods:

• Couchbase Web Console

Couchbase includes a built-in web server and administration interface that provides access to the administration and sta-tistic information for your cluster.

For more information, read Chapter 6, Administration Web Console.

• Command-line Toolkit

Provided within the Couchbase package are a number of command-line tools that all you to communicate and controlyour Couchbase cluster.

For more information, read Chapter 7, Couchbase Command-line Interface.

• Couchbase REST API

Couchbase Server includes a RESTful API that enables any tool capable of communicating over HTTP to administerand monitor a Couchbase cluster.

For more information, read Chapter 8, Couchbase Management REST API.

3.1. Working with Couchbase data files

By default, Couchbase will store the data files under the following paths:

Linux:

/opt/couchbase/<version>/data/ns_1

Windows:

C:\Program Files\Couchbase\Server\data\ns_1

This path can be changed for each node at setup either via the Web UI setup wizard, using theREST API or using theCouchbase CLI:

Linux:

/opt/couchbase/bin/membase node-init -c <node_IP>:8091 » --node-init-data-path=<new_path> -u <user> -p » <password>

Windows:

C:\Program Files\Couchbase\Server\bin\membase node-init -c » <node_IP>:8091 --node-init-data-path=<new_path> -u » <user> -p <password>

Once a node or cluster has already been setup and storing data, you can still change the path but it is not an "online" opera-tion:

1. Change the path on a running node either via the REST API or using the Couchbase CLI (commands above). Thischange won't actually take effect until the node is restarted.

Administration Basics

32

2. Shut the service down

3. Copy all the data files from their original location into the new location

4. Start the service again and monitor the "warmup" of the data.

3.2. Startup and Shutdown of Couchbase Server

3.2.1. Startup and Shutdown on Linux

The Couchbase Server installation automatically adds a standard initialization script within/etc/init.d that will safe-ly startup and shutdown your Couchbase Server installation.

3.2.2. Startup and Shutdown on Windows

On Windows, Couchbase Server is installed as a Windows service. You can use the Services tab within the Windows TaskManager to start and stop Couchbase Server.

Note

You will need Power User or Administrator privileges, or have been separately granted the rights tomanage services to start and stop Couchbase Server.

By default, the service should start automatically when the machine boots. To manually start the service, open the Win-dows Task Manager and choose the Services tab, or select the Start, choose Run and then type Services.msc to openthe Services management console.

Once open, find the CouchbaseServer service, right-click and then choose to Start or Stop the service as appropriate. Youcan also alter the configuration so that the service is not automatically started during boot.

Alternatively, you can start and stop the service from the command-line, either by using the system net command. For ex-ample, to start Couchbase Server:

shell> net start CouchbaseServer

To stop Couchbase Server:

shell> net stop CouchbaseServer

Start and Stop scripts are also provided in the standard Couchbase Server installation in the bin directory. To start theserver using this script:

shell> C:\Program Files\Couchbase\Server\bin\service_start.bat

To stop the server using the supplied script:

shell> C:\Program Files\Couchbase\Server\bin\service_stop.bat

3.2.3. Startup and Shutdown on Mac OS X

On Mac OS X, Couchbase Server is supplied as a standard application. You can start Couchbase Server by double clickingon the application. Couchbase Server runs as a background application which installs a menubar item through which youcan control the server.


33

Figure 3.1. Couchbase Server on Mac OS X Menubar Item

The individual menu options perform the following actions:

• About Couchbase

Opens a standard About dialog containing the licensing and version information for the Couchbase Server installed.

• Open Admin Console

Opens the Web Administration Console in your configured default browser.

• Visit Support Forum

Opens the Couchbase Server support forum within your default browser at the Couchbase website where you can askquestions to other users and Couchbase developers.

• Check for Updates

Checks for updated versions of Couchbase Server. This checks the currently installed version against the latest versionavailable at Couchbase and offers to download and install the new version. If a new version is available, you will bepresented with a dialog containing information about the new release.

If a new version is available, you can choose to skip the update, notify the existence of the update at a later date, or toautomatically update the software to the new version.

If you choose the last option, the latest available version of Couchbase Server will be downloaded to your machine, andyou will be prompted to allow the installation to take place. Installation will shut down your existing Couchbase Serverprocess, install the update, and then restart the service once the installation has been completed.

Once the installation has been completed you will be asked whether you want to automatically update Couchbase Serv-er in the future.

Note

Using the update service also sends anonymous usage data to Couchbase on the current versionand cluster used in your organization. This information is used to improve our service offerings.

You can also enable automated updates by selecting the Automatically download and install updates in the futurecheckbox.


34

• Launch Admin Console at Start

If this menu item is checked, then the Web Console for administrating Couchbase Server will be opened whenever theCouchbase Server is started. Selecting the menu item will toggle the selection.

• Automatically Start at Login

If this menu item is checked, then Couchbase Server will be automatically started when the Mac OS X machine starts.Selecting the menu item will toggle the selection.

• Quit Couchbase

Selecting this menu option will shut down your running Couchbase Server, and close the menubar interface. To restart,you must open the Couchbase Server application from the installation folder.

35

Chapter 4. Best PracticesWhen designing and building your Couchbase Server cluster you need to give some through to a number of different as-pects if you server and cluster configuration, including the configuration and hardware of individual nodes, in addition tothe overall cluster sizing and distribution configuration.

For more information on the basic design and configuration of your Couchbase Server cluster, see Section 4.1, “ClusterDesign Considerations”.

If you are hosting in the cloud, please read here

4.1. Cluster Design Considerations• RAM: Memory is one of the most important factor that determines how smoothly your cluster will operate. Couchbase

is well suited for applications that want most of its active dataset in memory. This data that is actively used at any giv-en point in time is called the Working Set. It is very important that you allocate enough memory for the entire Work-ing Set to live in memory. When there is not enough memory left for the new data that is written, some values are eject-ed from memory and will only exist on disk. Accessing values from disk is much slower than accessing data in memo-ry. As a result, if ejected data is accessed frequently, performance of the cluster suffers. Use the formula provided in thenext section to verify your configuration, to optimize performance and avoid this situation.

• Number of Nodes: Once you know how much memory you will need for the cluster the next decision you will make iswhether to have few large nodes or several small nodes:

• With several smaller nodes you are distributing I/O across several machines, however, the probability of a node fail-ing (across the whole cluster) is also higher.

• With fewer larger nodes, in case of a node failure the impact to the application will be greater

• It is a trade off between reliability and efficiency.

• Moxi: A client side moxi or a smart client is always preferred over a server side moxi but for development environ-ments or for faster, easier deployments you can use server side moxis. The downside of a server side moxi is that therecan be an additional hop if the server that receives the client request does not have data requested. Read more aboutclients here. Read more about different Deployment Strategies here.

• Number of cores - Couchbase is relatively more memory or I/O bound than is CPU bound. However, Couchbase ismore efficient on machines that have at least two cores.

• Storage type - You may choose either SSDs (solid state drives) or spinning disks to store data. SSDs are faster than ro-tating media, but currently have a higher cost. Couchbase needs less memory if a cluster uses SSDs as the I/O queuebuffer is smaller in case of SSDs.

• WAN Deployments - Couchbase is not intended to be used in WAN configurations. Couchbase requires that the latencyshould be very low between server nodes and between servers nodes and Couchbase clients.

4.2. Sizing GuidelinesThe primary considerations when planning your Couchbase Server cluster are:

• How many nodes do I node?

• How large (RAM, CPU, disk space) should those nodes be?

To answer the first question, you need to take into account the following different factors:

• RAM


Best Practices

36

• Disk throughput and sizing

• Network bandwidth

• Data distribution and safety

Each of these factors can be the determining factor for sizing, although due to the in-memory nature of Couchbase Server,RAM is normally the most important factor. How you choose your primary factor will depend on the data set and informa-tion that you are storing:

• If you have a very small data set that gets a very high load, you'll need to size more off of network bandwidth thanRAM.

• If you have a very high write rate, you'll need more nodes to support the disk throughput of persisting all that data (andlikely more RAM to buffer the incoming writes).

• Even with a very small dataset, under low load, you may still want 3 nodes for proper distribution and safety.

With Couchbase Server, you can increase the capacity of your cluster (RAM, Disk, CPU or network) by increasing thenumber of nodes within your cluster, since each limit will be increased linearly as the cluster size is increased.

4.2.1. RAM Sizing

RAM is usually the most critical sizing parameter. It's also the one that can have the biggest impact on performance andstability.

4.2.1.1. Working Set

Before we can decide how much memory will we need for the cluster, we should understand the concept of a 'workingset'. The 'working set' at any point of time is the data that your application actively uses. Ideally you would want all yourworking set to live in memory.

4.2.1.2. Memory quota

It is very important that a Couchbase cluster is sized in accordance with the working set size and total data you expect.

The goal is to size the RAM available to Couchbase so that all your document IDs, the document ID meta data, along withthe working set values fit into memory in your cluster, just below the point at which Couchbase will start evicting valuesto disk (the High Water Mark).

How much memory and disk space per node you will need depends on several different variables, defined below.

Note

Calculations are per bucket

Calculations below are per bucket calculations. The calculations need to be summed up across allbuckets. If all your buckets have the same configuration, you can treat your total data as a singlebucket, there is no per-bucket overhead that needs to be considered.

Table 4.1. Input Variables

Variable Description

documentss_num The total number of documents you expect in your working set

ID_size The average size of document IDs

value_size The average size of values

number_of_replicas number of copies of the original data you want to keep

Best Practices

37

Variable Description

working_set_percentage The percentage of your data you want in memory.

per_node_ram_quota How much RAM can be assigned to Couchbase

The following are the items that are used in calculating memory required and are assumed to be constants.

Table 4.2. Constants

Constant Description

Meta data per document(metadata_per_document)

This is the space that Couchbase needs to keep metadata per document. It is 120bytes. All the documents and their metadata need to live in memory at all times

SSD or Spinning SSDs give better I/O performance.

headroom a Typically 25% (0.25) for SSD and 30% (0.30) for spinning (traditional) hard disksas SSD are faster than spinning disks.

High Water Mark(high_water_mark)

By default it is set at 70% of memory allocated to the node

a The headroom is the additional overhead required by the cluster to store metadata about the information being stored. This requires approximately 25-30%more space than the raw RAM requirements for your dataset.

This is a rough guideline to size your cluster:

Variable Calculation

no_of_copies 1 + number_of_replicas

total_metadata a (documents_num) * (metadata_per_document + ID_size) *(no_of_copies)

total_dataset (documentss_num) * (value_size) * (no_of_copies)

working_set total_dataset * (working_set_percentage)

Cluster RAM quota required (total_metadata + working_set) * (1 + headroom) /(high_water_mark)

number of nodes Cluster RAM quota required / per_node_ram_quotaa All the documents need to live in the memory

Note

You will need at least the number of replicas + 1 nodes irrespective of your data size.

Example sizing calculation

Table 4.3. Input Variables

Input Variable value

documentss_num 1000,000

ID_size 100

value_size 10000

number_of_replicas 1

working_set_percentage 20%

Table 4.4. Constants

Constants value

Type of Storage SSD

Best Practices

38

Constants value

overhead_percentage 25%

metadata_per_document 120

high_water_mark 70%

Table 4.5. Variable Calculations

Variable Calculation

no_of_copies = 2 a

total_metadata = 100,0000 * (100 + 120) * (2) = 440,000,000

total_dataset = 100,0000 * (10000) * (2) = 20,000,000,000

working_set = 20,000,000,000 * (0.2) = 4,000,000,000

Cluster RAM quota required = (440,000,000 + 4000,000,000) * (1+0.25)/(0.7) = 7928,000,000a 1 for original and 1 for replica

For example, if you have 8GB machines and you want to use 6 GB for Couchbase:

number of nodes = Cluster RAM quota required/per_node_ram_quota = 7.9 GB/6GB = 1.3 or 2 nodes

Note

RAM quota

You will not be able to allocate all your machine RAM to the per_node_ram_quota as there maybeother programs running on your machine.

4.2.2. Disk Throughput and Sizing

One of the big advantages that Membase provides is the decoupling of disk IO and RAM. This basic concept allows us toprovide extremely high scales at very low and consistent latencies. It also makes Membase capable of handling very highwrite loads without affecting the application's performance.

However, Membase still needs to be able to write data to disk and so your disks need to be capable of handling a steadystream of incoming data. It is important to analyze your application's write load and provide enough disk throughput tomatch. Information is written to disk through the disk write queue. The internal statistics system monitors the number ofoutstanding items in the disk write queue and can give you the information you need. The peak the disk write queue loadshows how many items stored in Couchbase Server would be lost in the event of a server failure.

It is up to your own internal requirements to decide how much vulnerability you are comfortable with and size the clusteraccordingly so that the disk write queue level remains low across the entire cluster. Adding more nodes will provide moredisk throughput.

Disk space is also required to persist data. How much disk space you should plan for is dependent on how your datagrows. You will also want to store backup data on the system. A good guideline is to plan for at least 130% of the total da-ta you expect. 100% of this is for data backup and 30% for overhead during file maintenance.

4.2.3. Network Bandwidth

Network bandwidth is not normally a significant factor in your calculations and preparations for cluster sizing, but net-work bandwidth is vital for accessing information from your cluster by clients, and for exchanging information betweennodes.

Best Practices

39

In general you can calculate your network bandwidth requirements using the formula:

Bandwidth = (operations per second * item size) + overhead for rebalancing

Where the operations per second can be calculated using:

Operations per second = Application reads + (Application writes * Replica copies)

4.2.4. Data Safety

To ensure data safety you need to ensure there are enough nodes within the cluster to support the safety requirements foryour data. This involves retaining a suitable number of nodes, and node configration, within your cluster. There are twoaspects to consider, the distribution of information across your nodes, and the number of replicas of information storedacross your cluster.

4.2.4.1. Data distribution

The basic idea is that more nodes are better than less. If you only have 2 nodes, your data will be split across the twonodes half, and half. This means that half of your dataset will be "impacted" if one goes away. On the other hand, with10 nodes, only 10% of the dataset will be "impacted" if one goes away. Even with Automatic Failover, there will still besome period of time when data is unavailable if nodes fail. This can be mitigated by having more nodes.

You also need to take into account the amount of extra load that the cluster will need to take on after a failover. Again,with only 2 nodes, each one needs to be ready to handle the entire load. With 10, each node only needs to be able to takeon an extra 10th of the workload should one fail.

While 2 nodes does provide a minimal level of redundancy, it is recommend always using at least 3 nodes.

4.2.4.2. Replication

Couchbase Server allows you to configure up to 3 replicas (creating 4 copies of the dataset). In the event of a failure, youcan only "failover" (either manually or automatically) as many nodes as you have replicas. For example:

• In a 5 node cluster with one replica, if one node goes down, you can fail it over. If a second node goes down, you nolonger have enough replica copies to fail over to and will have to go through a slower process to recover.

• In a 5 node cluster with 2 replicas, if one node goes down, you can fail it over. If a second node goes down, you can failit over as well. Should a 3rd one go down, you now no longer have replicas to fail over.

Note

After a node goes down and is failed over, you should attempt to replace that node as soon as pos-sible and rebalance. The rebalance is what will recreate the replica copies (if you still have enoughnodes to do so).

As a rule of thumb, it is recommended that you configure:

• 1 replica for up to 5 nodes

• 1 or 2 replicas for 5 to 10 nodes

• 1, 2 or 3 replicas for over 10 nodes

While there many be variations to this, there are definitely diminishing returns from having more replicas in smaller clus-ters.

Best Practices

40

4.2.4.3. Hardware Requirements

In general, Couchbase Server has very low hardware requirements and is designed to be run on commodity or virtualizedsystems. However, as a rough guide to the primary concerns for your servers:

• RAM: Your primary consideration as RAM is used to kep active items and is the key reason Couchbase Server has suchlow latency.

• CPU: Couchbase Server has very low CPU requirements. The server is multi-threaded and therefore benefits from amulti-core system. Machines with at least 4 or 8 physical cores are recommended.

• Disk: By decoupling the RAM from the IO layer, Couchbase Server can support low-performance disks better than oth-er databases. Known working configurations include SAN, SAS, SATA, SSD, and EBS, with the following recommen-dations:

• SSDs have been shown to provide a great performance boost both in terms of draining the write queue and also inrestoring data from disk (either on cold-boot or for purposes of rebalancing).

• RAID generally provides better throughput and reliability.

• Striping across EBS volumes (in Amazon EC2) has been shown to increase throughput.

• Network: Most configurations will work with Gigabit Ethernet interfaces. Faster solutions such as 10GBit and Inifini-band will provide spare capacity.

4.2.4.4. Considerations for Cloud environments (i.e. Amazon EC2)

Due to the unreliability and general lack of consistent IO performance in cloud environments, we highly recommend low-ering the per-node RAM footprint and increasing the number of nodes. This will give better disk throughput as well as im-prove rebalancing since each node will have to store (and therefore transmit) less data. By distributing the data further, itwill make the impact of losing a single node (which could be fairly common) even less.

Also, see the best practices for Section 4.6, “Using Couchbase in the Cloud”.

4.3. Deployment Considerations

• Restricted access to Moxi ports

Make sure that all the ports that Moxi uses are accessible only by trusted machines (including the other nodes in thecluster).

• Restricted access to web console (port 8091)

The web console is password protected. However, it is recommended that you restrict access to port 8091, as a abusercould do potentially harmful operations (like remove a node) from the web console.

• Node to Node communication on ports

All nodes in the cluster should be able to communicate with each other on 11210 and 8091.

• Swap configuration

Swap should be configured on the couchbase server, to avoid the operating system killing couchbase server if the sys-tem RAM is exhausted. Having swap provides more options on how to manage such a situation.

• Idle connection timeouts

Best Practices

41

Some firewall or proxy software will drop TCP connections which are idle for a certain amount of time (e.g., 20 min-utes). If the software does not allow changing that timeout, send a command from the client periodically to keep theconnection alive.

4.4. Ongoing Monitoring and MaintenanceImportant Statistics for Diagnosis

The 'watermark' determines when it is necessary to start freeing up available memory. Read more about this concept here.Some important statistics related to water marks are: High WaterMark (ep_mem_high_wat): The system will start eject-ing values out of memory when this watermark is met. Ejected values need to be fetched from disk, when accessed. LowWaterMark (ep_mem_low_wat): The system does not do anything when this watermark is reached but this is the 'goal' ofthe system when it starts ejecting data as a result of high watermark being met. Memory Used (mem_used):The currentsize of memory used.If mem_used hits the RAM quota then you will get OOM_ERROR. The mem_used must be less thanep_mem_high_wat which is the mark at which data is ejected from the disk. Disk Write Queue Size (ep_queue_size): Thesize of the queue that has data waiting to be written to the disk. Cache Hits (get_hits): The rule of thumb is that this shouldbe at least 90% of the total requests. Cache Misses (get_misses):

You can find values for these important stats with the following command: "/opt/couchbase/bin/ep_engine/manage-ment/stats <IP>:11210 all | egrep "todo|ep_queue_size|_eject|mem|max_data|hits|misses" will output: ep_flusher_todo:ep_max_data_size: ep_mem_high_wat: ep_mem_low_wat: ep_num_eject_failures: ep_num_value_ejects: ep_queue_size:mem_used: get_misses: get_hits:

Important UI Stats to watch

You can add the following graphs to watch on the Couchbase console. These graphs can be de/selected by clicking on the"Configure View" link at the top of the Bucket Details (Monitor->Data Buckets) page on the Couchbase console.

• Disk write queues - It should not keep growing. (the actual numbers will depend on your application and deployment )

• Ram ejections - no sudden spikes

• Vbucket errors (increasing value for vBucket errors is bad)

• OOM errors per sec (This should be 0)

• Temp OOM errors per sec (This should be 0)

• Connections count (This should remain flat in a long running deployment)

• Get hits per second

• Get misses per second (This should be much lower than Get hits per second)

Other monitoring

Make sure that you monitor disk space, CPU usage and swapping on all your nodes, using the standard monitoring tools.

Vacumming

Vacuuming reclaims disk space from sqlite by de-fragmenting the database. You should vacuum your sqlite files regularlyto free any space that is empty but unusable.

Post-rebalance

After the rebalancing operation itself is complete, the Couchbase cluster will start replicating any data that was moved. Inthe future we may include this replication process in the overall rebalancing itself, but for now you will have to be awarethat some data may not be replicated immediately following a rebalance.

Best Practices

42

It is a best practice to continue to monitor the system until you are confident that replication has completed. There are es-sentially two stages to replication:

• Backfilling - This is the first stage of replication and involves reading all data for a given active vBucket and send-ing it to the server that is responsible for the replica. This can put increased load on the disk subsystem as well asnetwork bandwidth but is not designed to impact any client activity. You can monitor the progress of this task bywatching for ongoing TAP disk fetches and/or watching 'stats tap': '/opt/couchbase/bin/ep_engine/management/stats<couchbase_node>:11210 tap | grep backfill' will return a list of TAP backfill processes and whether they are still run-ning (true) or done (false). When all have completed, you should see the Total Item count ( curr_items_tot _as opposedto the active item count, _curr_items) be equal to the number of active items times the replica count. If you are contin-uously adding data to the system, these values may not line up exactly at a given instant in time, but it should be clearfrom an order-of-magnitude sense whether your items are properly replicated. Until this is completed, you should avoidusing the "failover" functionality since that may result in loss of the data that has not been replicated yet.

• Draining - After the backfill process is completed, all nodes that had replicas materialized on them will need to also per-sist those items to disk. It is important to continue monitoring the disk write queue and memory usage of

4.5. Couchbase Behind a 2nd FirewallIf you are deploying Couchbase behind a 2nd firewall, what ports should you open? Of note, Couchbase should always bedeployed behind at least one firewall, where clients connections are allowed only from trusted web application servers. If,however, you have deployed Couchbase on an inner network behind a 2nd level firewall...

The ports Couchbase uses include: 11211, 11210, 4369, 8091 and the port range from 21100 to 21199.

More information...

• The server-side Moxi port is 11211. Pre-existing Couchbase and Memcached (non-smart) client libraries that are out-side the 2nd level firewall would just need port 11211 open to work.

• If you want to use the web admin console from outside the 2nd level firewall, also open up port 8091 (for REST/HTTPtraffic).

• If you're using smart clients or client-side Moxi from outside the 2nd level firewall, also open up port 11210 (in addi-tion to the above port 8091), so that the smart client libraries or client-side Moxi can directly connect to the data nodes.

Note

Server-side Couchbase nodes (aka, nodes joined into a Couchbase cluster) need all the above portsopen to work: 11211, 11210, 4369 (erlang), 8091, and the port range from 21100 to 21199 (erlang).

4.6. Using Couchbase in the CloudFor the purposes of this discussion, we will refer to "the cloud" as Amazon's EC2 environment since that is by far the mostcommon cloud-based environment. However, the same considerations apply to any environment that acts like EC2 (anorganization's private cloud for example). In terms of the software itself, we have done extensive testing within EC2 (andsome of our largest customers have already deployed Couchbase there for production use). Because of this, we have en-countered and resolved a variety of bugs only exposed by the sometimes unpredictable characteristics of this environment.

Being simply a software package, Couchbase Server is extremely easy to deploy in the cloud. From the software's per-spective, there is really no difference being installed on bare-metal or virtualized operating systems. On the other hand, themanagement and deployment characteristics of the cloud warrant a separate discussion on the best ways to use Couchbase.

We have written a number of RightScale templates to aid in your deployment within Amazon. You can very easily signup for a free RightScale account to try it out. The templates handle almost all of the special configuration needed to makeyour experience within EC2 successful. Direct integration with RightScale also allows us to do some pretty cool thingsaround auto-scaling and pre-packaged deployment. Check out the templates here Couchbase on RightScale

http://www.rightscale.com/

http://support.rightscale.com/27-Partners/Membase

Best Practices

43

We've also authored an AMI for use within EC2 independent of RightScale. When using these, you will have to handle thespecific complexities yourself. You can find this AMI by searching for 'couchbase' in Amazon's EC2 portal.

Some considerations to take into account when deploying within the cloud are:

• Local storage being ephemeral

• IP addresses of a server changing from runtime to runtime

• Security groups/firewall settings

• Swap Space

4.6.1. Local Storage

Dealing with the first point is not very much different than a data center deployment. However, EC2 provides an interest-ing solution. Through the use of EBS storage, you can obviate the largest concern of losing your data when an instancefails. Writing Couchbase data and configuration to EBS creates a reliable medium of storage. There is direct support forusing EBS within RightScale and of course you can set it up manually yourself.

Using EBS is definitely not required, but you should make sure to follow the best practices around performing backups.

Keep in mind that you will have to update the per-node disk path when configuring Couchbase to point to wherever youhave mounted an external volume.

4.6.2. IP addresses

The second issue is a bit trickier and requires configuring Couchbase to use a DNS entry instead of an IP address. By de-fault, Couchbase Servers use their IP address as a unique identifier. If the IP changes, an individual node will not be ableto identify its own configuration and other nodes that it was clustered to will not be able to access it. In order for a node toidentify itself via a DNS name rather than and IP address, the following instructions must be followed. Note that this con-figuration is automatically handled by the RightScale server template.

A few points to keep in mind when setting this up:

• Make sure that this hostname always resolves to the IP address of the node that it is on. This can be accomplished byusing a dynamic DNS service such as DNSMadeEasy which will allow you to automatically update the hostname whenan underlying IP address changes.

• It is best to make sure that the IP address registered with the hostname is the internal address for the node (rather thanthe external one provided by Amazon) so that other nodes and application machines can contact it

Warning

The below steps will completely wipe any data and configuration from the node, so it is best to startwith a fresh Couchbase install. If you already have a running cluster, you can easily rebalance a nodeout of the cluster, make the change and then rebalance it back into the cluster. Nodes with IP's andhostnames can exist in the same cluster.

For Linux:

1. Install the Couchbase software

2. Execute:

sudo /etc/init.d/couchbase-server stop

3. Edit the script located at /opt/couchbase/bin/couchbase-server

• Add a '\' to the end of the last line

Best Practices

44

• Add a new last line that reads:-name ns_1@<hostname>. Where hostname is either a DNS name or an IP ad-dress that you want this server to listen on (the 'ns_1@' prefix is mandatory). For example:

-detached \ -run ns_bootstrap -- \ -ns_server config_path "\"/opt/couchbase/etc/couchbase/static_config\"" \ -name [email protected]

4. Delete the files under:

• /opt/couchbase/var/lib/couchbase/data/*

• /opt/couchbase/var/lib/couchbase/mnesia/*

• /opt/couchbase/var/lib/couchbase/config/config.dat

5. Execute:

sudo /etc/init.d/couchbase-server start

6. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page (you will be tak-en back to the setup wizard since the configuration was cleared out, but after completing the wizard the node will benamed properly).

For Windows:

1. Install the Couchbase Server software

2. Stop the service by running:

C:\Program Files\Couchbase\Server\bin\service_stop.bat

3. Unregister the service by running:

C:\Program Files\Couchbase\Server\bin\service_unregister.bat

4. Edit the script located at C:\Program Files\Couchbase\Server\bin\service_register.bat:

• On the 7th line it says:set NS_NAME=ns_1@%IP_ADDR%

• Replace%IP_ADDR%with the hostname/IP address that you want to use/

5. Register the service by running the modified script: C:\Program Files\Couchbase\Server\bin\service_register.bat

6. Delete the files located under:C:\Program Files\Couchbase \Server\var\lib\couchbase\mnesia.

7. Start the service by running:

C:\Program Files\Couchbase\Server\bin\service_start.bat

8. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page (you will be tak-en back to the setup wizard since the configuration was cleared out, but after completing the wizard the node will benamed properly).

4.6.3. Security groups/firewall settings

It's important to make sure you have both allowed AND restricted access to the appropriate ports in a Couchbase deploy-ment. Nodes must be able to talk to one another on various ports and it is important to restrict external and/or internal ac-cess to only those individuals you want to have access. Unlike a typical data center deployment, cloud systems are open tothe world by default and steps must be taken to restrict access.

Best Practices

45

4.6.4. Swap Space

Certain cloud systems by default don't have a swap partition configured. While a system should not utilize a swap partitionheavily, it is our recommended practice to have at least some amount of swap configured to avoid the kernel from killingprocesses.

4.7. Deployment StrategiesBelow are a number of deployment strategies that you may want to employ when using Couchbase Server.

4.7.1. Using Couchbase Embedded Proxy

The first deployment option is to communicate with the embedded proxy in Couchbase over port 11211. It allows one toinstall Couchbase and begin using it with an existing application, via an on-the-client (OTC)memcached client library,without installing additional software.

The downside to this approach is that there are two network hops, so while the performance is still very good, it's not asgood as it could be with other deployment options. As shown in detail below, in a worst case scenario, server mappingwill happen twice (e.g. using ketama hashing to a server list on the OTC client, then using vBucket hashing and servermapping on the proxy) with an additional round trip network hop introduced.

Assume there is an existing application, with an OTC memcached client, with a server list of 3 servers (Servers A, B andC). Couchbase is installed in place of the memcached server software on each of these three servers. As shown in thefigure above, when the application wants to Get(ID), it will call a function in the OTC client library. Client library willhash(ID) and be directed, based on the server list and hashing function to Server C.

The Get operation is sent to Server C, port 11211. When it arrives to couchbase (now a proxy port), the document ID ishashed again to determine its vBucket and Server mapping. This time, the result is Server A. The proxy will contact Serv-er A on port 11210, perform the get operation and return the result to the client.

Figure 4.1. Deployment Strategy: Using the Embedded Proxy

4.7.2. Standalone Proxy

The second option is to deploy a standalone proxy, which performs substantially the same way as the embedded proxy, butpotentially eliminating a network hop. A standalone proxy deployed on a client may also be able to provide valuable ser-vices, such as connection pooling. The diagram below shows the flow with a standalone proxy installed on the applicationserver.

Best Practices

46

The memcached OTC client is configured to have just one server in its server list (localhost), so all operations are for-warded to localhost:11211 — a port serviced by the proxy. The proxy hashes the document ID to a vBucket, looksup the host server in the vBucket table, and then sends the operation to the appropriate couchbase server on port 11210.

Figure 4.2. Deployment Strategy: Standalone Proxy

4.7.3. Using a vBucket aware Client

In the final case, no proxy is installed anywhere in the data flow. The client has been updated and performs server selec-tion directly via the vBucket mechanism. In addition, these clients could send additional information using a modified on-the-wire couchbase protocol, for example to explicitly encode the destination vBucket. This data could be used to optimizesystem performance.

Figure 4.3. Deployment Strategy: Using a vBucket Aware Client

47

Chapter 5. Administration Tasks

This sections provides background information on specific administration tasks that you may need to perform with Couch-base Server.

5.1. Failover with Couchbase

In Couchbase, failing over a server (and thus, transferring load to replica servers for the document IDs that were previous-ly "mastered" on that server) can be done either manually, automatically, or programmatically.

Failover within Couchbase is the process that immediately activates any replica vBuckets for the vBuckets that were ac-tive on the node being failed over.

Ideally, you should only fail over a node that is down and also only if you know you have replicas available elsewhere inthe cluster:

If a node is not down, you should attempt to use the remove/rebalance functionality as this will ensure all the data is re-moved, re-replicated and maintained safe. See Section 5.3.3, “Rebalancing”. Failing over a live node may introduce asmall data-loss window as any data "in flight" (meaning not yet replicated) will be lost when the failover takes place. Ifyou don't have available replicas, failing over the node will result in perceived data loss as those vBuckets will be activat-ed as empty elsewhere in the cluster. You can still recover the data, but it will not be immediately available.

5.1.1. Automatic Failover

Due to a number of possible bad situations (Failover with Couchbase), we have placed a number of restrictions on the fea-ture:

• Automatic failover is off by default. We still maintain that the best practice would be to have an external system (ei-ther human or automated) monitoring the Couchbase cluster to prevent things like network partitions from causing moreharm than good.

• Automatic failover is only available on clusters of at least 3 nodes. Also to prevent a network partition from causingboth sides to fail each other over.

• The Automatic failover feature will only fail over 1 node before requiring administrative interaction. This is to prevent acascading failure from taking the cluster completely out of operation.

There is a minimum 30 second delay before a node will be failed over. This can be raised, but the software is hard codedto perform multiple "pings" of a node that is perceived down. This is to prevent a slow node or flaky network connectionfrom being failed-over inappropriately.

If there are any node failures, an email can be configured to be sent out both when an automatic failover occurs, and whenit doesn't.

To configure the feature, see Section 6.6.2, “Auto-Failover Settings”.

5.1.1.1. Resetting the Auto failover counter

After a node has been automatically failed over, the administrator must reset the counter in order for the autofailover fea-ture to work again.

This should only be done after restoring the cluster to a healthy and balanced state.

Administration Tasks

48

5.1.2. Automated failover considerations

Automatically failing components in a distributed system can be perilous. There are countless examples of high-profile ap-plications that have taken themselves offline through unchecked automated failover strategies. Some of the situations thatmight lead to pathological behavior include:

• Scenario 1 - Thundering herd

Imagine a scenario where a Couchbase cluster of 5 nodes is being run at 80-90% aggregate capacity in terms of net-work load. Everything is running well, though at the limit. Now a node fails and the software decides to automaticallyfailover that node. It is unlikely that all of the remaining 4 nodes would be able to handle the additional load successful-ly, which could lead to another node being automatically failed over. These failures can cascade leading to the eventualloss of the entire cluster. Clearly having 1/5th of the requests not being serviced would be more desirable than none ofthe requests being serviced.

As the data infrastructure architect of a large social networking site once said (paraphrased): "If half your servers aredown and you're still serving half your users, you're doing really well."

The solution in this case would be to live with the single node failure, add a new server to the cluster, mark the failednode for removal and then rebalance. This way there is a brief partial outage, rather than an entire cluster being downed.Ensure there is excess capacity to handle node failures with replicas taking over is another solution to this problem.

• Situation 2 - Network partition

If a network partition occurs within a Couchbase cluster, automatic failover would lead both sides to decide that theyare going to automatically failover each other. Each portion would now assume responsibility for the entire documentID space, so whilst there is consistency for a document ID within the partial clusters, there would start to be inconsis-tency of data between the two partial clusters. Reconciling those differences may be difficult, depending on the natureof your data and your access patterns.

Assuming one of the two partial clusters is large enough to cope with all traffic, the solution would be to direct all traf-fic for the cluster to that single partial cluster and then later add-in the previously in-accessible machines to restore theoriginal cluster size.

• Situation 3 - Misbehaving node

If one node loses connectivity to the cluster (or "thinks" that it has lost connectivity to the cluster) allowing it to auto-matically failover the rest of the cluster would lead to that node being a cluster-of-one. As a result the same partition sit-uation as described above arises again.

In this case the solution is to take down the node that has connectivity issues and let the rest of the cluster handle theload (assuming there is spare capacity available).

5.1.3. Monitored failover

The lesson is that automatically failing over a node can lead to problems, if the cause of the failure and the load that willbe placed on the remaining system is not well understood. The best solution is to use monitoring to drive the failover deci-sion. Monitoring can take two forms: human or external system.

Human intervention. One option is to have a human operator respond to alerts and make a decision on what to do. Humansare uniquely capable of considering a wide range of data, observations and experience to best resolve a situation. Many or-ganizations disallow automated failover without human consideration of the implications. But that's not always a feasiblesolution for companies (large or small).

External monitoring. Another option is to have a system monitoring the cluster via our Management REST API. Suchan external system is in the best position to order the failover of nodes because it can take into account system compo-


49

nents that are outside the scope of Couchbase visibility. For example, by observing that a network switch is flaking andthat there is a dependency on that switch by the Couchbase cluster, the management system may determine that failing theCouchbase nodes will not help the situation. If, however, everything around Couchbase and across the various nodes ishealthy and that it does indeed look like a single node problem, and that the aggregate traffic can support loading the re-maining nodes with all traffic, then the management system may fail the system over. Couchbase fully supports this modelthrough its REST interface.

5.2. Backup and Restore with CouchbaseCouchbase stores the persisted data in an sqlite3database. Each database is split into four shards with metadata in adatabase with the bucket's configured name. For example, a bucket named 'default' has the following five files:

defaultdefault-0.mbdefault-1.mbdefault-2.mbdefault-3.mb

The files are stored by default in /var/opt/couchbase/<version>/data/ns_1. This path can be changed on aper-node basis when initially setting up the node: Chapter 2, Getting Started

When backing up or restoring a bucket, all files must be included.

Warning

Make sure that any restoration of files also sets the proper ownership of those files to the couchbaseuser

5.2.1. Backup

To take a backup of a running Couchbase node (note, this needs to be performed on all buckets and on all nodes of a clus-ter to take a backup of that cluster):

Use the mbbackup script to backup data:

mbbackup [bucket_path_name] [dest_dir_path].

Make sure that there is enough disk space to take the backup.

The mbbackup script will also perform a vacuum of the database files to defragment them which provides faster start-up times. Depending on the amount of data, this script can take an extended amount of time to run. It is a best practice tomake sure that your connection to the server running the script is not broken.

• Linux

shell> mbbackup /var/opt/couchbase/1.6.4/data/ns_1/default /backups/2010-12-22/

Backup the configuration file located at: /etc/opt/couchbase/<version>/ns_1/config.dat

• Windows

• Open a command line. Click on Start ->All Programs-> Accessories-> Command Prompt.

• Start powershell by typing powershell and press Enter key.

• Type:

shell> set-executionpolicy remotesigned

Press the Enter key.


50

shell> mbbackup "C:/Program Files/Couchbase/Server/data/ns_1/default" \ "C:/backup/2010-12-22/"

Copy file config.dat located at C:\Program Files\Couchbase\Server\con-fig\ns_1\config.dat.

5.2.2. Restoring a Backup

Note

It is a best practice to backup and restore all nodes together to minimize any inconsistencies in data.Couchbase is always per-item consistent, but does not guarantee total cluster consistency or in-orderpersistence.

5.2.2.1. Restoring to a Previous State

There are two options for restoring a Couchbase Server cluster:

5.2.2.1.1. Restoring to the same cluster

Assumptions:

• The cluster must contain the same number of nodes and those nodes must have the same IP addresses (or hostnames ifyou followed the instructions listed here)

• You must restore all of the config.dat files as well as all of the database files to their original locations

1. Stop the Couchbase Server service on all nodes

2. Replace the database and configuration files

3. Restart the service.

This must be performed on each node within the entire cluster.

5.2.2.1.2. Restoring to a different cluster

You can use this feature to migrate an entire cluster into new set of machines. This is particularly useful when:

• In cloud environments, where the IP addresses of nodes will have changed

• To create dev/test clusters with the same data as the production cluster

Assumptions:

• The two clusters must have the same topology (number of nodes)

• The original cluster must be in a healthy state. This means that all nodes should be up and running and no rebalance orfailover operation should be running.

• It is a best practice for both clusters to be of the same OS and memory configuration

Instructions:

1. Take a backup of the data files of all nodes using the above procedure. Alternately, shut down the couchbase-server onall nodes and copy the DB files.

2. Install Couchbase Server (of at least version 1.7.1) on new nodes and cluster together. If using the web console to setupyour cluster, a 'default' bucket will be created. Please delete this bucket before proceeding.


51

3. Place the copies of the original files into the data directory on all the new nodes.

Warning

You do not have to "match" up the nodes one-for-one. However, ensure that each set of originaldata files gets placed onto one and only one node of the new cluster

Note

Please ensure that you retain file ownership properties for those files which you placed on the des-tination node.

4. Start couchbase-server on the new nodes

5. Create a bucket with the same name and SASL configuration on the new nodes.

6. After the bucket creation, each node will start loading items from the data files into memory.

7. The cluster will be in a balanced state after warm up.

8. Do not start a rebalance process while nodes are still warming up.

9. If any nodes go down during the warmup, it is a best practice to restart all nodes together.

5.2.2.2. Restoring using mbrestore tool

Warning

There are a number of bugs in older versions of the mbrestore script. Anyone using mbrestoreshould make sure to get the latest script to ensure proper functionality. You can download the latestfrom here. The latest version of the script WILL work with any previous versions of Couchbase.

If you are unable to meet the above requirements for restoring a cluster, you can use the mbrestore tool (included in ver-sions 1.6.4 and higher). This script will read the documents from a set of database files and send them as client commandsinto a bucket of your choosing.

This is useful if:

• You want to restore data into a cluster of a different size

• You want to transfer/restore data into a different bucket

• You have a broken or corrupted database file (usually from running out of space on a disk drive)

The mbrestore tool provides the following options:

%mbrestore opts db_files (use -h for detailed help)

-a --add Use add instead of set to avoid overwriting existing items -H --host Hostname of moxi server to connect to (default is 127.0.0.1) -p --port Port of moxi server to connect to (default is 11211) -t --threads Number of worker threads -u --username Username to authenticate with (this is the name of the bucket you are sending data into) -P --password Password to authenticate with (this is the password of the bucket you are sending data into)

Depending on the amount of data, this script can take an extended amount of time to run. It is a best practice to make surethat your connection to the server running the script is not broken, or that you are using something to let the script run inthe background (i.e. screen) Linux:

/opt/couchbase/bin/mbrestore -a default default-0.mb default-1.mb default-2.mb default-3.mb

https://github.com/couchbase/ep-engine/blob/master/management/mbrestore


52

In order to correctly restore you must put all of the database backup file names as command arguments Windows:

cd "C:/Program Files/Couchbase/Server/bin/".\mbrestore -a "C:/backup/2010-12-22/default" "C:/backup/2010-12-22/default-0.mb" \ "C:/backup/2010-12-22/default-1.mb" "C:/backup/2010-12-22/default-2.mb" "C:/backup/2010-12-22/default-3.mb"

In order to correctly restore you must put all of the database backup file names as command arguments.

5.3. Expanding and Shrinking your Cluster (Rebalancing)

Over the lifetime of your Couchbase cluster, there will very likely be cause to add or remove nodes. This can be based up-on a variety of factors including RAM/disk capacity and available network bandwidth. The process of adding or remov-ing nodes from your Couchbase cluster consists of a rebalancing process which at its core is meant to provide continuousavailability of data.

The same process can be used to perform software upgrades and hardware refreshes by removing a node, making changesand then rebalancing it back into the cluster. If possible, for these operations it is a best practice to add the additionalnodes first before removing nodes.

5.3.1. Growing your cluster

There are a few metrics off of which to base your decision to grow a Couchbase cluster:

1. Increasing RAM capacity. Arguably the single most important component in a Couchbase cluster is the amount ofavailable RAM. While filling up available RAM is not necessarily cause to grow a cluster, it should certainly be taken intoconsideration.

• As you see more disk fetches occurring (also evidenced by a growing "cache miss ratio"), that means that your applica-tion is requesting more and more data from disk that is not available in RAM. Increasing the RAM in a cluster will al-low it to store more data and therefore provide better performance to your application.

• If you want to add more buckets to your Couchbase cluster you will likely need more RAM to do so. Adding nodes willincrease the overall capacity of the system and then you can shrink any existing buckets in order to make room for newones.

• Increasing disk IO throughput. - By adding nodes to a Couchbase cluster, you will increase the aggregate amount ofdisk IO that can be performed. This is especially important in high-write environments, but can also be a factor whenyou need to read large amounts of data from the disk.

• Increasing disk capacity. - This one should go without saying. You can either add more disk space to your current nodesor add more nodes to add aggregate disk space to the cluster.

• Network bandwidth. If you see that you are or are close to saturating the network bandwidth your cluster, this is a verystrong indicator of the need for more nodes. More nodes will cause the network bandwidth to be spread out furtherwhich will reduce the bandwidth of each node individually.

5.3.2. Shrinking your cluster

Choosing to shrink a Couchbase cluster is a more subjective decision. It is usually based upon cost considerations and/ornot needing as large a cluster to support the load requirements of your application.

When deciding to shrink a cluster, it is extremely important to ensure you have enough capacity in the remaining nodes tosupport your dataset as well as your application load.

It is also not recommended to remove multiple nodes at once, rather one at a time to understand the impact on the clusteras a whole.


53

It is recommended that you remove a node rather than fail it over. When a node fails and is not coming back to the cluster,the failover functionality will promote its replica vBuckets become active immediately. If a healthy node is failed over,there might be some data loss for the replication data that was in flight during that operation. Using the remove functional-ity will ensure that all data is properly replicated and continuously available.

5.3.3. Rebalancing

Once you decide to add or remove nodes to your Couchbase cluster, there are a few things to take into consideration:

• If you're planning on adding and/or removing multiple nodes in a short period of time, it is best to add them all at onceand then kick-off the rebalancing operation rather than rebalance after each addition. This will reduce the overall loadplaced on the system as well as the amount of data that needs to be moved.

• Choose a quiet time for adding nodes. While the rebalancing operation is meant to be performed online, it is not a "free"operation and will undoubtedly put increased load on the system as a whole in the form of disk IO, network bandwidth,CPU resources and RAM usage.

• It is our recommended best practice to do any "voluntary" (i.e., not to resolve a failure) rebalancing operation during aperiod of low usage of the system. Obviously with today's 24x7 web applications there may not be a period of completeinactivity but it is up to the administrator to understand the impact that a rebalancing operation may have on the clusterand application.

• Memory required for rebalancing. The rebalancing operation requires moving large amounts of data around the cluster.The more RAM that is available will allow the operating system to cache more disk access which will allow it to per-form the rebalancing operation much faster. If there is not enough memory in your cluster the rebalancing maybe quiteslow. It is recommended that you don't wait for your cluster to reach full capacity before adding capacity and rebalanc-ing.

54

Chapter 6. Administration Web Console

The Couchbase Web Console is the main administration method for managing your Couchbase installation. The WebConsole provides the following functionality:

• Cluster Overview provides a quick guide to the status of your Couchbase cluster.

For more information, read Section 6.1, “Cluster Overview”.

• Data Buckets provides access to your data bucket configuration, including creating new buckets, editing existing con-figurations, and provided detailed analysis and statistics on the bucket activity.

SeeSection 6.3, “Data Buckets”.

• Server Nodes details your active nodes and their configuration and activity. You can also fail over nodes and removethem from your cluster, and view server-specific performance and monitoring statistics.

ReadSection 6.4, “Server Nodes”.

• Log access allows you to view errors and problems.

SeeSection 6.7, “Log”for more information.

• Settings configures the console and cluster settings.

SeeSection 6.6, “Settings”for more information.

In addition to the navigable sections of the Couchbase Web Console, there are additional systems within the web console,including:

• Update Notifications

Update notifications warn you when there is an update available for the installed Couchbase Server. See Section 6.8,“Update Notifications” for more information on this feature.

• Warnings and Alerts

The warnings and alerts system will notify you through the web console where there is an issue that needs to be ad-dressed within your cluster. The warnings and alerts can be configured through the Section 6.6, “Settings”.

For more information on the warnings and alerts, see Section 6.9, “Warnings and Alerts”.

6.1. Cluster Overview

The Cluster Overview page is the home page for the Couchbase Web Console. The page is designed to give you a quickoverview of your cluster health, including RAM and disk usage and activity.

Administration Web Console

55

Figure 6.1. Couchbase Web Console - Cluster Overview

The page is divided into Cluster, Buckets, and Servers sections.

6.1.1. Cluster

The Cluster section provides information on the RAM and disk usage information for your cluster.

For the RAM information you are provided with a graphical representation of your RAM situation, including:

• Total in Cluster

Total RAM configured within the cluster. This is the total amount of memory configured for all the servers within thecluster.

• Total Allocated

The amount of RAM allocated to data buckets within your cluster.


56

• Unallocated

The amount of RAM not allocated to data buckets within your cluster.

• In Use

The amount of memory across all buckets that is actually in use (i.e. data is actively being stored).

• Unused

The amount of memory that is unused (available) for storing data.

The Disk Overview section provides similar summary information for disk storage space across your cluster.

• Total Cluster Storage

Total amount of disk storage available across your entire cluster for storing data.

• Usable Free Space

The amount of usable space for storing information on disk. This figure shows the amount of space available on theconfigured path after non-Couchbase files have been taken into account.

• Other Data

The quantity of disk space in use by data other than Couchbase information.

• In Use

The amount of disk space being used to actively store information on disk.

• Free

The free space available for storing objects on disk.

6.1.2. Buckets

The Buckets section provides two graphs showing the Operations per second and Disk fetches per second.

The Operations per second provides information on the level of activity on the cluster in terms of storing or retrieving ob-jects from the data store.

The Disk fetches per second indicates how frequently Couchbase is having to go to disk to retrieve information instead ofusing the information stored in RAM.

6.1.3. Servers

The Servers section indicates overall server information for the cluster:

• Active Servers is the number of active servers within the current cluster configuration.

• Servers Failed Over is the number of servers that have failed over due to an issue that should be investigated.

• Servers Down shows the number of servers that are down and not-contactable.

• Servers Pending Rebalance shows the number of servers that are currently waiting to be rebalanced after joining a clus-ter or being reactivated after failover.


57

6.2. Web Console Statistics

Within the Data Bucket monitor display, information is shown by default for the entire Couchbase Server cluster. The in-formation is aggregated from all the server nodes within the configured cluster for the selected bucket.

The standard Data Buckets monitor screen is shown below. Each individual graph is updated in real-time according to theselected time interval.

The following functionality is available through this display, and is common to all the graphs and statistics display withinthe web console.

• Bucket Selection

The Data Buckets selection list allows you to select which of the buckets configured on your cluster is to be used as thebasis for the graph display.

• Server Selection

The Server Selection option enables you to enable the display for an individual server instead of for the entire cluster.You can select an individual node, which displays the Section 6.4, “Server Nodes” for that node. Selecting All ServerNodes shows the Section 6.3, “Data Buckets” page.

• Interval Selection

The Interval Selection at the top of the main graph changes interval display for all graphs displayed on the page. For ex-ample, selecting Minute shows information for the last minute, continuously updating.

Note

As the selected interval increases, the amount of statistical data displayed will depend on how longyour cluster has been running.

• Statistic Selection

All of the graphs within the display update simultaneously. Clicking on any of the smaller graphs will promote thatgraph to be displayed as the main graph for the page.

• Individual Server Selection

Clicking the blue triangle next to any of the smaller statistics graphs enables you to view the statistics on each individ-ual server, instead of the default view which shows the entire cluster information for a given statistic.

6.3. Data Buckets

Couchbase Server incorporates a range of statistics and user interface available through the Data Buckets and ServerNodes that shows overview and detailed information so that administrators can better understand the current state of indi-vidual nodes and the cluster as a whole.

The Data Buckets page displayed a list of all the configured buckets on your system (of both Membase and memcachedtypes). The page provides a quick overview of your cluster health from the perspective of the configured buckets, ratherthan whole cluster or individual servers.

The information is shown in the form of a table, as seen in the figure below.


58

Figure 6.2. Couchbase Web Console - Data Buckets Overview

For each bucket, the following information is provided in each column:

• Bucket type is indicated by the displayed icon. A couch icon indicates a Membase bucket. The M icon indicates a mem-cached bucket.

• Bucket name is the given name for the bucket. Clicking on the bucket name takes you to the individual bucket statisticspage. For more information, see Section 6.3.3, “Individual Bucket Monitoring”.

• RAM Usage/Quota shows the amount of RAM used (for active objects) against the configure bucket size.

• Disk Usage shows the amount of disk space in use for active object data storage.

• Item Count indicates the number of objects stored in the bucket.

• Ops/sec shows the number of operations per second for this data bucket.

• Disk Fetches/sec shows the number of operations required to fetch items from disk.

• Clicking the Information button opens the basic bucket information summary. For more information, see Section 6.3.2,“Bucket Information”.

To create a new data bucket, click the Create New Data Bucket. See Section 6.3.1, “Creating and Editing Data Buckets”for details on creating new data buckets.

6.3.1. Creating and Editing Data Buckets

When creating a new data bucket, or editing an existing one, you will be presented with the bucket configuration screen.From here you can set the memory size, access control and other settings, depending on whether you are editing or creat-ing a new bucket, and the bucket type.

6.3.1.1. Creating a new Bucket

When creating a new bucket, you are presented with the Create Bucket dialog, as shown in the figure below.


59

Figure 6.3. Couchbase Web Console - Create Bucket

• Bucket Name

The bucket name. The bucket name can only contain characters in range A-Z, a-z, 0-9 as well as underscore, period,dash and percent symbols.

• Bucket Type

Specifies the type of the bucket to be created, either Memcached or Membase.

• Access Control

The access control configures the port your clients will use to communicate with the data bucket, and whether the buck-et requires a password.


60

To use the TCP standard port (11211), the first bucket you create can use this port without requiring SASL authentica-tion. For each subsequent bucket, you must specify the password to be used for SASL authentication, and client com-munication must be made using the binary protocol.

To use a dedicated port, select the dedicate port radio button and enter the port number you want to use. Using a dedi-cated port supports both the text and binary client protocols, and does not require authentication.

• Memory Size

This option specifies the amount of available RAM configured on this server which should be allocated to the defaultbucket. Note that the allocation is the amount of memory that will be allocated for this bucket on each node, not the to-tal size of the bucket across all nodes.

• Replication

For Membase buckets you can enable replication to support multiple replicas of the default bucket across the serverswithin the cluster. You can configure up to three replicas. Each replica receives copies of all the documents that aremanaged by the bucket. If the host machine for a bucket fails, a replica can be promoted to take its place, providing con-tinuous (high-availability) cluster operations in spite of machine failure.

You can disable replication by setting the number of replica copies to zero (0). This will configure the default bucket aslocal-only and therefore only accessible on this machine.

Once you selected the options for the new bucket, you can click Create button to create and activate the button within yourcluster. You can cancel the bucket creation using the Cancel button.

6.3.1.2. Editing Couchbase Buckets

You can edit a limited number of settings for an existing Couchbase bucket:

• Access Control, including the standard port/password or custom port settings.

• Memory Size can be modified providing you have unallocated space within your Cluster configuration. You can reducethe amount of memory allocated to a bucket if that space is not already in use.

The bucket name cannot be modified.

To delete the configured bucket entirely, click the Delete button.

6.3.1.3. Editing Memcached Buckets

For Memcached buckets, you can modify the following settings when editing an existing bucket:

• Access Control, including the standard port/password or custom port settings.

• Memory Size can be modified providing you have unallocated space within your Cluster configuration. You can reducethe amount of memory allocated to a bucket if that space is not already in use.

You can delete the bucket entirely by clicking the Delete button.

You can empty a Memcached bucket of all the cached information that it stores by using the Flush button.

Warning

Using the Flush button removes all the objects stored in the Memcached bucket. Using this button onactive Memcached buckets may delete important information.


61

6.3.2. Bucket Information

You can obtain basic information about the status of your data buckets by clicking on the ibutton within the Data Bucketspage. The bucket information shows memory size, access, and replica information foe the bucket, as shown in the figurebelow.

Figure 6.4. Couchbase Web Console - Bucket Information

You can edit the bucket information by clicking the Edit button within the bucket information display.

6.3.3. Individual Bucket Monitoring

Bucket monitoring within the Couchbase Web Console has been updated to show additional detailed information. The fol-lowing statistic groups are available for Couchbase bucket types.

• Summary


62

The summary section provides a quick overview of the cluster activity. For more information, see Section 6.3.3.1,“Bucket Monitoring — Summary Statistics”.

• vBucket Resources

This section provides detailed information on the vBucket resources across the cluster, including the active, replica andpending operations. For more information, see Section 6.3.3.2, “Bucket Monitoring — vBucket Resources”.

• Disk Queues

Disk queues show the activity on the backend disk storage used for persistence within a data bucket. The informationdisplayed shows the active, replica and pending activity. For more information, see Section 6.3.3.3, “Bucket Monitoring— Disk Queues”.

• TAP Queues

The TAP queues section provides information on the activity within the TAP queues across replication, rebalancing andclient activity. For more information, see Section 6.3.3.4, “Bucket Monitoring — TAP Queues”.

• Top Keys

This shows a list of the top 10 most actively used keys within the selected data bucket.

For Memcached bucket types, the Memcached static summary is provided. See Section 6.3.3.5, “Bucket Monitoring —Memcached Buckets”.

6.3.3.1. Bucket Monitoring — Summary Statistics

The summary section is designed to provide a quick overview of the cluster activity. Each graph (or selected graph) showsinformation based on the currently selected bucket.

Figure 6.5. Couchbase Web Console - Summary Statistics

The following graph types are available:

• ops per second

The total number of operations per second on this bucket.

• cache miss %


63

Percentage of reads per second to this bucket which required a read from disk rather than RAM.

• creates per second

Number of new items created in this bucket per second.

• updates per second

Number of existing items updated in this bucket per second.

• disk reads

Number of reads per second from disk for this bucket.

• back-offs per second

Number of 'back-offs' sent per second to clients due to out of memory situations for this bucket.

6.3.3.2. Bucket Monitoring — vBucket Resources

The vBucket statistics provide information for all vBucket types within the cluster across three different states. Within thestatistic display the table of statics is organized in four columns, showing the Active, Replica and Pending states for eachindividual statistic. The final column provides the total value for each statistic.

Figure 6.6. Couchbase Web Console - vBucket Resources statistics


64

The Active column displays the information for vBuckets within the Active state. The Replica column displays the sta-tistics for vBuckets within the Replica state (i.e. currently being replicated). The Pending columns shows statistics forvBuckets in the Pending state, i.e. while data is being exchanged during rebalancing.

These states are shared across all the following statistics. For example, the graph news items per sec within the Activestate column displays the number of new items per second created within the vBuckets that are in the active state.

The individual statistics, one for each state, shown are:

• active vBuckets

The number of vBuckets within the specified state.

• active items

Number of items within the vBucket of the specified state.

• % resident items

Percentage of items within the vBuckets of the specified state that are resident (in RAM).

• new items per second

Number of new items created in vBuckets within the specified state. Note that new items per second is not valid for thePending state.

• ejections per second

Number of items ejected per second within the vBuckets of the specified state.

• user data in RAM

Size of user data within vBuckets of the specified state that are resident in RAM.

• metadata in RAM

Size of item metadata within the vBuckets of the specified state that are resident in RAM.

6.3.3.3. Bucket Monitoring — Disk Queues

The Disk Queues statistics section displays the information for data being placed into the disk queue. Disk queues are usedwithin Couchbase Server to store the information written to RAM on disk for persistence. Information is displayed foreach of the disk queue states, Active, Replica and Pending.


65

Figure 6.7. Couchbase Web Console - Disk Queue Statistics

The Active column displays the information for the Disk Queues within the Active state. The Replica column displays thestatistics for the Disk Queues within the Replica state (i.e. currently being replicated). The Pending columns shows statis-tics for the disk Queues in the Pending state, i.e. while data is being exchanged during rebalancing.

These states are shared across all the following statistics. For example, the graph fill rate within the Replica state columndisplays the number of items being put into the replica disk queue for the selected bucket.

The displayed statistics are:

• items

The number of items waiting to be written to disk for this bucket for this state.

• fill rate

The number of items per second being added to the disk queue for the corresponding state.

• drain rate

Number of items actually written to disk from the disk queue for the corresponding state.

• average age

The average age of items (in seconds) within the disk queue for the specified state.

6.3.3.4. Bucket Monitoring — TAP Queues

The TAP queues statistics are designed to show information about the TAP queue activity, both internally, between clusternodes and clients. The statistics information is therefore organized as a table with columns showing the statistics for TAPqueues used for replication, rebalancing and clients.


66

Figure 6.8. Couchbase Web Console - TAP Queue Statistics

The statistics in this section are detailed below:

• # tap senders

Number of TAP queues in this bucket for internal (replica), rebalancing or client connections.

• # items

Number of items in the corresponding TAP queue for this bucket.

• fill rate

Number of items per second being put into the corresponding TAP queue for this bucket.

• drain rate

Number of items per second being sent over the corresponding TAP queue connections to this bucket.

• back-off rate

Number of back-offs per second received when sending data through the corresponding TAP connection to this bucket.

• # backfill remaining

Number of items in the backfill queue for the corresponding TAP connection for this bucket.

• # remaining on disk

Number of items still on disk that need to be loaded to service the TAP connection to this bucket.


67

6.3.3.5. Bucket Monitoring — Memcached Buckets

For Memcached buckets a separate suite of Memcached-specific statistics are displayed.

Figure 6.9. Couchbase Web Console - Memcached Statistics

The Memcached statistics are:

• Operations per sec.

Total operations per second serviced by this bucket

• Hit Ratio %

Percentage of get requests served with data from this bucket

• Memory bytes used

Total amount of RAM used by this bucket

• Items count

Number of items stored in this bucket

• RAM evictions per sec.

Number of items per second evicted from this bucket

• Sets per sec.

Number of set operations serviced by this bucket

• Gets per sec.

Number of get operations serviced by this bucket


68

• Net. bytes TX per sec

Number of bytes per second sent from this bucket

• Net. bytes RX per sec.

Number of bytes per second sent into this bucket

• Get hits per sec.

Number of get operations per second for data that this bucket contains

• Delete hits per sec.

Number of delete operations per second for data that this bucket contains

• Incr hits per sec.

Number of increment operations per second for data that this bucket contains

• Decr hits per sec.

Number of decrement operations per second for data that this bucket contains

• Delete misses per sec.

Number of delete operations per second for data that this bucket does not contain

• Decr misses per sec.

Number of decr operations per second for data that this bucket does not contain

• Get Misses per sec.

Number of get operations per second for data that this bucket does not contain

• Incr misses per sec.

Number of increment operations per second for data that this bucket does not contain

• CAS hits per sec.

Number of CAS operations per second for data that this bucket contains

• CAS badval per sec.

Number of CAS operations per second using an incorrect CAS ID for data that this bucket contains

• CAS misses per sec.

Number of CAS operations per second for data that this bucket does not contain

6.4. Server Nodes

In addition to monitoring buckets over all the nodes within the cluster, Couchbase Server also includes support for moni-toring the statistics for an individual node.


69

The Server Nodes monitoring overview shows summary data for the Swap Usage, RAM Usage, CPU Usage and ActiveItems across all the nodes in your cluster.

Figure 6.10. Couchbase Web Console - Server Nodes

Clicking the server name provides server node specific information, including the IP address, OS, Couchbase version andMemory and Disk allocation information.

Selecting a server from the list shows a server-specific version of the Bucket Monitoring overview, showing a combina-tion of the server-specific performance information, and the overall statistic information for the bucket across all nodes.


70

Figure 6.11. Couchbase Web Console- Data Bucket/Server view

The graphs specific to the server are:

• swap usage

Amount of swap space in use on this server.

• free memory


71

Amount of RAM available on this server.

• CPU utilization

Percentage of CPU utilized across all cores on the selected server.

• connection count

Number of connects to this server of all types for client, proxy, TAP requests and internal statistics.

You can select an individual bucket and server to view a statistic for using the popup selections for the server and bucket,and clicking on the mini-graph for a given statistic.

Figure 6.12. Couchbase Web Console - Server specific view

For more information on the data bucket statistics, see Section 6.3, “Data Buckets”.

6.5. Couchbase Server States

Couchbase Server nodes can be in a number of different states depending on their current activity and availability. Thedisplayed states are:

• Up

Host is up, replicating data between nodes and servicing requests from clients.

• Down


72

Host is down, not replicating data between nodes and not servicing requests from clients.

Figure 6.13. Down Status

• Pend

Host is up and currently filling RAM with data, but is not servicing requests from clients. Client access will be support-ed once the RAM has been pre-filled with information.

Figure 6.14. Pend Status

You can monitor the current server status using both the Manage: Server Nodes and Monitor: Server Nodes screens withinthe Web Console.

6.6. Settings

The Settings interface sets the global settings for your Couchbase Server instance.

6.6.1. Update Notification Settings

You can enable or disable Update Notifications by checking the Enable software update notifications checkbox within theUpdate Notifications screen. Once you have changed the option, you must click Save to record the change.

If update notifications are disabled then the Update Notifications screen will only notify you of your currently installedversion, and no alert will be provided.


73

Figure 6.15. Web Console Settings — Update Notifications

For more information on how Update Notifications work, see Section 6.8, “Update Notifications”.

6.6.2. Auto-Failover Settings

The Auto-Failover settings enable auto-failover, and the timeout before the auto-failover process is started when a clusternode failure is detected.

To enable Auto-Failover, check the Enable auto-failover checkbox. To set the delay, in seconds, before auto-failover isstarted, enter the number of seconds it the Timeout box. The default timeout is 30 seconds.

Figure 6.16. Web Console Settings — Auto-Failover

For more information on Auto-Failover, see Section 5.1.1, “Automatic Failover”.

6.6.3. Alerts

You can enable email alerts to be raised when a significant error occurs on your Couchbase Server cluster. The email alertsystem works by sending email directly to a configured SMTP server. Each alert email is send to the list of configuredemail recipients.

The available settings are:

• Enable email alerts


74

If checked, email alerts will be raised on the specific error enabled within the Available Alerts section of the configura-tion.

• Host

The hostname for the SMTP server that will be used to send the email.

• Port

The TCP/IP port to be used to communicate with the SMTP server. The default is the standard SMTP port 25.

• Username

For email servers that require a username and password to send email, the username for authentication.

• Password

For email servers that require a username and password to send email, the password for authentication.

• Sender email

The email address from which the email will be identified as being sent from. This email address should be one that isvalid as a sender address for the SMTP server that you specify.

• Recipients

A list of the recipients of each alert message. You can specify more than one recipient by separating each address by aspace, comma or semicolon.

• Available alerts

You can enable individual alert messages that can be sent by using the series of checkboxes. The supported alerts are:

• Node was auto-failovered

The sending node has been auto-failovered.

• Maximum number of auto-failovered nodes was reached

The auto-failover system will stop auto-failover when the maximum number of spare nodes available has beenreached.

• Node wasn't auto-failovered as other nodes are down at the same time

Auto-failover does not take place if there are no spare nodes within the current cluster.

• Node wasn't auto-failovered as the cluster was too small (less than 3 nodes)

You cannot support auto-failover with less than 3 nodes.


75

Figure 6.17. Web Console Settings — Alerts

For more information on Auto-Failover, see Section 5.1.1, “Automatic Failover”.

6.7. Log

The Log section of the website allows you to view the built-in event log for Couchbase Server so that you can identify ac-tivity and errors within your Couchbase cluster.


76

Figure 6.18. Web Console Log Viewer

The Couchbase Web Console displays the following information about each logged event.

Name Description

Event Event that triggered the alert.

Module Code Software module in which the event occurred and aninternal alert code associated with the event.

Server Node Server Node that raised the log entry.

Time Time and date when the event occurred.

6.7.1. Diagnostic Report

You can run a diagnostic report to get a snapshot of your deployment, including version information, the state of the clus-ter, and log output. To generate a diagnostic report: Under Monitor in the left-hand navigation menu, click Log. Click


77

Generate Diagnostic Report (http://hostname:8091/diag). The Couchbase Web Console opens a new browserwindow and downloads the text of the diagnostic report.

6.8. Update Notifications

During installation you can select to enable the Update Notification function. Update notifications allow a client accessingthe Couchbase Web Console to determine whether a newer version of Couchbase Server is available for download.

If you select the Update Notifications option, the Web Console will communicate with Couchbase Servers to confirm theversion number of your Couchbase installation. During this process, the client submits the following information to theCouchbase Server:

• The current version of your Couchbase Server installation. When a new version of Couchbase Server becomes avail-able, you will be provided with notification of the new version and information on where you can download the newversion.

• Basic information about the size and configuration of your Couchbase cluster. This information will be used to help usprioritize our development efforts.

You can enable/disable software update notifications

Note

The process occurs within the browser accessing the web console, not within the server itself, andno further configuration or internet access is required on the server to enable this functionality. Pro-viding the client accessing the Couchbase server console has internet access, the information can becommunicated to the Couchbase Servers.

The update notification process the information anonymously, and the data cannot be tracked. The in-formation is only used to provide you with update notification and to provide information that willhelp us improve the future development process for Couchbase Server and related products.

If the browser or computer that you are using to connect to your Couchbase Server web console doesnot have Internet access, the update notification system will not work.

6.8.1. Notifications

If an update notification is available, the counter within the button display within the Couchbase Console will be displayedwith the number of available updates.

6.8.2. Viewing Available Updates

To view the available updates, click on the Update Notifications link. This displays your current version and update avail-ability. From here you can be taken to the download location to obtain the updated release package.

6.9. Warnings and Alerts

A new alerting systems has been built into the Couchbase Web Console. This is sued to highlight specific issues and prob-lems that you should be aware of and may need to check to ensure the health of your Couchbase cluster. Alerts are provid-ed as a popup within the web console.

The following errors and alerts are supported:

• IP Address Changes


78

If the IP address of a Couchbase Server in your cluster changes, you will be warned that the address is no longer avail-able. You should check the IP address on the server, and update your clients or server configuration.

• OOM (Hard)

Indicates if the bucket memory on a node is entirely used for metadata.

• OOM (Temporary)

Indicates when a bucket on a node is temporarily unable to accept new items and is now waiting for additional items tobe ejected to disk to free up active memory.

• Commit Failure

Indicates that writing data to disk for a specific bucket has failed.

• Metadata Overhead

Indicates that a bucket is now using more than 50% of the allocated RAM for storing metadata and keys, reducing theamount of RAM available for data values.

• Disk Usage

Indicates that the available disk space used for persistent storage has reached at least 90% of capacity.

79

Chapter 7. Couchbase Command-line InterfaceCouchbase Server includes a number of command-line tools that can be used to manage and monitor a Couchbase Servercluster or server. All operations are mapped to their appropriate Chapter 8, Couchbase Management REST APIcall (whereavailable).

There are a number of command-line tools that perform different functions and operations, these are described individual-ly within the following sections. Tools can be located in a number of directories, dependent on the tool in question in eachcase.

Table 7.1. Couchbase Server Command-line Tools

Tool Description

couchbase-cli The main tool for communicating and managing yourCouchbase Server.

cbstats A tool for probing a Couchbase Server node for its localstats

cbflushctl A tool for controlling the disk persistence behavior.

tap.py A tool for accessing the TAP interface of a CouchbaseServer node.

cbvbucketctl A tool for controlling the vBucket states on a CouchbaseServer node.

cbdbmaint A tool for managing the SQLite database on a CouchbaseServer node.

cbcollect_info A support tool for gathering a multitude of statistics from aCouchbase Server node.

7.1. couchbase-cli Tool

Linux /opt/couchbase/bin/couchbase-cli

Windows C:\Program Files\Couchbase\Server\bin\cli\couchbase-cli.exe

This tool provides access to various management operations for Couchbase Server clusters, nodes and buckets. The basicusage format is:

couchbase-cli COMMAND CLUSTER [OPTIONS]

Where:

• COMMAND is a command from Table 7.2, “couchbase Tool Commands”

• CLUSTER is a cluster specification. You can use either:

--cluster=HOST[:PORT]

Or the shortened form:

-c HOST[:PORT]

• OPTIONS is zero or more options.

Couchbase Command-line Interface

80

Table 7.2. couchbase Tool Commands

Command Description

server-list List all servers in a cluster

server-info Show details on one server

server-add Add one or more servers to the cluster

server-readd Re-add a server that was failed over

rebalance Start a cluster rebalancing

rebalance-stop Stop current cluster rebalancing

rebalance-status Show status of current cluster rebalancing

failover Failover one or more servers

cluster-init Set the username,password and port of the cluster

node-init Set node specific parameters

bucket-list List all buckets in a cluster

bucket-create Add a new bucket to the cluster

bucket-edit Modify an existing bucket

bucket-delete Delete an existing bucket

bucket-flush Flush a given bucket

help Show longer usage/help and examples

Table 7.3. Standard couchbase Options

Option Command Description

--user=USERNAME,-u USER-NAME

Administrator username for accessingthe cluster

--password=PASSWORD,-pPASSWORD

Administrator password for accessingthe cluster

--output=KIND,-o KIND Output kind, either json for JSON for-mat, or standard for the native for-mat for the command.

--debug,-d Output debug information.

--server-add=HOST[:PORT] server-add,server-readd, re-balance

Server to be added

--server-add-username=USERNAME

server-add,server-readd, re-balance

Admin username for the server to beadded

--server-add-password=PASSWORD

server-add,server-readd, re-balance

Admin password for the server to beadded

--serv-er-remove=HOST[:PORT]

rebalance The server to be removed

--serv-er-failover=HOST[:PORT]

failover Server to failover

--cluster-init-username=USER

cluster-init New admin username

--cluster-init-password=PASSWORD

cluster-init New admin password


81

Option Command Description

--cluster-init-port=PORT cluster-init New cluster port (must be at least 6characters)

--cluster-init-ramsize=300 cluster-init New RAM quota

--bucket=BUCKETNAME bucket* Bucket to act on

--bucket-type=TYPE bucket* Memcached or Memhbase

--bucket-port=PORT bucket* Supports ASCII protocol and is auth-less

--bucket-password=PASSWORD bucket* Standard port, exclusive with buck-et-port

--bucket-ramsize=RAMSIZEMB bucket* RAM quota in MB

--bucket-replica=COUNT bucket* Replication count

Examples:

• List servers in a cluster:

shell> couchbase-cli server-list -c 192.168.0.1:8091

• Server information

shell> couchbase-cli server-info -c 192.168.0.1:8091

• Add a node to a cluster, but do not rebalance:

shell> couchbase-cli server-add -c 192.168.0.1:8091 \ --server-add=192.168.0.2:8091

• Add a node to a cluster and rebalance:

shell> couchbase-cli rebalance -c 192.168.0.1:8091 \ --server-add=192.168.0.2:8091

• Remove a node from a cluster and rebalance:

shell> couchbase-cli rebalance -c 192.168.0.1:8091 \ --server-remove=192.168.0.2:8091

• Remove and add nodes from/to a cluster and rebalance:

shell> couchbase-cli rebalance -c 192.168.0.1:8091 \ --server-remove=192.168.0.2 \ --server-add=192.168.0.4

• Stop the current rebalancing:

shell> couchbase-cli rebalance-stop -c 192.168.0.1:8091

• Change the username, password and port:

shell> couchbase-cli cluster-init -c 192.168.0.1:8091 \ --cluster-init-username=Administrator \ --cluster-init-password=password \ --cluster-init-port=8080

• List buckets in a cluster:

shell> couchbase-cli bucket-list -c 192.168.0.1:8091


82

• Create a new couchbase bucket with a dedicated port:

shell> couchbase-cli bucket-create -c 192.168.0.1:8091 --bucket=test_bucket \ --bucket-type=couchbase --bucket-port=11222 --bucket-ramsize=200 \ --bucket-replica=1

• Create a new sasl memcached bucket:

shell> couchbase-cli bucket-create -c 192.168.0.1:8091 --bucket=test_bucket \ --bucket-type=memcached--bucket-password=password \ --bucket-ramsize=200

• Modify a dedicated port bucket:

shell> couchbase-cli bucket-edit -c 192.168.0.1:8091 --bucket=test_bucket \ --bucket-port=11222 --bucket-ramsize=400

• Delete a bucket:

shell> couchbase-cli bucket-delete -c 192.168.0.1:8091 --bucket=test_bucket

7.2. cbstats Tool

Linux /opt/couchbase/bin/cbstats

Windows C:\Program Files\Couchbase\Server\bin\cbstats.exe

A tool for obtaining the couchbase node statistics. The general format for usage is:

shell> cbstats HOST COMMAND [options] [username password]

Where HOST is the hostname and port (HOSTNAME[:PORT]) combination for a Couchbase bucket, and username andpassword are the authentication details. COMMAND(and[options]) are one of:

alldispatcherhashresettaptimingsvkey keyname vbid

Example: The stats output can be used with other command-line tools to sort and filter the data.

watch --diff "cbstats \ ip-10-12-19-81:11210 all | egrep 'item|mem|flusher|ep_queue|bg|eje|resi|warm'"

7.3. cbflushctl Tool

Linux /opt/couchbase/bin/cbflushctl

Windows C:\Program Files\Couchbase\Server\bin\cbflushctl.exe

Usage:


83

flushctl Usage:flushctl host:port evict key orflushctl host:port set param value orflushctl host:port start orflushctl host:port stop

Available params:

min_data_age - minimum data age before flushing dataqueue_age_cap - maximum queue age before flushing datamax_txn_size - maximum number of items in a flusher transactionbg_fetch_delay - delay before executing a bg fetch (test feature)max_size - max memory used by the servermem_high_wat - high water markmem_low_wat - low water mark

7.4. tap.py Tool

Linux /opt/couchbase/bin/ep_engine/manage-ment/tap.py

Usage:

/opt/couchbase/bin/ep_engine/management/tap.py host:port

7.5. cbvbucketctl Tool

Linux /opt/couchbase/bin/cbvbucketctl

Windows C:\Program Files\Couchbase\Server\bin\cbvbucketctl.exe

Usage:

mbvbucketctl Usage:mbvbucketctl host:port list [username password] ormbvbucketctl host:port rm vbid [username password] ormbvbucketctl host:port set vbid vbstate [username password]

7.6. cbvbuckettool Tool

Usage:

shell> cbvbuckettool mapfile key0 [key1 ... keyN]

The cbvbuckettool expects a vBucketServerMap JSON mapfile, and will print the vBucketId and servers each key shouldlive on. You may use '-' instead for the filename to specify stdin.

Examples:

shell> cbvbuckettool file.json some_key another_key

Or

shell> curl http://host:8091/pools/default/buckets/default | cbvbuckettool - some_key another_key

An example of running it with output:

shell> curl http://127.0.0.1:8091/pools/default/buckets/default | cbvbuckettool - some_key another_keykey: some_key vBucketId: 260 master: http://127.0.0.1:11210/ replicas:key: another_key vBucketId: 1022 master: http://127.0.0.1:11210/ replicas:


84

7.7. cbdbmaint Tool

Linux /opt/couchbase/bin/cbdbmaint

Windows C:\Program Files\Couchbase\Server\bin\cbdbmaint.exe

Usage:

cbdbmaint [--vacuum] [--backupto=<dest_dir>] [--port=11210]

7.8. cbcollect_info Tool

Linux /opt/couchbase/bin/cbcollect_info

Usage:

cbcollect_info [options] output_file

Options: -h, --help show this help message and exit -v increase verbosity level

85

Chapter 8. Couchbase Management REST APIThe Couchbase Management REST API provides an easy to use interface for controlling your Couchbase installation.

The Couchbase Management REST API provides the ability to manage a Couchbase Server deployment. It conforms toRepresentational State Transfer (REST) constraints (is a RESTful architecture). The Couchbase Management REST APIis used to manage clusters, server nodes, and buckets, and to retrieve run-time statistics, within your Couchbase Server de-ployment.

The REST API is not used to directly manage data in memory or on disk. The cache data management operations (setand get, for example) are handled by clients and the appropriate SDKs. See Couchbase SDKs.

The REST API makes use and reference to a number of different systems within the Couchbase Server product. For moreinformation, readSection 8.1, “Key Concepts”.

The REST API is built on a number of basic principles:

• JSON Responses

The Couchbase Management REST API returns many responses in the JavaScript Object Notation (JSON). Some re-sponses have an empty body, but indicate the response with standard HTTP codes. For more information, see RFC 4627(http://www.ietf.org/rfc/rfc4627.txt) andwww.json.org.

• HTTP Basic Access Authentication

The Couchbase Management REST API uses HTTP basic authentication. The browser-based Chapter 6, AdministrationWeb Consoleand Chapter 7, Couchbase Command-line Interfacealso use HTTP basic authentication.

• Versatile Server Nodes

All server nodes in a cluster share the same properties and can handle any requests made via the Couchbase Manage-ment REST API. It does not matter which specific server node you communicate with via the API. If the server nodecannot service a request directly (due to lack of access to state or some other information), it will forward the request tothe appropriate server node, retrieve the results, and send the results back to the client.

8.1. Key Concepts

In order to use the REST API you should be aware of the different terms and concepts highlighted in this section.

8.1.1. Resources

There are a number of different resources within the Couchbase Server and these resources can optionally have one ormore controllers, which have RESTful endpoints in the representation of the item one would control.

• Pools/Clusters

A pool (or cluster) is a collection of physical resources that are grouped together and provide services and a manage-ment interface. A single, default pool exists for every deployment of Couchbase Server. A Node (server node) is amember of a pool. Couchbase Server collects run-time statistics for pools, maintaining an overall pool-level data viewof counters and periodic metrics of the overall system. The Couchbase Management REST API can be used to retrievehistoric statistics for pools.

• Server Nodes

A Server Node is a physical or virtual machine running Couchbase Server as a member of a pool.

http://couchbase.com/develop

http://www.ietf.org/rfc/rfc4627.txt

http://www.json.org

Couchbase Management REST API

86

• Data Buckets

A data bucket is a logical grouping of resources within a pool. In support of pool and resource management, a bucketprovides unconstrained namespaces to users to define whatever bucket name makes sense to the user. It also allows thesame key in the same application to be available in multiple places.

Couchbase Server collects run-time statistics for buckets, maintaining an overall bucket-level data view of counters andperiodic metrics of the overall system. Buckets are categorized by storage type. Buckets can implement different kindsof behaviors, such as cache only or persisted (either synchronous or asynchronous) document stores.

8.1.2. HTTP Request Headers

The following HTTP request headers are supported when making requests to any URI within the REST API. The requestheaders are validated server-side during requests.

Table 8.1. Supported Request Headers

Header Supported Values Description of Use Required

Accept Comma-delimited list of me-dia types or media type pat-terns.

Indicates to the server what media type(s) thisclient is prepared to accept.

Recommended

Authorization Basic plus username andpassword (per RFC 2617).

Identifies the authorized user making this re-quest.

No, unless secured

Content-Length Body Length (in bytes) Describes the size of the message body. Yes, on requests thatcontain a messagebody.

Content-Type Content type Describes the representation and syntax of therequest message body.

Yes, on requests thatcontain a messagebody.

Host Origin hostname Required to allow support of multiple originhosts at a single IP address.

All requests

X-YYYYY-Client-Specifica-tion-Version

String Declares the specification version of theYYYYY API that this client was programmedagainst.

No

8.1.3. HTTP Status Codes

The Couchbase Management REST API returns the following standard HTTP response codes under the conditions ex-plained in the associated description.

Table 8.2. HTTP Status Codes

HTTP Status Description

200 OK The request was successfully completed. If this request created a new resource that isaddressable with a URI, and a response body is returned containing a representation ofthe new resource, a 200 status will be returned with a Location header containing thecanonical URI for the newly created resource.

201 Created A request that created a new resource was completed, and no response body containinga representation of the new resource is being returned. A Location header containing thecanonical URI for the newly created resource should also be returned.

202 Accepted The request has been accepted for processing, but the processing has not been complet-ed. Per the HTTP/1.1 specification, the returned entity (if any) SHOULD include an in-


87

HTTP Status Description

dication of the request's current status, and either a pointer to a status monitor or someestimate of when the user can expect the request to be fulfilled.

204 No Content The server fulfilled the request, but does not need to return a response message body.

400 Bad Request The request could not be processed because it contains missing or invalid information(such as validation error on an input field, a missing required value, and so on).

401 Unauthorized The authentication credentials included with this request are missing or invalid.

403 Forbidden The server recognized your credentials, but you do not possess authorization to performthis request.

404 Not Found The request specified a URI of a resource that does not exist.

405 Method Not Allowed The HTTP verb specified in the request (DELETE, GET, HEAD, POST, PUT) is notsupported for this request URI.

406 Not Acceptable The resource identified by this request is not capable of generating a representation cor-responding to one of the media types in the Accept header of the request.

409 Conflict A creation or update request could not be completed, because it would cause a conflictin the current state of the resources supported by the server (for example, an attemptto create a new resource with a unique identifier already assigned to some existing re-source).

500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling therequest.

501 Not Implemented The server does not (currently) support the functionality required to fulfill the request.

503 Service Unavailable The server is currently unable to handle the request due to temporary overloading ormaintenance of the server.

8.2. OperationsThis section describes Couchbase Management REST API operations.

Launching the Couchbase Web Console

Although not strictly part of the REST API, the Couchbase Web Console does use many of the REST API endpoints forthe management and information gathering within the console itself.

The Couchbase Web Console loads and runs in a common Web Browser user agent. It consists of HTML pages, images,CSS, and JavaScript. For a list of supported browsers, see System Requirements. For the Couchbase Web Console, a sep-arate UI hierarchy is served from each node of the system (though asking for the root "/" would likely return a redirect tothe user agent). To launch the Couchbase Web Console, point your browser to the appropriate host and port:

GET https://node.in.pool.com/

Bootstrapping

Clients must bootstrap to gain an entry point into the RESTful service. Bootstrapping requires looking for pools, as shownin the request blow. The URI space, in Couchbase's implementation may appear to have very specific URI and in someways may even appear as RPC or some other architectural style using HTTP operations and semantics. That is only an ar-tifact of the URIs Couchbase chose. Clients are advised to be properly RESTful and will not expect to receive any han-dling instructions resource descriptions or presume any conventions on URI structure for resources represented. Also notethat the hierarchies shown here can allow reuse of agent handling of representations, since they are similar for differentparts of the hierarchy.

Request


88

GET /poolsHost: node.in.your.pool.comAuthorization: Basic xxxxxxxxxxxxxxxxxxxAccept: application/jsonX-memcachekv-Store-Client-Specification-Version: 0.1

Response

HTTP/1.1 200 OKContent-Type: application/jsonContent-Length: nnn

{ "implementationVersion": "253", "pools" : [ { "name": "Default Pool", "uri": "/pools/default", }, { "name": "Couchbase kvcaching pool name", "uri": "/pools/anotherpool", }, { "name": "A pool elsewhere", "uri": "https://a.node.in.another.pool.com:4321/pools/default" } ] "uri": "https://node.in.your.pool.com/pools", "specificationVersion": [ "0.1" ]}

Only one pool per group of systems will be known and it will likely be defaulted to a name. POSTing back a changed poolwill return a 403.

As can be seen, the "build" number of the implementation is apparent in the implementation_version, the specificationssupported are apparent in the specification_version. While this node can only be a member of one pool, there is flexibilitywhich allows for any given node to be aware of other pools.

The Client-Specification-Version is optional in the request, but advised. It allows for implementations to adjust to adjustrepresentation and state transitions to the client, if backward compatibility is desirable.

Provisioning a Node

Before a node can be used in a cluster, a few things may need to be configured. Specifically, if creating a new cluster, thememory quota per node for that cluster must be set. Whether the node is joining an existing cluster or starting a new clus-ter, it's storage path must be configured.

Either creating a new cluster or adding a node to a cluster is referred to as provisioning, and has several steps required. Af-ter bootstrapping the following will need to be accomplished.

• Configure the node's disk path.

• Configure the cluster's memory quota. This is optional. It will inherit the memory quota if the node is to be joined to acluster and it will default to 80% of physical memory if not specified.

The next step depends on whether a new cluster is being created or an existing cluster will be joined. If a new cluster is tobe created, it will need

to be "secured" by providing a username and password for the administrator. If the node is to be joined to another cluster,then it will need the location and credentials to the REST interface of that cluster.

Configuring the disk path for a node


89

Node resources can be configured through a controller on the node. The primary resource to be configured is the path onthe node for persisting files. This must be configured prior to creating a new cluster or configuring the node into an exist-ing cluster.

Request

POST /nodes/self/controller/settings HTTP/1.1Host: node.in.your.cluster:8091Content-Type: application/x-www-form-urlencoded; charset=UTF-8Authorization: Basic YWRtaW46YWRtaW4=Content-Length: xx path=/var/tmp/test

Response

HTTP/1.1 200 OKContent-Type: application/jsonContent-Length: 0

Example

shell> curl -d path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

Configuring a cluster's memory quota Request

POST /pools/default HTTP/1.1Host: node.in.your.cluster:8091Content-Type: application/x-www-form-urlencoded; charset=UTF-8Authorization: Basic YWRtaW46YWRtaW4=Content-Length: xx memoryQuota=400

Response

HTTP/1.1 200 OKContent-Type: application/jsonContent-Length: 0

Example

shell> curl -d memoryQuota=400 http://localhost:8091/pools/default

Setting a node's username and password

While this can be done at any time for a cluster, it is the last step in provisioning a node into being a new cluster. The re-sponse will indicate the new base URI if the parameters are accepted. Clients will want to re-bootstrap based on this re-sponse.

Request Response

POST /settings/web HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded; charset=UTF-8 Authorization: Basic YWRtaW46YWRtaW4= Content-Length: xx username=Administrator&password=letmein&port=8091

HTTP/1.1 200 OK Content-Type: application/json Server: Couchbase Server 1.6.0 Pragma: no-cache Date: Mon, 09 Aug 2010 18:50:00 GMT Content-Type: application/json Content-Length: 39 Cache-Control: no-cache no-store max-age=0 {"newBaseUri":"http://localhost:8091/"}

Example

curl -d username=Administrator -d password=letmein -d port=8091 http://localhost:8091/settings/web

Note

Note that even if it is not to be changed, the port number must be specified.

Global settings

Setting whether statistics should be sent to the outside or not

It's a global setting for all clusters. You need to be authenticated to change this value.

Request

POST /settings/stats HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded Authorization: Basic YWRtaW46YWRtaW4= Content-Length: 14 sendStats=true


90

Response

200 OK 400 Bad Reqeust, The value of "sendStats" must be true or false. 401 Unauthorized

Example

curl -i -u Administrator:letmein -d sendStats=true http://localhost:8091/settings/stats

Getting whether statistics should be sent to the outside or not

It's a global setting for all clusters. You need to be authenticated to read this value.

Request

GET /settings/stats HTTP/1.1 Host: node.in.your.pool.com Authorization: Basic YWRtaW46YWRtaW4= Accept: */*

Response

HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn

Example

curl -u Administrator:letmein http://localhost:8091/settings/stats

Setting whether auto-failover should be enabled or disabled

This endpoint isn't available yet.


Possible parameters are:

• enabled (true|false) (required): whether to enable or disable auto-failover

• timeout (integer biger (or equal) than 30) (required; optional when enabled=false): The number of seconds a node mustbe down before it is automatically failovered

Request

POST /settings/autoFailover HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded Authorization: Basic YWRtaW46YWRtaW4= Content-Length: 14 enabled=true&timeout=60

Response

200 OK 400 Bad Request, The value of "enabled" must be true or false. 400 Bad Request, The value of "timeout" must bea positive integer bigger or equal to 30. 401 Unauthorized

Example

curl -i -u Administrator:letmein -d enabled=true&timeout=60http://localhost:8091/settings/autoFailover

Getting information about the auto-failover settings



Request Response

GET /settings/autoFailover HTTP/1.1 Host: node.in.your.pool.com Authorization: Basic YWRtaW46YWRtaW4= Accept: */*

HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "enabled": true, "timeout": 60, "count": 0 }

Example

curl -u Administrator:letmein http://localhost:8091/settings/autoFailover

Reset the auto-failover count


91


It's a global setting for all clusters. You need to be authenticated to change this value. Reset the number of nodes thatwhere automatically failovered to zero. No parameters

Request

POST /settings/autoFailover/resetCount HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded Authorization: Basic YWRtaW46YWRtaW4=

Response

200 OK 401 Unauthorized

Example

curl -i -u Administrator:letmein http://localhost:8091/settings/autoFailover/resetCount

Setting whether email notification should be enabled or disabled



You will receive an email when certain events happen (currently only events cause by auto-failover are supported).

Possible parameters are:

• enabled (true|false) (required): whether to enable or disable email notifications

• sender (string) (optional, default: couchbase@localhost): The sender address of the email

• recipients (string) (required): A comma separated list of recipients of the of the alert emails.

• emailHost (string) (optional, default: localhost): Host address of the SMTP server

• emailPort (integer) (optional, default: 25): Port of the SMTP server

• emailEncrypt (true|false) (optional, default: false): Whether you'd like to use TLS or not

• emailUser (string) (optional, default: ""): Username for the SMTP server

• emailPass (string) (optional, default: ""): Password for the SMTP server

• alerts (string) (optional, default: auto_failover_node, auto_failover_maximum_reached,auto_failover_other_nodes_down, auto_failover_cluster_too_small): Comma separated list of alerts thatshould cause an email to be sent. Possible values are: auto_failover_node, auto_failover_maximum_reached,auto_failover_other_nodes_down, auto_failover_cluster_too_small.

Request

POST /settings/alerts HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded Authorization: Basic YWRtaW46YWRtaW4= Content-Length: 14 enabled=true&sender=couchbase@localhost&recipients=admin@localhost,membi@localhost&emailHost=localhost&emailPort=25&emailEncrypt=false#

Response

200 OK 400 Bad Request: JSON object ({"errors": {"key": "error"}}) with errors. Possible errors are:

• alerts: alerts contained invalid keys. Valid keys are: [list_of_keys].

• email_encrypt: emailEncrypt must be either true or false.

• email_port: emailPort must be a positive integer less than 65536.

• enabled: enabled must be either true or false.


92

• recipients: recipients must be a comma separated list of valid email addresses.

• sender: sender must be a valid email address.

• general: No valid parameters given.

• 401 Unauthorized

Example

curl -i -u Administrator:letmein -d 'enabled=true&sender=couchbase@localhost&recipients=admin@localhost,membi@localhost&emailHost=localhost&emailPort=25&emailEncrypt=false' http://localhost:8091/settings/alerts

Getting information about the email notification settings



Request

GET /settings/alerts HTTP/1.1 Host: node.in.your.pool.com Authorization: Basic YWRtaW46YWRtaW4= Accept: */*

Response Example

HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "alerts": [ "auto_failover_node", "auto_failover_maximum_reached" ], "emailServer": { "encrypt": false, "host": "localhost", "pass": "", "port": 25, "user": "" }, "enabled": true, "recipients": [ "user@localhost" ], "sender": "couchbase@localhost" }

curl -u Administrator:letmein http://localhost:8091/settings/alerts

Send test email


It's a global setting for all clusters. You need to be authenticated to change this value. Sends a test email with the currentconfiguration (see mail alerts section)

Takes the same parameters as /settings/alerts and additionally: subject (string) (required): The subject of the test emailbody (string) (required): The body of the test email

Request

POST /settings/alerts/sendTestEmail HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded Authorization: Basic YWRtaW46YWRtaW4=

Response

200 OK 400 Bad Request: Unknown macro: {"error"} 401 Unauthorized

Example

curl -i -u Administrator:letmein http://localhost:8091/settings/alerts/sendTestEmail -d 'subject=Test+email+from+Couchbase&body=This+email+was+sent+to+you+to+test+the+email+alert+email+server+settings.&enabled=true&recipients=vmx%40localhost&sender=couchbase%40localhost&emailUser=&emailPass=&emailHost=localhost&emailPort=25&emailEncrypt=false&alerts=auto_failover_node%2Cauto_failover_maximum_reached%2Cauto_failover_other_nodes_down%2Cauto_failover_cluster_too_small'

Pool Details

Request

GET /pools/default Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1

Response


93

HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn

{ "name":"default", "nodes":[{

"hostname":"10.0.1.20",

"status":"healthy", "uptime":"14", "version":"1.6.0", "os":"i386-apple-darwin9.8.0", "memoryTotal":3584844000.0, "memoryFree":74972000, "mcdMemoryReserved":64, "mcdMemoryAllocated":48, "ports":{

"proxy":11213, "direct":11212 }, "otpNode":"[email protected]", "otpCookie":"fsekryjfoeygvgcd", "clusterMembership":"active" }], "storageTotals":{

"ram":{ "total":2032558080, "used":1641816064

}, "hdd":{ "total":239315349504.0, "used": 229742735523.0} }, "buckets":{

"uri":"/pools/default/buckets" }, "controllers":{ "ejectNode":{>

"uri":"/pools/default/controller/ejectNode" }, "addNode":{

"uri":"/controller/addNode" }, "rebalance":{

"uri":"/controller/rebalance" }, "failover":{

"uri":"/controller/failOver" }, "reAddNode":{

"uri":"/controller/reAddNode" }, "stopRebalance":{

"uri":"/controller/stopRebalance" } }, "rebalanceProgress":{

"uri":"/pools/default/rebalanceProgress" }, "balanced": true, "etag":"asdas123", "initStatus":

"stats":{ "uri":"/pools/default/stats" } }


94

At the highest level, a pool describes a cluster (as mentioned above). This cluster exposes a number of properties whichdefine attributes of the cluster and "controllers" which allow users to make certain requests of the cluster.

Note that since buckets could be renamed and there is no way to determine what the default bucket for a pool is, the sys-tem will attempt to connect non-SASL, non-proxied to a bucket clients to a bucket named "default". If it does not exist, theconnection will be dropped.

Clients MUST NOT rely on the node list here to create their "server list" for when connecting. They MUST instead issuean HTTP get call to the bucket to get the node list for that specific bucket.

The controllers, all of which accept parameters as x-www-form-urlencoded, for this list perform the following functions:

Table 8.3. Controller Functions

Function Description

ejectNode Eject a node from the cluster. Required parameter: "otpN-ode", the node to be ejected.

addNode Add a node to this cluster. Required parameters: "host-name", "user" (which is the admin user for the node), and"password".

rebalance Rebalance the existing cluster. This controller requires both"knownNodes" and "ejectedNodes". This allows a client tostate the existing known nodes and which nodes should beremoved from the cluster in a single operation. To ensureno cluster state changes have occurred since a client last gota list of nodes, both the known nodes and the node to beejected must be supplied. If the list does not match the setof nodes, the request will fail with an HTTP 400 indicatinga mismatch. Note rebalance progress is available via the re-balanceProgress uri.

failover Failover the vBuckets from a given node to the nodes whichhave replicas of data for those vBuckets. The "otpNode" pa-rameter is required and specifies the node to be failed over.

reAddNode The "otpNode" parameter is required and specifies the nodeto be re-added.

stopRebalance Stop any rebalance operation currently running. This takesno parameters.

The list of nodes will list each node in the cluster. It will, additionally, list some attributes of the nodes.

Table 8.4. Node Attributes

Attribute Description

memoryTotal The total amount of memory available to Couchbase, allo-cated and free. May or may not be equal to the amount ofmemory configured in the system.

memoryFree The amount of memory available to be allocated. This isequal to the memoryTotal, subtracting out all memory allo-cated as reported by the host operating system.

mcdMemoryReserved The amount of memory reserved for use by Couchbaseacross all buckets on this node. This value does not include


95

Attribute Description

some overhead for managing items in the node or handlingreplication or other TAP streams.

mcdMemoryAllocated The amount of memory actually used by all buckets on thisnode.

List Buckets and Bucket Operations

Clients to the system can choose to use either the proxy path or the direct path. If they use the direct path, they will not beinsulated from most reconfiguration changes to the bucket. This means they will need to either poll the bucket's URI orconnect to the streamingUri to receive updates when the bucket configuration changes. This happens, for instance, whennodes are added, removed, or may fall into an unhealthy state.

Request

GET /pools/default/bucketsHost: node.in.your.pool.comAuthorization: Basic xxxxxxxxxxxxxxxxxxxAccept: application/jsonX-memcachekv-Store-Client-Specification-Version: 0.1

Response


96

HTTP/1.1 200 OKServer: Couchbase Server 1.6.0Pragma: no-cacheDate: Wed, 03 Nov 2010 18:12:19 GMTContent-Type: application/jsonContent-Length: nnnCache-Control: no-cache no-store max-age=0

[ { "name": "default", "bucketType": "couchbase", "authType": "sasl", "saslPassword": "", "proxyPort": 0, "uri": "/pools/default/buckets/default", "streamingUri": "/pools/default/bucketsStreaming/default", "flushCacheUri": "/pools/default/buckets/default/controller/doFlush", "nodes": [ { "uptime": "784657", "memoryTotal": 8453197824.0, "memoryFree": 1191157760, "mcdMemoryReserved": 6449, "mcdMemoryAllocated": 6449, "clusterMembership": "active", "status": "unhealthy", "hostname": "10.1.15.148:8091", "version": "1.6.0", "os": "windows", "ports": { "proxy": 11211, "direct": 11210 } } ], "stats": { "uri": "/pools/default/buckets/default/stats" }, "nodeLocator": "vbucket", "vBucketServerMap": { "hashAlgorithm": "CRC", "numReplicas": 1, "serverList": [ "192.168.1.2:11210" ], "vBucketMap": [ [ 0, -1 ], [ 0, -1 ], [ 0, -1 ], [ 0, -1 ], [ 0, -1 ], [ 0, -1 ]] }, "replicaNumber": 1, "quota": { "ram": 104857600, "rawRAM": 104857600 }, "basicStats": { "quotaPercentUsed": 24.360397338867188, "opsPerSec": 0, "diskFetches": 0, "itemCount": 0, "diskUsed": 0, "memUsed": 25543728 } }, { "name": "test-application", "bucketType": "memcached", "authType": "sasl", "saslPassword": "", "proxyPort": 0, "uri": "/pools/default/buckets/test-application", "streamingUri": "/pools/default/bucketsStreaming/test-application", "flushCacheUri": "/pools/default/buckets/test-application/controller/doFlush", "nodes": [ { "uptime": "784657", "memoryTotal": 8453197824.0, "memoryFree": 1191157760, "mcdMemoryReserved": 6449, "mcdMemoryAllocated": 6449, "clusterMembership": "active", "status": "healthy", "hostname": "192.168.1.2:8091", "version": "1.6.0", "os": "windows", "ports": { "proxy": 11211, "direct": 11210 } } ], "stats": { "uri": "/pools/default/buckets/test-application/stats" }, "nodeLocator": "ketama", "replicaNumber": 0, "quota": { "ram": 67108864, "rawRAM": 67108864 }, "basicStats": { "quotaPercentUsed": 4.064150154590607, "opsPerSec": 0, "hitRatio": 0, "itemCount": 1385, "diskUsed": 0, "memUsed": 2727405 } }]


97

Named Bucket and Bucket Streaming URI

The individual bucket request is exactly the same as what would be obtained from the item in the array for the entire buck-ets list above. The streamingUri is exactly the same except it streams HTTP chunks using chunked encoding. A responseof "\n\n\n\n" delimits chunks. This will likely be converted to a "zero chunk" in a future release of this API, and thus thebehavior of the streamingUri should be considered evolving.

Request

GET /pools/default/buckets/defaultHost: node.in.your.pool.comAuthorization: Basic xxxxxxxxxxxxxxxxxxxAccept: application/jsonX-memcachekv-Store-Client-Specification-Version: 0.1

Response


98

HTTP/1.1 200 OKContent-Type: application/jsonContent-Length: nnn

{ "name": "default", "bucketType": "couchbase", "authType": "sasl", "saslPassword": "", "proxyPort": 0, "uri": "/pools/default/buckets/default", "streamingUri": "/pools/default/bucketsStreaming/default", "flushCacheUri": "/pools/default/buckets/default/controller/doFlush", "nodes": [ { "uptime": "308", "memoryTotal": 3940818944.0, "memoryFree": 1608724480, "mcdMemoryReserved": 3006, "mcdMemoryAllocated": 3006, "replication": 1.0, "clusterMembership": "active", "status": "healthy", "hostname": "172.25.0.2:8091", "clusterCompatibility": 1, "version": "1.6.4r_107_g49a149d", "os": "i486-pc-linux-gnu", "ports": { "proxy": 11211, "direct": 11210 } }, { "uptime": "308", "memoryTotal": 3940818944.0, "memoryFree": 1608724480, "mcdMemoryReserved": 3006, "mcdMemoryAllocated": 3006, "replication": 1.0, "clusterMembership": "active", "status": "healthy", "hostname": "172.25.0.3:8091", "clusterCompatibility": 1, "version": "1.6.4r_107_g49a149d", "os": "i486-pc-linux-gnu", "ports": { "proxy": 11211, "direct": 11210 } }, { "uptime": "308", "memoryTotal": 3940818944.0, "memoryFree": 1608597504, "mcdMemoryReserved": 3006, "mcdMemoryAllocated": 3006, "replication": 1.0, "clusterMembership": "active", "status": "healthy", "hostname": "172.25.0.4:8091", "clusterCompatibility": 1, "version": "1.6.4r_107_g49a149d", "os": "i486-pc-linux-gnu", "ports": { "proxy": 11211, "direct": 11210 } } ], "stats": { "uri": "/pools/default/buckets/default/stats" }, "nodeLocator": "vbucket", "vBucketServerMap": { "hashAlgorithm": "CRC", "numReplicas": 1, "serverList": [ "172.25.0.2:11210", "172.25.0.3:11210", "172.25.0.4:11210" ], "vBucketMap": [ [1,0], [2,0], [1,2], [2,1], [1,2], [0,2], [0,1], [0,1] ] }, "replicaNumber": 1, "quota": { "ram": 1887436800, "rawRAM": 629145600 }, "basicStats": { "quotaPercentUsed": 14.706055058373344, "opsPerSec": 0, "diskFetches": 0, "itemCount": 65125, "diskUsed": 139132928, "memUsed": 277567495 }}


99

Flushing a Bucket

Warning

This operation is data destructive.The service makes no attempt to double check with the user. It sim-ply moves forward. Clients applications using this are advised to double check with the end user be-fore sending such a request.

Note

As of couchbase 1.6 bucket flushing via REST API is not supported. Flushing via memcached proto-col works.

The bucket details provide a bucket URI at which a simple request can be made to flush the bucket.

Request

POST /pools/default/buckets/default/controller/doFlush Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx X-memcachekv-Store-Client-Specification-Version: 0.1

Any parameters are accepted. Since the URI was defined by the bucket details, neither the URI nor the parameters controlwhat is actually done by the service. The simple requirement is for a POST with an appropriate Authorization header, ifthe system is secured.

Response

204 No Content - if the flush is successful 404 Not Found - if the URI is invalid or does not correspond to a bucket the system is familiar with.

Deleting a Bucket

Warning

This operation is data destructive.The service makes no attempt to double check with the user. It sim-ply moves forward. Clients applications using this are advised to double check with the end user be-fore sending such a request.

Request

DELETE /pools/default/buckets/default Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx X-memcachekv-Store-Client-Specification-Version: 0.1

Response

Getting Statistics

Statistics can be gathered via the REST interface at the bucket level. Stats URL should be taken from stats.uri property ofbucket response. By default it returns stats samples for last minute and hot keys. Via query parameters another zoom levelcan be requested.

• zoom - stats zoom level (minute | hour | day | week | month | year)

• resampleForUI - pass 1 if you need 60 samples

• haveTStamp - request sending only samples newer than given timestamp

Request Response

GET /pools/default/stats Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1


100

HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn { "op": { "samples": { "hit_ratio": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_cache_miss_rate": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_resident_items_rate": [ 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0 ], "ep_replica_resident_items_rate": [ 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0 ], "bytes_read": [ 151, 130, 72, 158, 120, 223, 103, 127, 127, 127, 103, 130, 72, 158, 72, 220, 103, 175, 130, 127, 103, 127, 120, 158, 72, 271, 103, 124, 130, 127, 103, 130, 72, 155, 72, 223, 103, 124, 178, 127, 103, 130, 72, 206, 69, 223, 103, 175, 130, 127, 103, 130, 72, 155, 72, 223, 103, 127, 130 ], "bytes_written": [ 36776, 40699, 15212, 41705, 15260, 41326, 36728, 20360, 40658, 20388, 36728, 40699, 15212, 41705, 15212, 41283, 36764, 20408, 40700, 20369, 36728, 40656, 15274, 41705, 15212, 41374, 36728, 20338, 40700, 20388, 36728, 40699, 15212, 41662, 15212, 41366, 36728, 20338, 40748, 20388, 36728, 40699, 15212, 41753, 15195, 41326, 36728, 20408, 40739, 20369, 36728, 40699, 15212, 41662, 15212, 41366, 36728, 20360, 40700 ], "cas_badval": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cas_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cas_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cmd_get": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "cmd_set": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "curr_connections": [ 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139 ], "curr_items": [ 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125 ], "curr_items_tot": [ 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250 ], "decr_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "decr_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "delete_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "delete_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "disk_writes": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_bg_fetched": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_flusher_todo": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_io_num_read": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_keys_size": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_num_active_non_resident": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_num_eject_replicas": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_num_non_resident": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_num_not_my_vbuckets": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_num_value_ejects": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_oom_errors": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_overhead": [ 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200, 399200 ], "ep_queue_size": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_tap_bg_fetched": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_tmp_oom_errors": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_total_persisted": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ep_values_size": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "evictions": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "get_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "get_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "incr_hits": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "incr_misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "mem_used": [ 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950, 275869950 ], "misses": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "ops": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ], "replica_resident_items_tot": [ 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250, 130250 ], "resident_items_tot": [ 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125, 65125 ], "timestamp": [ 1292513719166.0, 1292513720166.0, 1292513721166.0, 1292513722166.0, 1292513723166.0, 1292513724166.0, 1292513725166.0, 1292513726166.0, 1292513727167.0, 1292513728166.0, 1292513729166.0, 1292513730166.0, 1292513731166.0, 1292513732166.0, 1292513733166.0, 1292513734167.0, 1292513735166.0, 1292513736166.0, 1292513737166.0, 1292513738166.0, 1292513739166.0, 1292513740167.0, 1292513741166.0, 1292513742166.0, 1292513743166.0, 1292513744166.0, 1292513745166.0, 1292513746167.0, 1292513747167.0, 1292513748166.0, 1292513749166.0, 1292513750166.0, 1292513751166.0, 1292513752167.0, 1292513753167.0, 1292513754166.0, 1292513755166.0, 1292513756167.0, 1292513757167.0, 1292513758166.0, 1292513759166.0, 1292513760166.0, 1292513761166.0, 1292513762166.0, 1292513763167.0, 1292513764167.0, 1292513765167.0, 1292513766167.0, 1292513767166.0, 1292513768166.0, 1292513769166.0, 1292513770166.0, 1292513771166.0, 1292513772167.0, 1292513773167.0, 1292513774166.0, 1292513775166.0, 1292513776166.0, 1292513777166.0 ], "updates": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ] }, "samplesCount": 60, "isPersistent": true, "lastTStamp": 1292513777166.0, "interval": 1000 }, "hot_keys": [ { "name": "48697", "ops": 0.0009276437847866419 }, { "name": "8487", "ops": 0.0009276437847866419 }, { "name": "77262", "ops": 0.0009276437847866419 }, { "name": "58495", "ops": 0.0009276437847866419 }, { "name": "21003", "ops": 0.0009276437847866419 }, { "name": "26850", "ops": 0.0009276437847866419 }, { "name": "73717", "ops": 0.0009276437847866419 }, { "name": "86218", "ops": 0.0009276437847866419 }, { "name": "80344", "ops": 0.0009276437847866419 }, { "name": "83457", "ops": 0.0009276437847866419 } ] }


101

Managing Bucket Resources

A new bucket may be created with a POST command to the URI to the buckets defined URI for the pool. This can be usedto create either a couchbase or a memcached type bucket. The bucket name cannot have a leading underscore.

When creating a bucket, an authType parameter must be specified:

• If the authType is "none". then a proxyPort number MUST be specified.

• If the authType is "sasl". then a "saslPassword" parameter MAY be optionally specified. In release 1.6.0, any SASLauth based access must go through the proxy which is fixed to port 11211.

The ramQuotaMB attribute allows the user to specify how much memory (in megabytes) will be allocated on each nodefor the bucket.

• In the case of memcached buckets, going beyond the ramQuotaMB will cause an item to be evicted on a mostly-LRUbasis. The type of item evicted may not be the exact LRU due to object size, whether or not it is currently being refer-enced or other situations.

• In the case of couchbase buckets, the system may return temporary failures if the ramQuotaMB is reached. The systemwill try to keep 25% of the available ramQuotaMB free for new items by ejecting old items from occupying memory. Inthe event these items are later requested, they will be retrieved from disk.

Creating a memcached Bucket

Request

POST /pools/default/buckets HTTP/1.1 Host: node.in.your.cluster:8091 Content-Type: application/x-www-form-urlencoded; charset=UTF-8 Authorization: Basic YWRtaW46YWRtaW4= Content-Length: xx name=newcachebucket&ramQuotaMB=128&authType=none&proxyPort=11216&bucketType=memcached

Response

202: bucket will be created asynchronously with the location header returned.

No URI to check for the completion task is available, but most bucket creations complete within a few seconds.

Example

curl -d name=newcachebucket -d ramQuotaMB=128 -d authType=none -d proxyPort=11216 -d bucketType=memcached http://localhost:8080/pools/default/buckets

Creating a Couchbase Bucket

In addition to the aforementioned parameters, a replicaNumber parameter specifies the number of replicas.

Request

POST /pools/default/buckets HTTP/1.1 Host: node.in.your.cluster:8080 Content-Type: application/x-www-form-urlencoded; charset=UTF-8 Authorization: Basic YWRtaW46YWRtaW4= Content-Length: xx name=newbucket&ramQuotaMB=20&authType=none&replicaNumber=2&proxyPort=11215

Response Example

202: bucket will be created.

curl -d name=newbucket -d ramQuotaMB=20 -d authType=none -d replicaNumber=2 -d proxyPort=11215 http://localhost:8080/pools/default/buckets

Getting a Bucket Request

GET /pools/default/buckets/Another bucket

Response

HTTP/1.1 200 OK Content-Type: application/com.couchbase.store+json Content-Length: nnn { "name" : "Another bucket" "bucketRules" : { "cacheRange" : { "min" : 1, "max" : 599 }, "replicationFactor" : 2 } "nodes" : [ { "hostname" : "10.0.1.20", "uri" : "/addresses/10.0.1.20", "status" : "healthy", "ports" : { "routing" : 11211, "kvcache" : 11311 } }, { "hostname" : "10.0.1.21", "uri" : "/addresses/10.0.1.21", "status" : "healthy", "ports" : { "routing" : 11211, "kvcache" : 11311 } } ] }


102

Clients MUST use the nodes list from the bucket, not the pool to indicate which are the appropriate nodes to connect to.

Modifying Bucket Properties

Buckets may be modified by POSTing the same kind of parameters used to create the bucket to the bucket's URI. Since anomitted parameter can be equivalent to not setting it in some cases, it is recommended that clients get existing parameters,make modifications where necessary, and then POST to the URI. The name of a bucket cannot be changed.

Increasing the Memory Quota for a Bucket

Increasing a bucket's ramQuotaMB from the current level. Note, the system will not let you decrease the ramQuotaMB fora couchbase bucket type and memcached bucket types will be flushed when the ramQuotaMB is changed.

Warning

As of 1.6.0, there are some known issues with changing the ramQuotaMB for memcached buckettypes.

Request (curl)

curl -d ramQuotaMB=25 -d authType=none -d proxyPort=11215 http://localhost:8080/pools/default/buckets/newbucket

Response

The response will be 202, indicating the quota will be changed asynchronously throughout the servers in the cluster.

HTTP/1.1 202 OK Server: Couchbase Server 1.6.0 Pragma: no-cache Date: Wed, 29 Sep 2010 20:01:37 GMT Content-Length: 0 Cache-Control: no-cache no-store max-age=0

Changing Bucket Authentication

Changing a bucket from port based authentication to SASL authentication can be done with:

curl -d ramQuotaMB=130 -d authType=sasl -d saslPassword=letmein http://localhost:8080/pools/default/buckets/acache

Cluster and Pool Operations

Creating a new pool is not currently supported. A new pool may be supported in future releases. At that time a pool can becreated by posting to the /pools URI.

Request

POST /pools/mynewpool name=mynewpool

Response

201: pool was created and valid URIs for referencing it returned 403: user not authorized (or no users are authorized because it is administratively disabled to all users)

Warning

For this release, this will always return 405 Method Not Allowed.

Joining a Cluster

Clusters (a.k.a. pools) cannot be merged if they are made of multiple nodes. However, a single node can be asked to join acluster. It will need several parameters to be able to negotiate a join to the cluster.

Request

POST /node/controller/doJoinCluster Host: target.node.to.do.join.from:8091 Authorization: Basic xxxxxxxxxxxx Accept: */* Content-Length: xxxxxxxxxx Content-Type: application/x-www-form-urlencoded clusterMemberHostIp=192.168.0.1&clusterMemberPort=8091&user=admin&password=admin123


103

The following arguments are required:

Table 8.5. Cluster Joining Arguments

Argument Description

clusterMemberHostIp Hostname or IP address to a member of the cluster the nodereceiving this POST will be joining

clusterMemberPort Port number for the RESTful interface to the system

If the server has been "secured" via the Couchbase Web Console, the following are also required

Table 8.6. Additional Arguments

Argument Description

user Administration user

password Password associated with the Administration user

Response

200 OK with Location header pointing to pool details of pool just joined - successful join 400 Bad Request - missing parameters, etc. 401 Unauthorized - credentials required, but not supplied 403 Forbidden bad credentials - invalid credentials

Example (curl)

curl -d clusterMemberHostIp=192.168.0.1 -d clusterMemberPort=8091 -d user=admin -d password=admin123 http://localhost:8091/node/controller/doJoinCluster

Ejecting a Node from a Cluster

In situations where a node is down either temporarily or permanently, we may need to eject it from the cluster. It may alsobe important to eject a node from another node participating in the same cluster.

Request

POST /controller/ejectNode Host: altnernate.node.in.cluster:8091 Authorization: Basic xxxxxxxxxxxx Accept: */* Content-Length: xxxxxxxxxx Content-Type: application/x-www-form-urlencoded [email protected]

Response

200 OK - node ejected 400 Error, the node to be ejected does not exist 401 Unauthorized - Credentials were not supplied and are required 403 Forbidden - Credentials were supplied and are incorrect

Example (curl)

curl -d [email protected] http://192.168.0.106:8091/controller/ejectNode

System Logs

System modules log various messages, which are available via this interface. These log messages are optionally catego-rized by the module. Both a generic list of recent log entries and recent log entries for a particular category are available.A GET without specifying a category returns all categories.

Returned types may be "info" "crit" or "warn". Accessing logs requires administrator credentials if the system is secured.

Request

GET /pools/default/logs?cat=crit Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1

Response

201: bucket was created and valid URIs returned HTTP/1.1 200 OK Content-Type: application/json Content-Length: nnn [{"cat":"info", "date": "", "code": "302", "message": "Some information for you."}, {"cat":"warn", "date": "", "code": "502", "message": "Something needs attention."}]

Client Logging Interface


104

Clients might want to add entries to the service's central logger. These entries would typically be responses to exceptionallike difficulty handling a server response. For instance, the Web UI uses this functionality to log client error conditions forlater diagnosis.

Request

POST /logClientError Host: node.in.your.pool.com Authorization: Basic xxxxxxxxxxxxxxxxxxx Accept: application/json X-memcachekv-Store-Client-Specification-Version: 0.1

Response

200 - OK

105

Chapter 9. Developing with CouchbaseYou can find information on client libraries for use with Couchbase Server at Develop with Couchbase Server SDKs

9.1. Use CasesCouchbase is a generalized database management system, but looking across Couchbase deployments, it is clear that thereare some patterns of use. These patterns tend to rely Couchbase's unique combination of linear, horizontal scalability; sus-tained low latency and high throughput performance; and the extensibility of the system facilitated through Tap and Node-Code. This page highlights these use cases.

9.1.1. Session store

User sessions are easily stored in Couchbase, such as by using a document ID naming scheme like "user:USERID". Theitem expiration feature of Couchbase can be optionally used to have Couchbase automatically delete old sessions. (Notethat expiration algorithm is lazy and will consume resources until the item is actually retrieved, at which point the expira-tion timestamps are checked.)

Besides the usual SET operation, CAS identifiers can be used to ensure concurrent web requests from a single user do notlose data.

Many web application frameworks such as Ruby on Rails and various PHP and Python web frameworks also provide pre-integrated support for storing session data using Memcached protocol. These are supported automatically by Couchbase.

9.1.2. Social gaming

Game state, property state, timelines, conversations & chats can also be modeled in Couchbase. The asynchronous persis-tence algorithms of Couchbase were designed, built and deployed to support some of the highest scale social games on theplanet. In particular, the heavy dual read & write storage access patterns of social games (nearly every user gesture mu-tates game state) is serviced by Couchbase by asynchronously queueing mutations for disk storage and also by collaps-ing mutations into the most recently queued mutation. For example, a player making 10 game state mutations in 5 sec-onds (e.g., planting 10 flowers in 5 seconds) will likely be collapsed by Couchbase automatically into just one queued diskmutation. Couchbase also will force-save mutated item data to disk, even if an item is heavily changed (the user keeps onclicking and clicking). Additionally, game state for that player remains instantly readable as long as it is in the memoryworking set of Couchbase.

9.1.3. Ad, offer and content targeting

The same underpinnings that power social games is well suited to real-time ad and content targeting. For example, Couch-base provides a fast storage capability for counters. Counters are useful for tracking visits, associating users with varioustargeting profiles (eg, user-1234 is visited a page about "automobiles" and "travel") and in tracking ad-offers and ad-in-ventory.

Multi-GET operations in Couchbase allow ad applications to concurrently "scatter-gather" against profiles, counters, orother items in order to allow for ad computation and serving decisions under a limited response latency budget.

9.1.4. Real-time logging and alerting

Other features of Couchbase, such as the ability to PREPEND and APPEND values onto existing items, allow for highperformance event tracking. Couchbase is also well suited as a aggregation backend, where events need to be consolidatedfor fast, real-time analysis.

For example, if your application needs to process the "firehose" of events from high-scale conversation services such asTwitter, such as by matching user interest in terms (eg, user-1234 is interested in conversations about "worldcup" and


Developing with Couchbase

106

"harrypotter"), Couchbase can be used as the database for fast topic to subscriber matching, allowing your application toquickly answer, "who is interested in event X?"

9.2. Best practices

Included below are a number of best practives that you should follow when developing applications using Couchbase.

9.2.1. What should I store in an object

Couchbase is most suited towards fast-changing data items of relatively small size. By relatively small, Couchbase inher-its Memcached's default configured 1 megabyte limit for each item value. For example, think shopping carts, user pro-file, user sessions, timelines, game states, pages, conversations and product catalog, instead of large audio or video mediablobs.

9.2.2. How should I store an object?

Couchbase, similar to Memcached, can store any binary bytes, and the encoding is up to you or your client library. Somememcached client libraries, for example, offer convenience functions to serialize/deserialize objects from your favoriteweb application programming language (Java, Ruby, PHP, Python, etc) to a blob for storage. Please consult your client li-brary API documentation for details.

An additional consideration on object encoding/seralization is whether your objects will need to be handled by multipleprogramming languages. For example, it might be inconvenient for a Java client application to decode a serialized PHPobject. In these cases, consider cross-language encodings such as JSON, XML, Google Protocol Buffers or Thrift.

The later two (Protocol Buffers and Thrift) have some advantages in providing more efficient object encodings than text-based encodings like JSON and XML. One key to Couchbase performance is to watch your working set size, so the moreworking set items you can fit into memory, the better.

On that note, some client libraries offer the additional feature of optionally compressing/decompressing objects stored in-to Couchbase. The CPU-time versus space tradeoff here should be considered, in addition to how you might want to ver-sion objects under changing encoding schemes. For example, you might consider using the 'flags' field in each item to de-note the encoding kind and/or optional compression. When beginning application development, however, a useful mantrato follow is to just keep things simple.

9.2.3. Objects that refer to other objects

Although Couchbase is a document store and you can store any byte-array value that you wish, there are some commonpatterns for handling items that refer to other items. Some example use cases. For example: User 1234 is interested in top-ics A, B, X, W and belongs to groups 1, 3, 5

• Shopping Cart 222 points to product-1432 and product-211

• A Page has Comments, and each of those Comments has an Author. Each Author, in turn, has a "handle", an avatar im-age and a karma ranking.

9.2.4. Nested Items

You can store serialized, nested structures in Couchbase, such as by using encodings like JSON or XML (or Google Proto-col Buffers or Thrift). A user profile item stored in Couchbase can then track information such as user interests. For exam-ple, in JSON...

{ "key": "user-1234", "handle": "bobama", "avatarURL": ...,"interests": [ "A", "B", "X", "W" ], "groups": [ 1, 3, 5 ], ... }


107

If the above is stored in Couchbase under document ID "user-1234", you can then know the interests for that user by doinga simple GET for user-1234 and decoding the JSON response.

9.2.5. Simple Lists

To handle reverse lookups (who are the users interested in topic X?), a common solution is to store simple lists. For exam-ple, under document ID "topic-X", you might have store the following list:

user-1234,user-222,user-987,

Such lists can be easily constructed by using Couchbase's APPEND or PREPEND operations, where you append/prependvalues that look like "user-XXXXX,".

Note that the list is delimited by commas, but that can be any character you choose.

9.2.5.1. Handling List Item Deletion

The above works when a user registers her interest in a topic, but how can you handle when a user wants to unregistertheir interest (eg, unsubscribe or unfollow)?

One approach is to use the CAS identifiers to do atomic replacement. A client application first does a GET-with-caS (a"gets" request in the ascii protocol) of the current list for a topic. Then the client removes the given user from the list re-sponse, and finally does a SET-with-CAS-identifier operation (a "cas" request in the ascii protocol) while supplying thesame CAS identifier that was returned with the earlier "gets" retrieval.

If the SET-with-CAS request succeeds, the client has successfully replaced the list item with a new, shorter list with therelevant list entry deleted.

The SET-with-CAS-identifier operation might fail, however, if another client mutated the list while the first client was at-tempting a deletion. In this case the first client can try to repeat the list item delete operation.

Under a highly contended or fast mutating list however (such as users trying to follow a popular user or topic), the delet-ing client will have a difficult time making progress. Some approaches to handle this situation are described next...

9.2.5.2. Handling Highly Contended List Item Deletion

Instead of performing a SET-with-CAS to perform list item deletion, one pattern is to explicitly track deleted items. Thiscould be done using APPEND for list additions and PREPENDS for list deletions, with an additional "tombstone" deletioncharacter. For example, anything before the "|" character is considered deleted:

user-222,|user-1234,user-222,user-987,

So, after the client library retrieves that list and does some post-processing, the effective, actual list of interested sub-scribers is user-1234 and user-987.

Care must be taken to count correctly, in case user-222 decides to add themselves again to the list (and her clicks are fasterthan whatever logic your application has to prevent duplicate clicks):

user-222,|user-1234,user-222,user-987,user-222

A similar encoding scheme would use '+' or '-' delimiter characters to the same effect, where the client sends an APPENDof "+ID" to add an entry to a list, and an APPEND of "-ID" to remove an entry from a list. The client application wouldstill perform post-processing on the list response, tracking appropriate list entry counts. In this and other encodings, wemust take care not to use the delimiter characters that were chosen:

+1234+222+987-222

Yet another variation on this would be store deleted items to a separate paired list. So your application might have twolists for a topic, such as a "follow-X" and "unfollow-X".


108

9.2.5.3. Compressing Lists

Eventually, your application may need to garbage collect or compress the lists. To do so, you might have your client appli-cation do so by randomly piggy-backing on other requests to retrieve the list.

Again, with heavily contended, fast mutating list, attempts to compress a list may be fruitless as SET-with-CAS attemptscan fail. Some solutions, as with many in software engineering, involve adding a level of indirection. For example, youcould keep two lists for each topic, and use marker items to signal to clients which list is considered active:

topic-X.a => +1234+222+987-222 topic-X.b => (empty)topic-X.active => topic-X.a

A client could multi-GET on topic-X.a and topic-X.b, and the combined result would contain the full list. To mutate thelist, the client would look at the "pointer" item of topic-X.active, and know to APPEND values to topic-X.a.

A randomly self-chosen client may choose to garbage-collect the active list when it sees the list length is large enough, bywriting a compressed version of topic-X.a into topic-X.b (note: XXX) and by flipping the topic-X.active item to point to"b". New clients will start APPEND'ing values to topic-X.b. Old, concurrent clients might still be APPEND'ing values tothe old active item of topic-X.a, so other randomly self-selected clients can choose to help continue to compress topic-X.ainto topic-X.b so that topic-X.a will be empty and ready for the next flip.

An alternative to a separate "topic-X.active" pointer item would be instead to PREPEND a tombstone marker value ontothe front of the inactivated list item. For example, if '^' was the tombstone marker character, all concurrent clients wouldbe able to see in that a certain list should not be appended to:

topic-X.a => +1234+222+987-222 topic-X.b => ^+1234

There are concurrency holes in this "active flipping" scheme, such as if there's a client process failure at the step notedabove at "XXX", so for periods of time there might be duplicates or reappearing list items.

In general, the idea is that independent clients try to make progress towards an eventually stabilized state. Please consideryour application use cases as to whether temporary inconsistencies are survivable.

9.2.5.4. Large Lists

If your lists get large (e.g., some user has 200,000 followers), you may soon hit the default 1 megabyte value byte sizelimits of Couchbase. Again, a level of indirection is useful here, by have another item that lists the lists...

topic-X => +0+1 topic-X.0 => ... many actual items ...topic-X.1 => ... more actual items ...

The "topic-X" item just lists pointers to items that have the actual lists.

In this approach, you could have randomly self-selected clients decide to add new topic sub-lists (topic-X.N) andAPPEND'ing updated info to the "index" item (topic-X).

Other randomly self-chosen clients could attempt to compress topic sub-lists that are old.

9.2.6. Multi-GET

Once your client application has a list of document IDs, the highest performance approach to retrieve the actual items is touse a multi-GET request. Doing so allows for concurrent retrieval of items across your Couchbase cluster. This will per-form better than a serial loop that tries to GET for each item individually and sequentially.

9.2.7. Locking

Advisory locks can be useful to control access to scarce resources. For example, retrieving information from backend orremote systems might be slow and consume a lot of resources. Instead of letting any client access the backend system and


109

potentially overwhelm the backend system with high concurrent client requests, you could create an advisory lock to allowonly one client at a time access the backend.

Advisory locks in Couchbase or Memcached can be created by setting expiration times on a named data item and by usingthe 'add' and 'delete' commands to access that named item. The 'add' or 'delete' commands are atomic, so you can be knowthat only one client will become the advisory lock owner.

The first client that tries to ADD a named lock item (with an expiration timeout) will succeed. Other clients will see er-ror responses to an ADD command on that named lock item, so they can know that some other client is owning the namedlock item. When the current lock owner is finished owning the lock, it can send an explicit DELETE command on thenamed lock item to free the lock.

If the lock owning client crashes, the lock will automatically become available to the next client that polls for the lock (us-ing 'add') after the expiration timeout.

9.2.8. Data partitioning with buckets

Couchbase allows you to partition your data into separate containers or namespaces. These containers are called 'buckets'.Couchbase will keep item storage separated for different buckets, allowing you to perform operations like statistics gather-ing and flush_all on a per-bucket basis, which are not workable using other techniques such as simulating namespaces bydocument ID-prefixing.

Couchbase Server supports two different bucket types, Couchbase and memcached. For a full discussion of the major dif-ferences, see Section 11.6, “Buckets”.

9.3. Couchbase for Memcached Users

Configuring Couchbase as Memcached

While Couchbase Server is completely compatible with the open-source memcached protocol, we realize that thereare still good use cases for using a cache. For this reason, we have included standard memcached functionality into theCouchbase Server product. Simply configure a bucket to be of type "Memcached" and it will behave almost identically tothe open source version. There are a few differences around memory management but your clients and applications willnot see a difference.

Q: What are the behavioral differences between Couchbase and Memcached?

A: The biggest difference is that Couchbase is a database. It will persist your data and return an error if there is not enoughRAM or disk space available. Memcached is a cache, and will evict older data to make room for new data. Couchbase al-so provides replication so that you always have access to your data in the event of a failure. Memcached runs only out ofRAM and has no replication so the loss of a server will result in the loss of that cache.

Q: What are the advantages to using this Memcached over the open source version?

A: We provide a much enhanced UI for the purposes of configuration and monitoring. Also, through the use of "smartclients", your application can be dynamically updated with cluster topology changes. Using this server also gives you aneasy path to upgrade to a Couchbase bucket type for the enhanced HA, persistence and querying capabilities.

110

Chapter 10. Developing Couchbase ClientsA couchbase client library implemention should be very similar to a memcached(binary protocol) client library imple-mentation (and might even be an extension of some existing memcached binary-protocol client library), but just supportsa different key hashing approach. Instead of using modulus or ketama/consistent hashing, the new hashing approach incouchbase is instead based around "vbuckets", which you can read up more about here

In the vBucket approach, to find a server to talk to for a given key, your client library should hash the key string into avBucket-Id (a 16-bit integer). The default hash algorithm for this step is plain old CRC, masked by the required numberof bits. The vBucket-Id is then used as an array index into a server lookup table, which is also called a vBucket-to-servermap. Those two steps will allow your client library to find the right couchbase server given a key and a vBucket-to-servermap. This extra level of indirection (where we have an explicit vBucket-to-server map) allows couchbase to easily controlitem data rebalancing, migration and replication.

10.1. REST/JSON

The mapping between the vBucket designation and the client interface operates through a request to the Couchbase Serv-er via client-initiated REST/JSON requests. The REST/JSON URLs should be provided to the client library as some initialconfiguration parameter by the user's application. The client application should bootstrap the REST/JSON information by"chasing URLs" from a standard base, starting URL. For more information on REST/JSON "bootstrapping" and chasingURLs, please follow the links at Chapter 8, Couchbase Management REST API.

Eventually, after following the bootstrapping sequence, your client library will have a REST/JSON URL that looks like:

http://HOST:PORT/pools/default/bucketsStreaming/BUCKET_NAME

For example:

http://couchbase1:8080/pools/default/bucketsStreaming/default

Here's an example response when requesting that URL, in JSON:

{ "name" : "default", "bucketType" : "couchbase",... "vBucketServerMap" : { "hashAlgorithm" : "CRC", "numReplicas" : 1, "serverList" : ["10.1.2.14:11210"], "vBucketMap" : [[0,-1],[0,-1],[0,-1],[0,-1],[0,-1] : ] }}

The REST/JSON URLs might be under HTTP Basic Auth authentication control, so the client application may also haveto provide (optional) user/password information to the your client library so that the proper HTTP/REST request can bemade.

The REST/JSON URLs are "streaming", in that the couchbase REST server doesn't close the HTTP REST connection af-ter responding with one vBucket-to-server map. Instead, couchbase keeps the connection open, and continues to streamnew maps to your client library when there are cluster changes (new server nodes being added, removed, and/or vBucketsgetting reassigned to different servers). In the couchbase streaming approach, new vBucket-to-server map JSON messagesare delimited by 4 newlines ("\n\n\n\n") characters.

The above section describes what we call "per-bucket" REST/JSON URLs. That is, each port-based bucket has a stream-ing REST/JSON URL of the form:

http://HOST:PORT/pools/default/bucketsStreaming/BUCKET_NAME

http://dustin.github.com/2010/06/29/memcached-vbuckets.html

Developing Couchbase Clients

111

There is another kind of REST/JSON URL, which describes all SASL-authenticated buckets. This SASL REST/JSONURL has the form of:

http://HOST:PORT/pools/default/saslBucketsStreaming

Sample output:

{"buckets":[ {"name":"default", "nodeLocator":"vbucket", "saslPassword":"", "nodes":[ {"clusterMembership":"active","status":"healthy","hostname":"10.1.4.11:8091", "version":"1.6.1rc1","os":"x86_64-unknown-linux-gnu", "ports":{"proxy":11211,"direct":11210}}, {"clusterMembership":"active","status":"healthy","hostname":"10.1.4.12:8091", "version":"1.6.1pre_21_g5aa2027","os":"x86_64-unknown-linux-gnu", "ports":{"proxy":11211,"direct":11210}}], "vBucketServerMap":{ "hashAlgorithm":"CRC","numReplicas":1, "serverList":["10.1.4.11:11210","10.1.4.12:11210"], "vBucketMap":[[0,-1],[1,-1],...,[0,-1],[0,-1]]}}

] }

One main difference between the SASL REST/JSON response versus the per-bucket REST/JSON response is that theSASL REST/JSON response can describe more than one bucket. In the SASL REST/JSON response, these multiple buck-ets would be found under the "buckets" array.

10.1.1. Parsing the JSON

Once your client library has received a complete vBucket-to-server map message, it should use its favorite JSON parser toprocess the map into more useful data structures. An implementation of this kind of JSON parsing in C exists as a helperlibrary in libvbucket, or for Java, jvbucket.

The libvbucket and jvbucket helper libraries don't do any connection creation, socket management, protocol se-rialization, etc. That's the job of your higher-level library. These helper libraries instead just know how to parse a JSONvBucket-to-server map and provide an API to access the map information.

10.1.1.1. The vBucketMap

The vBucketMap value within the returned JSON describes the vBucket organization. For example:

"serverList":["10.1.4.11:11210","10.1.4.12:11210"], "vBucketMap":[[0,1],[1,0],[1,0],[1,0],:,[0,1],[0,1]]

The vBucketMap is zero-based indexed by vBucketId. So, if you have a vBucket whose vBucketId is 4, you'd look upvBucketMap[4]. The entries in the vBucketMap are arrays of integers, where each integer is a zero-based index into theserverList array. The 0th entry in this array describes the primary server for a vBucket. Here's how you read this stuff,based on the above config:

The vBucket with vBucketId of 0 has a configuration of vBucketMap[0], or[0, 1]. So vBucket 0's primary server isat serverList[0], or 10.1.4.11:11210.

While vBucket 0's first replica server is at serverList[1], which is 10.1.4.12:11210.

The vBucket with vBucketId of 1 has a configuration of vBucketMap[1], or[1, 0]. So vBucket 1's primaryserver is at serverList[1], or 10.1.4.12:11210. And vBucket 1's first replica is at serverList[0], or10.1.4.11:11210.

This structure and information repeats for every configured vBucket.

If you see a -1 value, it means that there is no server yet at that position. That is, you might see:

http://github.com/membase/libvbucket

http://github.com/membase/jvbucket


112

"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],:]

Sometimes early before the system has been completely configured, you might see variations of:

"serverList":[], "vBucketMap":[]

10.1.2. Encoding the vBucketId into each request

As the user's application makes item data API invocations on your client library (mc.get("some_key"),mc.delete("some_key"), your client library will hash the key ("some_key") into a vBucketId. Your client library must alsoencode a binary request message (following memcached binary protocol), but also also needs to include the vBucketId aspart of that binary request message.

Note

Python-aware readers might look at this implementation for an example.

Each couchbase server will double-check the vBucketId as it processes requests, and would return NOT_MY_VBUCKETerror responses if your client library provided the wrong vBucketId to the wrong couchbase server. This mismatch is ex-pected in the normal course of the lifetime of a cluster -- especially when the cluster is changing configuration, such asduring a Rebalance.

10.1.3. Client Libraries and Rebalancing

A major operation in a cluster of couchbase servers is called Rebalancing. A couchbase system administrator may chooseto initiate a Rebalance because new servers might have been added, old servers need to be decommissioned and need to beremoved, etc. An underlying part of Rebalancing is the controlled migration of vBuckets (and the items in those migratingvBuckets) from one couchbase server to another.

There's a window of time, given the distributed nature of couchbase servers and clients, where vBuckets ownership mayhave changed and migrated from one server to another server, but your client library has not heard the news yet. So, yourclient library would be trying to talk to the 'wrong' or outdated server for a given item, since your client library is operat-ing with an out-of-date vBucket-to-server map.

Below is a walk through of this situation in more detail and how to handle this case:

Before the Rebalance starts, any existing, connected clients should be operating with the cluster's pre-Rebalance vBuck-et-to-server map.

As soon as the Rebalance starts, couchbase will "broadcast" (via the streaming REST/JSON channels) a slightly updatedvBucket-to-server map message. The assignment of vBuckets to servers doesn't really change at this point at the start ofthe Rebalance, but the serverList of all the servers in the couchbase cluster does change. That is, vBuckets have not yetmoved (or are just starting to move), but now your client library knows the addresses of any new couchbase servers thatare now part of the cluster. Knowing all the servers in the cluster (including all the newly added servers) is important, asyou'll soon see.

At this point, the couchbase cluster will be busy migrating vBuckets from one server to another.

Concurrently, your client library will be trying to do item data operations (Get/Set/Delete's) using its pre-RebalancevBucket-to-server map. However, some vBuckets might have been migrated to a new server already. In this case, the serv-er your client library was trying to use will return a NOT_MY_VBUCKET error response (as the server knows the vBuck-etId which your client library encoded into the request).

Your client library should handle that NOT_MY_VBUCKET error response by retrying the request against another serverin the cluster. The retry, of course, might fail with another NOT_MY_VBUCKET error response, in which your client li-brary should keep probing against another server in the cluster.

http://github.com/membase/ep-engine/blob/master/management/mc_bin_client.py


113

Eventually, one server will respond with success, and your client library has then independently discovered the new, cor-rect owner of that vBucketId. Your client library should record that knowledge in its vBucket-server-map(s) for use in fu-ture operations time.

An implementation of this can be seen in the libvBucket API vbucket_found_incorrect_master().

The following shows a swim-lane diagram of how moxi interacts with libvBucket during NOT_MY_VBUCKET errorslibvbucket_notmyvbucket.pdf .

At the end of the Rebalance, the couchbase cluster will notify streaming REST/JSON clients, finally, with a new vBuck-et-to-server map. This can be handled by your client library like any other vBucket-to-server map update message. How-ever, in the meantime, your client library didn't require granular map updates during the Rebalancing, but found the cor-rect vBucket owners on its own.

10.1.4. Fast Forward Map

A planned, forthcoming improvement to the above NOT_MY_VBUCKET handling approach is that couchbase will soonsend an optional second map during the start of the Rebalance. This second map, called a "fast forward map", providesthe final vBucket-to-server map that would represent the cluster at the end of the Rebalance. A client library can use theoptional fast forward map during NOT_MY_VBUCKET errors to avoid linear probing of all servers and can instead justjump straight to talking with the eventual vBucket owner.

Please see the implementation in libvBucket that handles a fast-forward-map here.

The linear probing, however, should be retained by client library implementations as a good fallback, just-in-case errorhandling codepath.

10.1.5. Redundancy & Availability

Client library authors should allow their user applications to specify multiple URLs into the couchbase cluster for redun-dancy. Ideally, the user application would specify an odd number of URLs, and the client library should compare respons-es from every REST/JSON URL until is sees a majority of equivalent cluster configuration responses. With an even num-ber of URLs which provide conflicting cluster configurations (such as when there's only 2 couchbase servers in the clus-ter and there's a split-brain issue), the client library should provide an error to the user application rather than attempting toaccess items from wrong nodes (nodes that have been Failover'ed out of the cluster).

The libvBucket C library has an API for comparing two configurations to support these kinds of comparisons. See thevbucket_compare()function here.

As an advanced option, the client library should keep multiple REST/JSON streams open and do continual "majority vote"comparisons between streamed configurations when there are re-configuration events.

As an advanced option, the client library should "learn" about multiple cluster nodes from its REST/JSON responses. Forexample, the user may have specified just one URL into a multi-node cluster. The REST/JSON response from that onenode will list all other nodes, which the client library can optionally, separately contact. This allows the client library toproceed even if the first URL/node fails (as long as the client library continues running).

10.2. SASL Authentication Example

In order to connect to a given bucket you need to run a SASL authentication to the memcached server. The SASL authen-tication for memcached is specified in SASLAuthProtocol (binary protocol only).

vbucketmigrator implements SASL Authentication by using libsasl in C if you want some example code.

http://www.couchbase.com/wiki/download/attachments/3342379/libvbucket_notmyvbucket.pdf?version=2&modificationDate=1289323788000

http://github.com/membase/libvbucket/blob/master/tests/testapp.c#L67

http://github.com/membase/libvbucket/blob/master/include/libvbucket/vbucket.h

http://code.google.com/p/memcached/wiki/SASLAuthProtocol

http://github.com/membase/vbucketmigrator


114

10.2.1. List Mechanisms

We start the SASL authentication by asking the memcached server for the mechanisms it supports. This is achieved bysending the following packet:

Byte/ 0 | 1 | 2 | 3 |/ | | | ||0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|+---------------+---------------+---------------+---------------+0| 80 | 20 | 00 | 00 |+---------------+---------------+---------------+---------------+4| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+8| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+12| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+16| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+20| 00 | 00 | 00 | 00 |

Header breakdownField (offset) (value)Magic (0): 0x80 (PROTOCOL_BINARY_REQ)Opcode (1): 0x20 (sasl list mechs)Key length (2-3): 0x0000 (0)Extra length (4): 0x00Data type (5): 0x00vBucket (6-7): 0x0000 (0)Total body (8-11): 0x00000000 (0)Opaque (12-15): 0x00000000 (0)CAS (16-23): 0x0000000000000000 (0)

If the server supports SASL authentication the following packet is returned:

Byte/ 0 | 1 | 2 | 3 |/ | | | ||0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|+---------------+---------------+---------------+---------------+0| 81 | 20 | 00 | 00 |+---------------+---------------+---------------+---------------+4| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+8| 00 | 00 | 00 | 05 |+---------------+---------------+---------------+---------------+12| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+16| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+20| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+24| 50 ('P') | 4c ('L') | 41 ('A') | 49 ('I') |+---------------+---------------+---------------+---------------+

28| 4e ('N') |

Header breakdownField (offset) (value)Magic (0): 0x81 (PROTOCOL_BINARY_RES)Opcode (1): 0x20 (sasl list mechs)Key length (2-3): 0x0000 (0)Extra length (4): 0x00Data type (5): 0x00Status (6-7): 0x0000 (SUCCESS)Total body (8-11): 0x00000005 (5)Opaque (12-15): 0x00000000 (0)CAS (16-23): 0x0000000000000000 (0)Mechanisms (24-28): PLAIN

Please note that the server may support a different set of mechanisms. The list of mechanisms is a space-separated list ofSASL mechanism names (e.g. "PLAIN CRAM-MD5 GSSAPI").


115

10.2.2. Authentication request

After choosing the desired mechanism (from the ones that the server supports), you need to create an authentication re-quest packet and send it to the server. The following packet shows a packet using PLAIN authentication of "foo" with thepassword "bar":

Byte/ 0 | 1 | 2 | 3 |/ | | | ||0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|+---------------+---------------+---------------+---------------+0| 80 | 21 ('!') | 00 | 05 |+---------------+---------------+---------------+---------------+4| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+8| 00 | 00 | 00 | 10 |+---------------+---------------+---------------+---------------+12| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+16| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+20| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+24| 50 ('P') | 4c ('L') | 41 ('A') | 49 ('I') |+---------------+---------------+---------------+---------------+28| 4e ('N') | 66 ('f') | 6f ('o') | 6f ('o') |+---------------+---------------+---------------+---------------+32| 00 | 66 ('f') | 6f ('o') | 6f ('o') |+---------------+---------------+---------------+---------------+36| 00 | 62 ('b') | 61 ('a') | 72 ('r') |

Header breakdownField (offset) (value)Magic (0): 0x80 (PROTOCOL_BINARY_REQ)Opcode (1): 0x21 (sasl auth)Key length (2-3): 0x0005 (5)Extra length (4): 0x00Data type (5): 0x00vBucket (6-7): 0x0000 (0)Total body (8-11): 0x00000010 (16)Opaque (12-15): 0x00000000 (0)CAS (16-23): 0x0000000000000000 (0)Mechanisms (24-28): PLAINAuth token (29-39): foo0x00foo0x00bar

If the server accepts this username/password combination, it may return one of two status codes: Success or "Authentica-tion Continuation". Success means that you're done

Byte/ 0 | 1 | 2 | 3 |/ | | | ||0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|+---------------+---------------+---------------+---------------+0| 81 | 21 ('!') | 00 | 00 |+---------------+---------------+---------------+---------------+4| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+8| 00 | 00 | 00 | 0d |+---------------+---------------+---------------+---------------+12| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+16| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+20| 00 | 00 | 00 | 00 |+---------------+---------------+---------------+---------------+24| 41 ('A') | 75 ('u') | 74 ('t') | 68 ('h') |+---------------+---------------+---------------+---------------+28| 65 ('e') | 6e ('n') | 74 ('t') | 69 ('i') |+---------------+---------------+---------------+---------------+32| 63 ('c') | 61 ('a') | 74 ('t') | 65 ('e') |+---------------+---------------+---------------+---------------+36| 64 ('d') |


116

Header breakdownField (offset) (value)Magic (0): 0x81 (PROTOCOL_BINARY_RES)Opcode (1): 0x21 (sasl auth)Key length (2-3): 0x0000 (0)Extra length (4): 0x00Data type (5): 0x00Status (6-7): 0x0000 (SUCCESS)Total body (8-11): 0x0000000d (13)Opaque (12-15): 0x00000000 (0)CAS (16-23): 0x0000000000000000 (0)Info (24-36): Authenticated

117

Chapter 11. Couchbase ArchitectureFrom a client perspective, couchbase speaks memcached protocol, which is well understood by many, if not most applica-tion developers. The difference, of course, is that couchbase has persistence and replication capabilities while still allow-ing for memcached like speed.

11.1. Cluster DesignIndividual couchbase nodes are clustered together. Within a cluster data is automatically replicated between nodes of acluster. Cluster nodes can be added and removed without interrupting access to data within the cluster.

All clusters start with a single node, typically one installed from a package. Either through the Web UI or from the RESTinterface, couchbase allows one or more nodes to be added to the cluster. When a node is added to the cluster, it does notimmediately start performing data operations. This is to allow the user to perform one or more changes to the cluster be-fore initiating a rebalance. During a rebalance the data and replicas of that data, contained in sub-partitions of the clustercalled vBuckets, are redistributed throughout the cluster. By design, a given vBucket is only active in one place within thecluster at a given point in time. By doing so, couchbase is always consistent for any given item.

Data is moved between nodes, both when rebalancing and replicating, using a set of managed vBucketmigrator process-es in the cluster. This process uses a new protocol called TAP. TAP is generic in nature though, and it has very clear usecases outside replication or migration of data. Incidentally, TAP doesn't actually stand for anything. The name came aboutwhen thinking about how to "tap into" a couchbase node. This could be thought of along the lines of a 'wiretap' or tappinginto a keg.

Cluster replication defaults to asynchronous, but is designed to be synchronous. The benefit of replication being asynchro-nous is that couchbase has speeds similar to memcached in the default case, taking a data safety risk for a short interval.

Cluster coordination and communication is handled by the ns_server erlang process. Generally, users of couchbase neednot be aware of the details about how ns_server performs its tasks, as interfacing with the cluster is done with the afore-mentioned couchbase REST interface. As part of keeping the system simple, all nodes of the cluster expose the state of thecluster.

11.2. Persistence DesignGenerally speaking, Couchbase Server is memory oriented, by which we mean that it tends to be designed around theworking set being resident in memory, as is the case with most highly interactive web applications. However, the set ofdata in memory at any given point in time is only the hot data. Data is persisted to disk by Couchbase Server asynchro-nously, based on rules in the system.

11.3. Component OverviewFrom a developer perspective, it is useful to know how all of the components of couchbase come together. A couchbasenode consists of

• ns_server

This is the main process that runs on each node. As it says in it's source repository summary, it is the supervisor. One ofthese runs on each node and then spawns processes, which then later spawn more processes.

• menelaus

Menelaus is really two components, which are part of the ns_server repository. The main focus of menelaus is provid-ing the RESTful interface to working with a cluster. Built atop that RESTful interface is a very rich, complex JQuerybased application which makes REST calls to the server.

Couchbase Architecture

118

• memcached

Though couchbase is different than memcached, it does leverage the core of memcached. The core includes network-ing and protocol handling.

The bulk of couchbase is implemented in two components:

• couchbase engine (ep-engine)

This is loaded through the memcached core and the bucket_engine. This core component provides persistence in anasynchronous fashion and implements the TAP protocol.

• bucket engine

The bucket engine provides a way of loading instances of engines under a single memcached process. This is howcouchbase provides multitenancy.

• vbucketmigrator

Effectively a TAP client, based on how ns_server starts one or more vBucketmigrator processes, data is either replicatedor transferred from one node to another.

• Moxi

A memcached proxy, moxi "speaks" vBucket hashing (implemented in libvbucket) and can talk to the REST interfaceto get cluster state and configuration, ensuring that clients are always routed to the appropriate place for a given vBuck-et.

Across multiple cloud instances, VMs or physical servers, all of these components come together to become a couch-base cluster.

11.4. Disk Storage (Growing Data Sets Beyond Memory)

Couchbase Server has asynchronous persistence as a feature. One feature-of-that-feature is that the working set stored byan individual couchbase node can be larger than the cache dedicated to that node. This feature is commonly referred to as"disk greater than memory".

11.4.1. Design

Each instance of ep-engine in a given node will have a certain memory quota associated with it. This memory quota issometimes referred to as >the amount of cache memory. That amount of memory will always store the index to the entireworking set. By doing so, we ensure most items are quickly fetched and checks for the existence of items is always fast.


119

Figure 11.1. Bucket Configuration

In addition to the quota, there are two watermarks the engine will use to determine when it is necessary to start freeing upavailable memory. These are mem_low_wat and mem_high_wat.

As the system is loaded with data, eventually the mem_low_wat is passed. At this time, no action is taken. This is the"goal" the system will move toward when migrating items to disk. As data continues to load, it will evenutally reachmem_high_wat. At this point a background job is scheduled to ensure items are migrated to disk and the memory is thenavailable for other couchbase items. This job will run until measured memory reaches mem_low_wat. If the rate of incom-ing items is faster than the migration of items to disk, the system may return errors indicating there is not enough space.This will continue until there is available memory.

11.4.2. Consequences of Memory faster than Disk

Obviously, the migration of data to disk is generally much slower and has much lower throughput than setting things inmemory. When an application is setting or otherwise mutating data faster than it can be migrated out of memory to makespace available for incoming data, the behavior of the server may be a bit different than the client expects with mem-cached. In the case of memcached, items are evicted from memory, and the newly mutated item is stored. In the case ofcouchbase, however, the expectation is that we'll migrate items to disk.

When Membase determines that there is not enough memory to store information immediately, the server will returnTMP_OOM, the temporary out of memory error. This is designed to indicate that the inability to store the requested infor-mation is only a temporary, not a permanent, lack of memory. When the client receives this error, the storage process caneither be tried later or fail, dependending on the client and application requirements.

11.4.2.1. DGM Implementation Details

The actual process of eviction is relatively simple now. When we need memory, we look around in hash tables and at-tempt to find things we can get rid of (i.e. things that are persisted on disk) and start dropping it. We will also eject data assoon as it's persisted iff it's for an inactive (e.g. replica) vBucket if we're above our low watermark for memory. If we haveplenty of memory, we'll keep it loaded.

The bulk of this page is about what happens when we encounter values that are not resident.

11.4.2.1.1. Get Flow

In the current flow, a get request against a given document ID will first fetch the value from the hash table. For any giv-en item we know about, there will definitely be a document ID and its respective metadata will always be available in the


120

hash table. In the case of an "ejected" record, the value will be missing, effectively pointed to NULL. This is useful forlarger objects, but not particularly efficient for small objects. This is being addressed in future versions.

When fetching a value, we will first look in the hash table. If we don't find it, we don't have it. MISS.

If we do have it and it's resident, we return it. HIT.

If we have it and it's not resident, we schedule a background fetch and let the dispatcher pull the object from the DB andreattach it to the stored value in memory. The connection is then placed into a blocking state so the client will wait untilthe item has returned from slower storage.

The background fetch happens at some point in the future via an asynchronous job dispatcher.

Figure 11.2. Background Fetch Workflow

When the job runs, the item is returned from disk and then the in-memory item is pulled and iff it is still not resident, willhave the value set with the result of the disk fetch.*

Once the process is complete, whether the item was reattached from the disk value or not, the connection is reawakened sothe core server will replay the request from the beginning.

It's possible (though very unlikely) for another eject to occur before this process runs in which case the entire fetch processwill begin again. The client has no particular action to take after the get request until the server is able to satisfy it.

Note

An item may be resident after a background fetch either in the case of another background fetch forthe same document ID having completed prior to this one or another client has modified the valuesince we looked in memory. In either case, we assume the disk value is older and will discard it.

11.4.2.1.2. Concurrent Reads and Writes

Concurrent reads and writes are sometimes possible under the right conditions. When these conditions are met, reads areexecuted by a new dispatcher that exists solely for read-only database requests, otherwise, the read-write dispatcher isused.

The underlying storage layer reports the level of concurrency it supports at startup time (specifically, post init-script evalu-ation). For stock SQLite, concurrent reads are allowed if both the journal-mode is WAL and read_uncommitted is enabled.


121

Future storage mechanisms may allow for concurrent execution under different conditions and will indicate this by report-ing their level of concurrency differently.

The concurrentDB engine parameter allows the user to disable concurrent DB access even when the DB reports it's possi-ble.

The possible concurrency levels are reported via the ep_store_max_concurrency, ep_store_max_readersand, ep_store_max_readwrite stats. The dispatcher stats will show the read-only dispatcher when it's available.

11.4.2.1.3. Mutation Flow

New data is better than old data, so a set always wins. Similarly, a delete always wins. Increment, decrement, add, etc areall atomic, but you can imagine them working as a get + store.

11.5. Couchbase APIs

A Couchbase cluster communicates with clients in two ways; the primary way clients interact with couchbase is throughmanipulating data through various operations supported by couchbase. This is always through memcached protocol, andalmost always through a client written for the particular programming language and platform you use.

In addition, there is also a RESTful interface which allows so-called "control plane" management of a cluster. Throughthis, a client may get information about or make changes to the cluster. For example, with the REST interface, a client cando things such as gather statistics from the cluster, define and make changes to buckets and even add/remove new nodes tothe cluster.

11.5.1. memcached protocol

Couchbase supports the textual memcached protocol as described in protocol.txt. The textual protocol is disabled for thedirect port to couchbase due to the lack of vBucket support in couchbase. All access to couchbase with the textual protocolmust go through moxi.

One minor difference with Couchbase compared to memcached is that Couchbase allows for larger item sizes. Wherememcached is 1MByte by default (tunable in more recent versions), Couchbase defaults to a maximum item size of20MByte.

11.5.1.1. Protocol extensions

In addition to the commands described in the link above, Couchbase supports the following command extensions:

• getl

getlis a "get with lock", and is under active development. The behavior and command associated with getl are de-scribed on getl extended operations.

11.5.1.2. Binary protocol

Couchbase supports the binary memcached protocol as described on the wiki for the memcached project except for therange query commands. To date, there are no implementations of the range query commands known, just a draft specifica-tion.

11.5.1.2.1. Protocol extensions

In addition to the textual commands described in the link above, Couchbase supports the following command extensions:

http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol


122

• CMD_STOP_PERSISTENCE

• CMD_START_PERSISTENCE

• CMD_SET_FLUSH_PARAM

• CMD_SET_VBUCKET

• CMD_GET_VBUCKET

• CMD_DEL_VBUCKET

• CMD_START_REPLICATION

• CMD_STOP_REPLICATION

• CMD_SET_TAP_PARAM

• CMD_EVICT_KEY

11.5.1.3. memcapable 1.0

memcapable is a tool included in lib memcached that is used to verify if a memcached implementation adheres to thememcached protocol. It does this by sending all of the commands specified in the protocol description and verifies theresult. This means that the server must implement an actual item storage and all of the commands to be able to pass thememcapable testsuite.

11.5.1.3.1. Command line options

memcapable supports a number of command line options you may find useful (try running memcapable-h to see the listof available options). If you run memcapable without any options it will try to connect to localhost:11211and runthe memcapable testsuite (see Section 11.5.1.3.2, “Example”). If you're trying to implement your own server and one ofthe tests fails, you might want to know why it failed. There is two options you might find useful for that: -v or-c. The -voption prints out the assertion why the test failed, and may help you figure out the problem. I'm a big fan of debuggers andcorefiles, so I prefer -c. When using-c memcapable will dump core whenever a test fails, so you can inspect the corefileto figure out why the test failed.

11.5.1.3.2. Example

The following example tests the server listening on port 11211 on the local host (in this example I've got the stock mem-cached server running there)


123

trond@opensolaris> memcapableascii quit [pass]ascii version [pass]ascii verbosity [pass]ascii set [pass]ascii set noreply [pass]ascii get [pass]ascii gets [pass]ascii mget [pass]ascii flush [pass]ascii flush noreply [pass]ascii add [pass]ascii add noreply [pass]ascii replace [pass]ascii replace noreply [pass]ascii CAS [pass]ascii CAS noreply [pass]ascii delete [pass]ascii delete noreply [pass]ascii incr [pass]ascii incr noreply [pass]ascii decr [pass]ascii decr noreply [pass]ascii append [pass]ascii append noreply [pass]ascii prepend [pass]ascii prepend noreply [pass]ascii stat [pass]binary noop [pass]binary quit [pass]binary quitq [pass]binary set [pass]binary setq [pass]binary flush [pass]binary flushq [pass]binary add [pass]binary addq [pass]binary replace [pass]binary replaceq [pass]binary delete [pass]binary deleteq [pass]binary get [pass]binary getq [pass]binary getk [pass]binary getkq [pass]binary incr [pass]binary incrq [pass]binary decr [pass]binary decrq [pass]binary version [pass]binary append [pass]binary appendq [pass]binary prepend [pass]binary prependq [pass]binary stat [pass]All tests passed

The following example runs the test named "binary prepend"

trond@opensolaris> memcapable -T "binary prepend"binary prepend [pass]All tests passed

The following example runs the test suite, but prompts the user before each test

trond@opensolaris> memcapable -P ascii quitPress <return> when you are ready?ascii quit [pass]ascii versionPress <return> when you are ready? quit


124

11.5.2. Additional Protocol Operations

Couchbase has a method of adding new operations and extending protocol. This allows developers and contributors toexperiment with new capabilities, then either add them to Couchbase as an extension or contribute it to the memcachedcore, as required.

11.5.2.1. TAP Protocol

The TAP protocol provides an interface to the changes that are occuring on your Couchbase Server.

11.5.2.2. getl

getl/unl: Get an item with a lock that has a timeout. It can then be unlocked with either a CAS operation or with an ex-plicit unlock command.

Note

Note, this documentation and functionality are still under development and may not necessarilymatch what you see here.

Through an extension, Couchbase has a feature allowing individual clients to be sure they have exclusive access to an itemfor a period of time. This is not yet supported functionality, and should be considered experimental.

Thegetlcommand takes two arguments a document ID and an expiration time. If no expiration time is set then the de-fault expiration is 15 seconds.getl also has a maximum expiration time of 29 seconds so any expiration time greater than29 seconds is automatically changed to 29 seconds. Below are four use cases that describe how getl works. In these usecases we use the set operation as an example of an operation that is blocked when a getl is applied to a document ID.Note however that the operations delete, incr, etc. also will be blocked by getl.

11.5.2.2.1. getl Use Cases

There are a number of potential use cases for the getl operation:

1. Timeout locks the document for the specified time

• Set a document

• Perform agetl on the document ID with a timeout of 15 seconds

• Attempt to set the document with a new value

• Should get "EXISTS"

• Perform a get on the document ID to verify the document has its original value

• Wait 15 seconds

• Attempt to set the document ID with a new value

• Perform a get on the document ID to verify the document has the new value

2. CAS unlocks the document before the timeout

• Set a document

• Perform a getl on the document ID with a timeout of 15 seconds

• Attempt to CAS the document ID with a using the appropriate cas id


125

• Perform a get on the document ID to verify the document has its new value

• Attempt to set the document with a new value


3. getl fails against a locked document ID

• Set a document ID


• Perform a getl without a timeout

• Should get "LOCK_ERROR"

4. getl honors an old timeout

• Set a document ID

• Perform a getl on the document ID with a timeout of 3600 seconds (Will be set to 29 seconds by Couchbase)


• Wait 5 seconds

• Perform a set on the document ID with a new value

• Set should fail with "EXISTS"

• Wait 24 seconds

• Perform a set on the document ID with a new value

• Set should pass


11.6. Buckets

Buckets are used to compartmentalize data within Couchbase Server and are also used as the basic mechanism used toreplicate and duplicate information (if supported). Couchbase Server supports two different bucket types. These are:

• memcached Buckets

The memcached buckets are designed to fully support the core memcached protocol as an in-memory caching solu-tion. The support and functionality is therefore limited to the same functionality as within a standalone memcached im-plementation.

The main features are:

• Item size is limited to 1 Mbyte.

• Persistence is not supported.

• Replication is not supported; data is available only on one node.


126

• Statistics are limited to those directly related to the in-memory nature of the data. Statistics related to persistence,disk I/O and replication/rebalancing are not available.

• Client setup should use ketama consistent hashing

• memcached buckets do not use vBuckets, so there is no rebalancing.

• Couchbase Buckets

Couchbase buckets support the full range of Couchbase-specific functionality, including balancing, persistence andreplication. The main features are:

• Item size is limited to 20 Mbyte.

• Persistence, including data sets larger than the allocated memory size.

• Replication and rebalancing are fully supported.

• Full suite of statistics supported.

In addition to these overall bucket differences, there are also security and network port differences that enable you to con-figure and structure the connectivity to the different bucket types differently.

Figure 11.3. Couchbase Buckets


127

There are three bucket interface types that can be be configured:

• The default Bucket

The default bucket is a Couchbase bucket that always resides on port 11211 and is a non-SASL authenticating bucket.When Couchbase is first install this bucket is automatically set up during installation. This bucket may be removed afterinstallation and may also be re-added later, but when re-adding a bucket named "default", the bucket must be place onport 11211 and must be a non-SASL authenticating bucket. A bucket not named default may not reside on port 11211 ifit is a non-SASL bucket. The default bucket may be reached with a vBucket aware smart client, an ASCII client or a bi-nary client that doesn't use SASL authentication.

• Non-SASL Buckets

Non-SASL buckets may be placed on any available port with the exception of port 11211 if the bucket is not named"default". Only one Non-SASL bucket may placed on any individual port. These buckets may be reached with a vBuck-et aware smart client, an ASCII client or a binary client that doesn't use SASL authentication

• SASL Buckets

SASL authenticating Couchbase buckets may only be placed on port 11211 and each bucket is differentiated by itsname and password. SASL bucket may not be placed on any other port beside 11211. These buckets can be reachedwith either a vBucket aware smart client or a binary client that has SASL support. These buckets cannot be reached withASCII clients.

11.7. vBucketsWarning

For simplicity, in this section we completely ignore Couchbase Server multi-tenancy (or what wehave historically called a "bucket," which represents a "virtual couchbase instance" inside a singlecouchbase cluster). The bucket and vBucket concepts are not to be confused - they are not related.For purposes of this section, a bucket can simply be viewed as synonymous with "a couchbase clus-ter."

A vBucket is defined as the "owner" of a subset of the key space of a Couchbase Server cluster.

Every document ID "belongs" to a vBucket. A mapping function is used to calculate the vBucket in which a given docu-ment ID belongs. In couchbase, that mapping function is a hashing function that takes a document ID as input and outputsa vBucket identifier. Once the vBucket identifier has been computed, a table is consulted to lookup the server that "hosts"that vBucket. The table contains one row per vBucket, pairing the vBucket to its hosting server. A server appearing in thistable can be (and usually is) responsible for multiple vBuckets.

The hashing function used by couchbase to map document IDs to vBuckets is configurable - both the hashing algorithmand the output space (i.e. the total number of vBuckets output by the function). Naturally, if the number of vBuckets in theoutput space of the hash function is changed, then the table which maps vBuckets to Servers must be resized.

11.7.1. Couchbase Document ID-vBucket-Server Mapping Illustrated

The vBucket mechanism provides a layer of indirection between the hashing algorithm and the server responsible for agiven document ID. This indirection is useful in managing the orderly transition from one cluster configuration to another,whether the transition was planned (e.g. adding new servers to a cluster) or unexpected (e.g. a server failure).

The diagram below shows how document ID-Server mapping works when using the vBucket construct. There are 3servers in the cluster. A client wants to look up (get) the value of document ID . The client first hashes the ID to calculatethe vBucket which owns ID. Assume Hash(ID) = vB8. The client then consults the vBucket-server mapping table and de-termines Server C hosts vB8. The get operation is sent to Server C.


128

After some period of time, there is a need to add a server to the cluster (e.g. to sustain performance in the face of growingapplication use). Administrator adds Server D to the cluster and the vBucket Map is updated as follows [note: the vBuck-et-Server map is updated by an internal Couchbase algorithm and that _updated table is transmitted by Couchbase to allcluster participants - servers and proxies]

After the addition, a client once again wants to look up (get) the value of document ID . Because the hashing algorithm inthis case has not changed 1 Hash(ID) = vB8 as before. The client examines the vBucket-server mapping table and deter-mines Server D now owns vB8. The get operation is sent to Server D.

11.7.2. vBuckets in a world of memcached clients

Couchbase Commandment 1 requires couchbase to be a drop in replacement for an existing memcached server, whileadding persistence, replication, failover and dynamic cluster reconfiguration. Existing applications will likely (99.999%)be using an old memcached client to communicate with an memcached cluster. This client will probably be using a con-sistent hashing algorithm to directly map document IDs to servers.

To make this work, a proxy capability is required. Over the longer run, having a client that can implement the vBucketconcept directly will remove the need for a proxy (though a proxy will continue to be desirable in some environments).We do not expect vBucket-aware clients to emerge quickly.

Couchbase TCP ports

Couchbase can listen for data operations on two configurable ports: 11210 and 11211 (these are default port numbers incouchbase). Both ports are "memcapable," supporting the memcached ASCII and Binary protocols.

• Port 11211 is the port on which an embedded proxy listens (the standard memcached port). It can receive, and success-fully process, data operations for document IDs that are owned by vBuckets not hosted by this server. The proxy willforward the request to the right server then return the result to the client.

• Port 11210 is the port on which the couchbase data operation server listens. It will reject data operations for documentIDs owned by vBuckets not hosted by this server. To do this, a document ID-vBucket hash must be performed on all re-quests. The vBucket is then compared with the list of vBuckets hosted by this server.

11.7.2.1. Deployment Option 1 - Using Couchbase Embedded Proxy

The first deployment option is to communicate with the embedded proxy in Couchbase over port 11211. Supporting thisdeployment option is our highest priority. It allows the customer to install couchbase and begin using it with an existingapplication, via an memcached client, without also installing another piece of proxy software. The downside to this ap-proach is performance. We must do everything practical to minimize latency and throughput degradation.

In this deployment option (as shown in detail below) versus an memcached deployment, in a worst case scenario, servermapping will happen twice (e.g. using ketama hashing to a server list on the client, then using vBucket hashing and servermapping on the proxy) with an additional round trip network hop introduced.

Assume there is an existing application, with an memcached client, with a server list of 3 servers (Servers A, B and C).Couchbase is installed in place of the memcached server software on each of these 3 servers.

As shown in the figure above, when the application wants to Get(ID), it will call a function in the client library. Clientlibrary will Hash(ID)and be directed, based on the server list and hashing function to Server C. The Get operation issent to Server C, port 11211. When it arrives to couchbase (now a proxy port), the Key is hashed again to determine itsvBucket and Server mapping. This time, the result is Server A. The proxy will contact Server A on port 11210, performthe get operation and return the result to the client.

11.7.2.2. Deployment Option 2 - Standalone Proxy

The second option is to deploy a standalone proxy, which performs substantially the same way as the embedded proxy, butpotentially eliminating a network hop. A standalone proxy deployed on a client may also be able to provide valuable ser-


129

vices, such as connection pooling. The diagram below shows the flow with a standalone proxy installed on the applicationserver.

The memcached client is configured to have just one server in its server list (localhost), so all operations are forward-ed to localhost:11211- a port serviced by the proxy. The proxy hashes the key to a vBucket, looks up the host serv-er in the vBucket table, and then sends the operation to the appropriate couchbase server (Server A in this case) on port11210.

11.7.2.3. Deployment Option 3 - "vBucket aware" client

In the final case, no proxy is installed anywhere in the data flow. The client has been updated and performs server selec-tion directly via the vBucket mechanism. In addition, these clients could send additional information using a modified on-the-wire couchbase protocol, for example to explicitly encode the destination vBucket. This data could be used to optimizesystem performance.

See also vBuckets for an in-depth description.


130

Chapter 12. Monitoring CouchbaseThere are a number of different ways in which you can monitor Couchbase. You should be aware however of some of thebasic issues that you will need to know before starting your monitoring procedure.

12.1. Port numbers and accessing different bucketsIn a Couchbase Server cluster, any communication (stats or data) to a port other than 11210 will result in the request goingthrough a Moxi process. This means that any stats request will be aggregated across the cluster (and may produce some in-consistencies or confusion when looking at stats that are not "aggregatable").

In general, it is best to run all your stat commands against port 11210 which will always give you the information for thespecific node that you are sending the request to. It is a best practice to then aggregate the relevant data across nodes at ahigher level (in your own script or monitoring system).

When you run the below commands (and all stats commands) without supplying a bucket name and/or password, they willreturn results for the default bucket and produce an error if one does not exist.

To access a bucket other than the default, you will need to supply the bucket name and/or password on the end of the com-mand. Any bucket created on a dedicated port does not require a password.

12.2. Monitoring startup (warmup)If a Couchbase Server node is starting up for the first time, it will create whatever DB files necessary and begin servingdata immediately. However, if there is already data on disk (likely because the node rebooted or the service restarted) thenode needs to read all of this data off of disk before it can begin serving data. This is called "warmup". Depending on thesize of data, this can take some time.

When starting up a node, there are a few statistics to monitor. Use the /opt/couchbase/bin/ep_engine/management/stats(Linux) or C:\Program Files\Couchbase\Server\bin\ep_engine\management\stats (Windows) command to watch thewarmup and item stats:

shell> /opt/couchbase/bin/ep_engine/management/stats localhost:11210 all | » egrep "warm|curr_items"

curr_items: 0

curr_items_tot: 15687

ep_warmed_up: 15687

ep_warmup: false

ep_warmup_dups: 0

ep_warmup_oom: 0

ep_warmup_thread: running

ep_warmup_time: 787

And when it is complete:

shell> /opt/couchbase/bin/ep_engine/management/stats localhost:11210 all | » egrep "warm|curr_items"

curr_items: 10000

curr_items_tot: 20000

ep_warmed_up: 20000

Monitoring Couchbase

131

ep_warmup: true

ep_warmup_dups: 0

ep_warmup_oom: 0

ep_warmup_thread: complete

ep_warmup_time 1400

Table 12.1. Stats

Stat Description

curr_items The number of items currently active on this node. Duringwarmup, this will be 0 until complete

curr_items_tot The total number of items this node knows about (activeand replica). During warmup, this will be increasing andshould match ep_warmed_up

ep_warmed_up The number of items retrieved from disk. During warmup,this should be increasing.

ep_warmup_dups The number of duplicate items found on disk. Ideallyshould be 0, but a few is not a problem

ep_warmup_oom How many times the warmup process received an Out ofMemory response from the server while loading data intoRAM

ep_warmup_thread The status of the warmup thread. Can be either running orcomplete

ep_warmup_time How long the warmup thread was running for. Duringwarmup this number should be increasing, when complete itwill tell you how long the process took

12.3. Disk Write QueueCouchbase Server is a persistent database which means that part of monitoring the system is understanding how we inter-act with the disk subsystem.

Since Couchbase Server is an asynchronous system, any mutation operation is committed first to DRAM and then queuedto be written to disk. The client is returned an acknowledgement almost immediately so that it can continue working.There is replication involved here too, but we're ignoring it for the purposes of this discussion.

We have implemented disk writing as a 2-queue system and they are tracked by the stats. The first queue is where muta-tions are immediately placed. Whenever there are items in that queue, our "flusher" (disk writer) comes along and takesall the items off of that queue, places them into the other one and begins writing to disk. Since disk performance is so dra-matically different than RAM, this allows us to continue accepting new writes while we are (possibly slowly) writing newones to the disk.

The flusher will process 250k items a a time, then perform a disk commit and continue this cycle until its queue is drained.When it has completed everything in its queue, it will either grab the next group from the first queue or essentially sleepuntil there are more items to write.

12.3.1. Handling Reads

In its current implementation, SQLite only has one connection into the database for both reads and writes. This means thatwe need to do something special when reads come in while the flusher is writing to disk.


132

If a request comes in for an item that is on disk, it will preempt the writing process in order to serve this read.

12.3.2. Monitoring the Disk Write Queue

There are basically two ways to monitor the disk queue, at a high-level from the Web UI or at a low-level from the indi-vidual node statistics.

From the Web UI, click on Monitor Data Buckets and select the particular bucket that you want to monitor. Click "Con-figure View" in the top right corner and select the "Disk Write Queue" statistic. Closing this window will show that thereis a new mini-graph. This graph is showing the Disk Write Queue for all nodes in the cluster. To get a deeper view intothis statistic, you can monitor each node individually using the 'stats' output (see here for more information about gather-ing node-level stats). There are two statistics to watch here:

ep_queue_size (where new mutations are placed) flusher_todo (the queue of items currently being written to disk)

See The Dispatcher for more information about monitoring what the disk subsystem is doing at any given time.

12.4. Couchbase Server Statistics

Couchbase Server provides statistics at multiple levels throughout the cluster. These are used for regular monitoring, ca-pacity planning and to identify the performance characteristics of your cluster deployment. The most visible statistics arethose in the Web UI, but components such as the REST interface, the proxy and individual nodes have directly accessiblestatistics interfaces.

12.4.1. REST Interface Statistics

The easiest to use interface into the statistics provided by REST is to use theChapter 6, Administration Web Console . ThisGUI gathers statistics via REST and displays them to your browser. The REST interface has a set of resources that provideaccess to the current and historic statistics the cluster gathers and stores. See the REST documentation for more informa-tion.

12.4.2. Couchbase Server Node Statistics

Detailed stats documentation can be found in the respository.

Along with stats at the REST and UI level, individual nodes can also be queried for statistics either through a client whichuses binary protocol or through the stats utility shipped with couchbase.

For example:

shell> cbstats localhost:11210 all auth_cmds: 9 auth_errors: 0 bucket_conns: 10 bytes_read: 246378222 bytes_written: 289715944 cas_badval: 0 cas_hits: 0 cas_misses: 0 cmd_flush: 0 cmd_get: 134250 cmd_set: 115750...

The most commonly needed statistics are surfaced through the Web Console and have descriptions there and in the asso-ciated documentation. Software developers and system administrators wanting lower level information have it availablethrough the stats interface.

http://github.com/membase/ep-engine/blob/master/docs/stats.org


133

There are seven commands available through the stats interface:

• stats(referred to as 'all')

• dispatcher

• hash

• tap

• timings

• vkey

• reset

12.4.2.1. stats Command

This displays a large list of statistics related to the Couchbase process including the underlying engine (ep_* stats).

12.4.2.2. dispatcher Command

This statistic will show what the dispatcher is currently doing:

dispatcher runtime: 45ms state: dispatcher_running status: running task: Running a flusher loop.nio_dispatcher state: dispatcher_running status: idle

The first entry, dispatcher, monitors the process responsible for disk access. The second entry is a non-IO (non disk) dis-patcher. There may also be a ro_dispatcher dispatcher present if the engine is allowing concurrent reads and writes. Whena task is actually running on a given dispatcher, the "runtime" tells you how long the current task has been running. Newerversions will show you a log of recently run dispatcher jobs so you can see what's been happening.

12.5. Couchbase Server Moxi Statistics

Moxi, as part of it's support of memcached protocol, has support for the memcached stats command. Regular mem-cached clients can request statistics through the memcached stats command. The stats command accepts optional argu-ments, and in the case of Moxi, there is a stats proxy sub-command. A detailed description of statistics available throughMoxi can be found here.

For example, one simple client one may use is the commonly available netcat (output elided with ellipses):

http://www.couchbase.com/docs//moxi-manual-1.7/index.html


134

$ echo "stats proxy" | nc localhost 11211STAT basic:version 1.6.0STAT basic:nthreads 5...STAT proxy_main:conf_type dynamicSTAT proxy_main:behavior:cycle 0STAT proxy_main:behavior:downstream_max 4STAT proxy_main:behavior:downstream_conn_max 0STAT proxy_main:behavior:downstream_weight 0...STAT proxy_main:stats:stat_configs 1STAT proxy_main:stats:stat_config_fails 0STAT proxy_main:stats:stat_proxy_starts 2STAT proxy_main:stats:stat_proxy_start_fails 0STAT proxy_main:stats:stat_proxy_existings 0STAT proxy_main:stats:stat_proxy_shutdowns 0STAT 11211:default:info:port 11211STAT 11211:default:info:name default...STAT 11211:default:behavior:downstream_protocol 8STAT 11211:default:behavior:downstream_timeout 0STAT 11211:default:behavior:wait_queue_timeout 0STAT 11211:default:behavior:time_stats 0STAT 11211:default:behavior:connect_max_errors 0STAT 11211:default:behavior:connect_retry_interval 0STAT 11211:default:behavior:front_cache_max 200STAT 11211:default:behavior:front_cache_lifespan 0STAT 11211:default:behavior:front_cache_specSTAT 11211:default:behavior:front_cache_unspecSTAT 11211:default:behavior:key_stats_max 4000STAT 11211:default:behavior:key_stats_lifespan 0STAT 11211:default:behavior:key_stats_specSTAT 11211:default:behavior:key_stats_unspecSTAT 11211:default:behavior:optimize_setSTAT 11211:default:behavior:usr default...STAT 11211:default:pstd_stats:num_upstream 1STAT 11211:default:pstd_stats:tot_upstream 2STAT 11211:default:pstd_stats:num_downstream_conn 1STAT 11211:default:pstd_stats:tot_downstream_conn 1STAT 11211:default:pstd_stats:tot_downstream_conn_acquired 1STAT 11211:default:pstd_stats:tot_downstream_conn_released 1STAT 11211:default:pstd_stats:tot_downstream_released 2STAT 11211:default:pstd_stats:tot_downstream_reserved 1STAT 11211:default:pstd_stats:tot_downstream_reserved_time 0STAT 11211:default:pstd_stats:max_downstream_reserved_time 0STAT 11211:default:pstd_stats:tot_downstream_freed 0STAT 11211:default:pstd_stats:tot_downstream_quit_server 0STAT 11211:default:pstd_stats:tot_downstream_max_reached 0STAT 11211:default:pstd_stats:tot_downstream_create_failed 0STAT 11211:default:pstd_stats:tot_downstream_connect 1STAT 11211:default:pstd_stats:tot_downstream_connect_failed 0STAT 11211:default:pstd_stats:tot_downstream_connect_timeout 0STAT 11211:default:pstd_stats:tot_downstream_connect_interval 0STAT 11211:default:pstd_stats:tot_downstream_connect_max_reached 0...END

135

Chapter 13. TroubleshootingWhen troubleshooting your Couchbase Server deployment there are a number of different approaches available to you.For specific answers to individual problems, see Section 13.6, “Common Errors”.

13.1. General Tips

The following are some general tips that may be useful before performing any more detailed investigations:

• Try pinging the node.

• Try connecting to the Couchbase Server Web Console on the node.

• Try to use telnet to connect to the various ports that Couchbase Server uses.

• Try reloading the web page.

• Check firewall settings (if any) on the node. Make sure there isn't a firewall between you and the node. On a Windowssystem, for example, the Windows firewall might be blocking the ports (Control Panel > Windows Firewall).

• Make sure that the documented ports are open between nodes and make sure the data operation ports are available toclients.

• Check your browser's security settings.

• Check any other security software installed on your system, such as antivirus programs.

• Click Generate Diagnostic Report on the Log page to obtain a snapshot of your system's configuration and log informa-tion for deeper analysis. This information can be sent to Couchbase Technical Support to diagnose issues.

13.2. Responding to Specific Errors

The following table outlines some specific areas to check when experiencing different problems:

Table 13.1. Responses to Specific Errors

Severity Issue Suggested Action(s)

Critical Couchbase Server does not start up. Check that the service is running.

Check error logs.

Try restarting the service.

Critical A server is not responding. Check that the service is running.

Check error logs.

Try restarting the service.

Critical A server is down. Try restarting the server.

Use the command-line interface tocheck connectivity.

Informational Bucket authentication failure. Check the properties of the bucket thatyou are attempting to connect to.

The primary source for run-time logging information is the Couchbase Server Web Console. Run-time logs are automat-ically set up and started during the installation process. However, the Couchbase Server gives you access to lower-level

Troubleshooting

136

logging details if needed for diagnostic and troubleshooting purposes. Log files are stored in a binary format in the logs di-rectory under the Couchbase installation directory. You must use browse_logs to extract the log contents from the binaryformat to a text file.

13.3. Log File Entries

The log file contains entries for various events, such as progress, information, error, and crash. Note that somethingflagged as error or crash does not necessarily mean that there is anything wrong. Below is an example of the output:

=PROGRESS REPORT==== 4-Mar-2010::11:54:23 ===supervisor: {local,sasl_safe_sup}started: [{pid,<0.32.0>},{name,alarm_handler},{mfa,{alarm_handler,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]=PROGRESS REPORT==== 4-Mar-2010::11:54:23 ===supervisor: {local,sasl_safe_sup}started: [{pid,<0.33.0>},{name,overload},{mfa,{overload,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]

Look for ns_log in the output to see anything that would have been shown via the Couchbase Server Web Console.

13.4. Linux Logs

Couchbase Server stores its logs different places, depending on the component reporting the error. The core memory andreplication logs are stored in an efficient, space-limited binary format under /opt/couchbase/var/lig/mem-base/logs. A Couchbase shell script (cbbrowse_logs) will output these logs in a human-readable text format.

To dump the most recent 100MB of logs as text, run the following command:

shell> cbbrowse_logs

You can redirect the output of this batch file to a text file that you can save for later viewing or email to Couchbase Tech-nical Support. The following example redirects output to a text file named nslogs.txt.

shell> cbbrowse_logs > /tmp/nslogs.txt

13.5. Windows Logs

Couchbase Server stores its logs in an efficient, space-limited binary format. The binary run-time log files reside in thefollowing path<install_path>\log (where < install_path> is where you installed the Couchbase software). Youcannot open these binary files directly. To dump the run-time logs into a text file, run the browse_logs.bat file, located<install_path>\bin\browse_logs.bat. This command dumps the logs to standard output in a human-readable text format.You can redirect the output of this batch file to a text file that you can save for later viewing or email to Couchbase Tech-nical Support. The following example redirects output to a text file named nslogs.txt.

shell> "C:\Program Files\Couchbase\Memcached Server\bin\browse_logs.bat" > C:\nslogs.txt

13.6. Common Errors

This page will attempt to describe and resolve some common errors that are encountered when using Couchbase. It will bea living document as new problems and/or resolutions are discovered.

• Problems Starting Couchbase Server for the first time

http://www.couchbase.com/products-and-services/couchbase-support

http://www.couchbase.com/products-and-services/couchbase-support

http://www.couchbase.com/support

http://www.couchbase.com/support

Troubleshooting

137

If you are having problems starting Couchbase Server on Linux for the first time, there are two very common causes ofthis that are actually quite related. When the /etc/init.d/couchbase-server script runs, it tries to set the file descriptorlimit and core file size limit:

shell> ulimit -n 10240 ulimit -c unlimited

Depending on the defaults of your system, this may or may not be allowed. If Couchbase Server is failing to start, youcan look through the logs (see Section 13.3, “Log File Entries”) and pick out one or both of these messages:

ns_log: logging ns_port_server:0:Port server memcached on node '[email protected]' exited with status 71. »Restarting. Messages: failed to set rlimit for open files. »Try running as root or requesting smaller maxconns value.

Alternatively you may additional see or optionally see:

ns_port_server:0:info:message - Port server memcached on node '[email protected]' exited with status 71. »Restarting. Messages: failed to ensure corefile creation

The resolution to these is to edit the /etc/security/limits.conf file and add these entries:

couchbase hard nofile 10240couchbase hard core unlimited

138

Appendix A. FAQs• What kind of client do I use with couchbase?

couchbase is compatible with existing memcached clients. If you have a memcached client already, you can just pointit at couchbase. Regular testing is done with spymemcached(the Java client), libmemcached and fauna (Rubyclient). See the Client Libraries page

• What is a "vbucket"?

An overview from Dustin Sallings is presented here: memcached vBuckets

• What is a TAP stream?

A TAP stream is a when a client requests a stream of item updates from the server. That is, as other clients are request-ing item mutations (for example, SET's and DELETE's), a TAP stream client can "wire-tap" the server to receive astream of item change notifications.

When a TAP stream client starts its connection, it may also optionally request a stream of all items stored in the server,even if no other clients are making any item changes. On the TAP stream connection setup options, a TAP stream clientmay request to receive just current items stored in the server (all items until "now"), or all item changes from now on-wards into in the future, or both.

Trond Norbye's written a blog post about the TAP interface. See Blog Entry.

• What ports does couchbase Server need to run on?

The following TCP ports should be available:

• 8091 — GUI and REST interface

• 11211 — Server-side Moxi port for standard memcached client access

• 11210 — native couchbase data port

• 21100 to 21199 — inclusive for dynamic cluster communication

• What hardware and platforms does couchbase Server support?

Couchbase Server supports Red Hat (and CentOS) versions 5 starting with update 2, Ubuntu 9 and and Windows Serv-er 2008 (other versions have been shown to work but are not being specifically tested). There are both 32-bit and 64-bitversions available. Community support for Mac OS X is available. Future releases will provide support for additionalplatforms.

• How can I get couchbase on (this other OS)?

The couchbase source code is quite portable and is known to have been built on several other UNIX and Linux basedOSs. See Consolidated sources.

• Can I query couchbase by something other than the key name?

Not directly. It's possible to build these kinds of solutions atop TAP. For instance, via Cascading it is possible to streamout the data, process it with Cascading, then create indexes in Elastic Search.

• What is the maximum item size in couchbase?

The default item size for couchbase buckets is 20 MBytes. The default item size for memcached buckets is 1 MByte.Both are tunable, though not through the system console.



http://blog.couchbase.com/want-know-what-your-memcached-servers-are-doing-tap-them

http://www.couchbase.com/downloads/

http://www.cascading.org/2010/09/memcached-membase-and-elastics.html

FAQs

139

• How do I the change password?

shell> couchbase cluster-init -c cluster_IP:8091 -u current_username-p current password --cluster-init-username=new_username --cluster-init-password=new_password

• How do I change the per-node RAM quota?

shell> couchbase cluster-init -c \ cluster_IP:8091 -u username-p password --cluster-init-ramsize=RAM_in_MB

• How do I change the disk path?

Use the couchbase command-line tool:

shell> couchbase node-init -c cluster_IP:8091 -u \ username-p password--node-init-data-path=/tmp

• Why are some clients getting different results than others for the same requests?

This should never happen in a correctly-configured couchbase cluster, since couchbase ensures a consistent view of alldata in a cluster. However, if some clients can't reach all the nodes in a cluster (due to firewall or routing rules, for ex-ample), it is possible for the same key to end up on more than one cluster node, resulting in inconsistent duplication. Al-ways ensure that all cluster nodes are reachable from every smart client or client-side moxi host.

140

Appendix B. Uninstalling Couchbase ServerIf you want to uninstall Couchbase Server from your system you must choose the method appropriate for your operatingsystem.

Before removing Couchbase Server from your system, you should do the following:

• Shutdown your Couchbase Server. For more information on the methods of shutting down your server for your plat-form, see Section 3.2, “Startup and Shutdown of Couchbase Server”.

• If your machine is part of an active cluster, you should rebalance your cluster to take the node out of your configuration.See Section 5.3.3, “Rebalancing”.

• Update your clients to point to an available node within your Couchbase Server cluster.

B.1. Uninstalling on a RedHat Linux System

To uninstall the software on a RedHat Linux system, run the following command:

shell> sudo rpm -e couchbase-server

Refer to the RedHat RPM documentation for more information about uninstalling packages using RPM.

You may need to delete the data files associated with your installation. The default installation location is /opt. If you se-lected an alternative location for your data files, you will need to separately delete each data directory from your system.

B.2. Uninstalling on an Debian/Ubuntu Linux System

To uninstall the software on a Ubuntu Linux system, run the following command:

shell> sudo dpkg -r couchbase-server

Refer to the Ubuntu documentation for more information about uninstalling packages using dpkg.

You may need to delete the data files associated with your installation. The default installation location is /opt. If you se-lected an alternative location for your data files, you will need to separately delete each data directory from your system.

B.3. Uninstalling on a Windows System

To uninstall the software on a Windows system you must have Administrator or Power User privileges to uninstall Couch-base.

To remove, choose Start> Settings>Control Panel, choose Add or Remove Programs, and remove the Couchbase Serversoftware.

B.4. Uninstalling on a Mac OS X System

To uninstall on Mac OS X, open the Applications folder, and then drag the Couchbase Server application to thetrash. You may be asked to provide administrator credentials to complete the deletion.

To remove the application data, you will need to delete the Membase folder from the ~/Library/ApplicationSupport folder for the user that ran Couchbase Server.

141

Appendix C. Release NotesThe following sections provide release notes for individual release versions of Couchbase Server. To browse or submitnew issaues, see Couchbase Server Issues Tracker.

C.1. Release Notes for Couchbase Server 1.8.0 GA (23 January 2012)Couchbase Server 1.8 is the updated and rebranded release of Membase Server.

Important

In line with the rebranding and name changes, there are some significant changes to the following ar-eas of this release:

• Directory Changes

Couchbase Server is now installed into a couchbase directory, for example on Linux the defaultinstallation directory is /opt/couchbase instead of /opt/membase.

During an upgrade, the location of your data files will not be modified.

• Command Changes

The name of many of the core commands provided with Couchbase Server have been renamed,with the existing scripts deprecated. For example, the backup command mbbackup in MembaseServer is now called cbbackup. See the full release note entry below for more detailed informa-tion.

New Features and Feature Changes in 1.8.0

• The membase bucket type has been deprecated. The couchbase bucket type should be used instead to select the bal-anced, persistent and replicated buckets supported by a Couchbase cluster.

For more information, see Section 11.6, “Buckets”.

Tags: deprecation

• Internet Explorer 7 as a supported browser for the Couchbase Web Admin Console is now deprecated, and support willbe removed entirely in a future release.

Tags: deprecation

• Allow disk write queue upper limit to be modified at runtime.

Issues: MB-4418

• The SYNC protocol command has been deprecated. Applications and solutions should no longer use the SYNC com-mand. The command will be removed entirely in a future version.

Tags: deprecation

• Ubuntu 9.10 is no longer a supported platform.

Tags: removal

• The following command-line tools have been deprecated and replaced with the corresponding tool as noted in the tablebelow.

http://www.couchbase.com/issues/browse/MB

http://www.couchbase.com/issues/browse/MB-4418

Release Notes

142

Deprecated Tool Replacement Tool

membase couchbase-cli

mbadm-online-restore cbadm-online-restore

mbadm-online-update cbadm-online-update

mbadm-tap-registration cbadm-tap-registration

mbbackup-incremental cbbackup-incremental

mbbackup-merge-incremental cbbackup-merge-incremental

mbbackup cbbackup

mbbrowse_logs cbbrowse_logs

mbcollect_info cbcollect_info

mbdbconvert cbdbconvert

mbdbmaint cbdbmaint

mbdbupgrade cbdbupgrade

mbdumpconfig.escript cbdumpconfig.escript

mbenable_core_dumps.sh cbenable_core_dumps.sh

mbflushctl cbflushctl

mbrestore cbrestore

mbstats cbstats

mbupgrade cbupgrade

mbvbucketctl cbvbucketctl

Using a deprecated tool will result in a warning message that the tool is deprecated and will no longer be supported.

For more information, see Chapter 7, Couchbase Command-line Interface.

Tags: deprecation

Fixes in 1.8.0

• Rebalancing process might hang when adding or removing nodes in a large cluster where client load is running.

Issues: MB-4517, MB-4490

• Selecting the master node during rolling upgrade where there are different membase/couchbase servers in the cluster.

Issues: MB-4592

• The ep_num_not_my_vbuckets stat doesn't reset.

Issues: MB-3852

• Decreased memory usage by coalesce checkpoints in the replica side if item persistence is slower. Server will maintainonly two replica checkpoints for each vbucket.

Issues: MB-4578

• Installation of Membase on Amazon Linux would fail.






Release Notes

143

Issues: MB-3419

• Unable to create a default bucket using membase-cli.

Issues: MB-4453

• Fixed a case where there are two replicas where replication cursor would get stuck and slave node didn't replicate thedata into the other node in the replication chain.

Issues: MB-4461

• Installer dependencies on RHEL 5.4 have been updated.

Issues: MB-4561





144

Appendix D. LicensesThis documentation and associated software is subject to the following licenses.

D.1. Documentation License

This documentation in any form, software or printed matter, contains proprietary information that is the exclusive prop-erty of Couchbase. Your access to and use of this material is subject to the terms and conditions of your Couchbase Soft-ware License and Service Agreement, which has been executed and with which you agree to comply. This document andinformation contained herein may not be disclosed, copied, reproduced, or distributed to anyone outside Couchbase with-out prior written consent of Couchbase or as specifically provided below. This document is not part of your license agree-ment nor can it be incorporated into any contractual agreement with Couchbase or its subsidiaries or affiliates.

Use of this documentation is subject to the following terms:

You may create a printed copy of this documentation solely for your own personal use. Conversion to other formats is al-lowed as long as the actual content is not altered or edited in any way. You shall not publish or distribute this documenta-tion in any form or on any media, except if you distribute the documentation in a manner similar to how Couchbase dis-seminates it (that is, electronically for download on a Web site with the software) or on a CD-ROM or similar medium,provided however that the documentation is disseminated together with the software on the same medium. Any other use,such as any dissemination of printed copies or use of this documentation, in whole or in part, in another publication, re-quires the prior written consent from an authorized representative of Couchbase. Couchbase and/or its affiliates reserveany and all rights to this documentation not expressly granted above.

This documentation may provide access to or information on content, products, and services from third parties. CouchbaseInc. and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-partycontent, products, and services. Couchbase Inc. and its affiliates will not be responsible for any loss, costs, or damages in-curred due to your access to or use of third-party content, products, or services.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find anyerrors, please report them to us in writing.

D.2. Couchbase, Inc. Community Edition License Agreement

IMPORTANT-READ CAREFULLY: BY CLICKING THE "I ACCEPT" BOX OR INSTALLING, DOWNLOADINGOR OTHERWISE USING THIS SOFTWARE AND ANY ASSOCIATED DOCUMENTATION, YOU, ON BEHALFOF YOURSELF OR AS AN AUTHORIZED REPRESENTATIVE ON BEHALF OF AN ENTITY ("LICENSEE")AGREE TO ALL THE TERMS OF THIS COMMUNITY EDITION LICENSE AGREEMENT (THE "AGREEMENT")REGARDING YOUR USE OF THE SOFTWARE. YOU REPRESENT AND WARRANT THAT YOU HAVE FULLLEGAL AUTHORITY TO BIND THE LICENSEE TO THIS AGREEMENT. IF YOU DO NOT AGREE WITH ALLOF THESE TERMS, DO NOT SELECT THE "I ACCEPT" BOX AND DO NOT INSTALL, DOWNLOAD OR OTH-ERWISE USE THE SOFTWARE. THE EFFECTIVE DATE OF THIS AGREEMENT IS THE DATE ON WHICH YOUCLICK "I ACCEPT" OR OTHERWISE INSTALL, DOWNLOAD OR USE THE SOFTWARE.

1. License Grant. Couchbase Inc. hereby grants Licensee, free of charge, the non-exclusive right to use, copy, merge, pub-lish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnishedto do so, subject to Licensee including the following copyright notice in all copies or substantial portions of the Soft-ware:

Couchbase ©http://www.couchbase.comCopyright 2011 Couchbase, Inc.

As used in this Agreement, "Software" means the object code version of the applicable elastic data management serversoftware provided by Couchbase, Inc.

Licenses

145

2. Support. Couchbase, Inc. will provide Licensee with access to, and use of, the Couchbase, Inc. support forum availableat the following URL: http://forums.membase.org. Couchbase, Inc. may, at its discretion, modify, suspend or terminatesupport at any time upon notice to Licensee.

3. Warranty Disclaimer and Limitation of Liability. THE SOFTWARE IS PROVIDED "AS IS," WITHOUT WARRAN-TY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENTSHALL COUCHBASE INC. OR THE AUTHORS OR COPYRIGHT HOLDERS IN THE SOFTWARE BE LIABLEFOR ANY CLAIM, DAMAGES (INCLUDING, WITHOUT LIMITATION, DIRECT, INDIRECT OR CONSE-QUENTIAL DAMAGES) OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTH-ERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHERDEALINGS IN THE SOFTWARE.

D.3. Couchbase, Inc. Enterprise License Agreement: Free EditionIMPORTANT-READ CAREFULLY: BY CLICKING THE "I ACCEPT" BOX OR INSTALLING, DOWNLOADINGOR OTHERWISE USING THIS SOFTWARE AND ANY ASSOCIATED DOCUMENTATION, YOU, ON BEHALFOF YOURSELF OR AS AN AUTHORIZED REPRESENTATIVE ON BEHALF OF AN ENTITY ("LICENSEE")AGREE TO ALL THE TERMS OF THIS ENTERPRISE LICENSE AGREEMENT – FREE EDITION (THE "AGREE-MENT") REGARDING YOUR USE OF THE SOFTWARE. YOU REPRESENT AND WARRANT THAT YOU HAVEFULL LEGAL AUTHORITY TO BIND THE LICENSEE TO THIS AGREEMENT. IF YOU DO NOT AGREE WITHALL OF THESE TERMS, DO NOT SELECT THE "I ACCEPT" BOX AND DO NOT INSTALL, DOWNLOAD OROTHERWISE USE THE SOFTWARE. THE EFFECTIVE DATE OF THIS AGREEMENT IS THE DATE ON WHICHYOU CLICK "I ACCEPT" OR OTHERWISE INSTALL, DOWNLOAD OR USE THE SOFTWARE.

1. License Grant. Subject to Licensee's compliance with the terms and conditions of this Agreement, Couchbase Inc.hereby grants to Licensee a perpetual, non-exclusive, non-transferable, non-sublicensable, royalty-free, limited licenseto install and use the Software only for Licensee's own internal production use on up to two (2) Licensed Servers or forLicensee's own internal non-production use for the purpose of evaluation and/or development on an unlimited numberof Licensed Servers.

2. Restrictions. Licensee will not: (a) copy or use the Software in any manner except as expressly permitted in thisAgreement; (b) use or deploy the Software on any server in excess of the Licensed Servers for which Licensee has paidthe applicable Subscription Fee unless it is covered by a valid license; (c) transfer, sell, rent, lease, lend, distribute, orsublicense the Software to any third party; (d) use the Software for providing time-sharing services, service bureau ser-vices or as part of an application services provider or as a service offering primarily designed to offer the functionali-ty of the Software; (e) reverse engineer, disassemble, or decompile the Software (except to the extent such restrictionsare prohibited by law); (f) alter, modify, enhance or prepare any derivative work from or of the Software; (g) alter orremove any proprietary notices in the Software; (h) make available to any third party the functionality of the Softwareor any license keys used in connection with the Software; (i) publically display or communicate the results of internalperformance testing or other benchmarking or performance evaluation of the Software; or (j) export the Software in vi-olation of U.S. Department of Commerce export administration rules or any other export laws or regulations.

3. Proprietary Rights. The Software, and any modifications or derivatives thereto, is and shall remain the sole proper-ty of Couchbase Inc. and its licensors, and, except for the license rights granted herein, Couchbase Inc. and its licen-sors retain all right, title and interest in and to the Software, including all intellectual property rights therein and there-to. The Software may include third party open source software components. If Licensee is the United States Govern-ment or any contractor thereof, all licenses granted hereunder are subject to the following: (a) for acquisition by or onbehalf of civil agencies, as necessary to obtain protection as "commercial computer software" and related documen-tation in accordance with the terms of this Agreement and as specified in Subpart 12.1212 of the Federal AcquisitionRegulation (FAR), 48 C.F.R.12.1212, and its successors; and (b) for acquisition by or on behalf of the Department ofDefense (DOD) and any agencies or units thereof, as necessary to obtain protection as "commercial computer software"and related documentation in accordance with the terms of this Agreement and as specified in Subparts 227.7202-1 and227.7202-3 of the DOD FAR Supplement, 48 C.F.R.227.7202-1 and 227.7202-3, and its successors. Manufacturer isCouchbase, Inc.

Licenses

146

4. Support. Couchbase Inc. will provide Licensee with: (a) periodic Software updates to correct known bugs and errorsto the extent Couchbase Inc. incorporates such corrections into the free edition version of the Software; and (b) accessto, and use of, the Couchbase Inc. support forum available at the following URL: http://forums.membase.org. Licenseemust have Licensed Servers at the same level of Support Services for all instances in a production deployment runningthe Software. Licensee must also have Licensed Servers at the same level of Support Services for all instances in a de-velopment and test environment running the Software, although these Support Services may be at a different level thanthe production Licensed Servers. Couchbase Inc. may, at its discretion, modify, suspend or terminate support at anytime upon notice to Licensee.

5. Records Retention and Audit. Licensee shall maintain complete and accurate records to permit Couchbase Inc. to ver-ify the number of Licensed Servers used by Licensee hereunder. Upon Couchbase Inc.'s written request, Licensee shall:(a) provide Couchbase Inc. with such records within ten (10) days; and (b) will furnish Couchbase Inc. with a certifica-tion signed by an officer of Licensee verifying that the Software is being used pursuant to the terms of this Agreement.Upon at least thirty (30) days prior written notice, Couchbase Inc. may audit Licensee's use of the Software to ensurethat Licensee is in compliance with the terms of this Agreement. Any such audit will be conducted during regular busi-ness hours at Licensee's facilities and will not unreasonably interfere with Licensee's business activities. Licensee willprovide Couchbase Inc. with access to the relevant Licensee records and facilities. If an audit reveals that Licensee hasused the Software in excess of the authorized Licensed Servers, then (i) Couchbase Inc. will invoice Licensee, and Li-censee will promptly pay Couchbase Inc., the applicable licensing fees for such excessive use of the Software, whichfees will be based on Couchbase Inc.'s price list in effect at the time the audit is completed; and (ii) Licensee will payCouchbase Inc.'s reasonable costs of conducting the audit.

6. Confidentiality. Licensee and Couchbase Inc. will maintain the confidentiality of Confidential Information. The re-ceiving party of any Confidential Information of the other party agrees not to use such Confidential Information for anypurpose except as necessary to fulfill its obligations and exercise its rights under this Agreement. The receiving partyshall protect the secrecy of and prevent disclosure and unauthorized use of the disclosing party's Confidential Informa-tion using the same degree of care that it takes to protect its own confidential information and in no event shall use lessthan reasonable care. The terms of this Confidentiality section shall survive termination of this Agreement. Upon termi-nation or expiration of this Agreement, the receiving party will, at the disclosing party's option, promptly return or de-stroy (and provide written certification of such destruction) the disclosing party's Confidential Information.

7. Disclaimer of Warranty. THE SOFTWARE AND ANY SERVICES PROVIDED HEREUNDER ARE PROVID-ED "AS IS" WITHOUT WARRANTY OF ANY KIND. COUCHBASE INC. DOES NOT WARRANT THAT THESOFTWARE OR THE SERVICES PROVIDED HEREUNDER WILL MEET LICENSEE'S REQUIREMENTS,THAT THE SOFTWARE WILL OPERATE IN THE COMBINATIONS LICENSEE MAY SELECT FOR USE,THAT THE OPERATION OF THE SOFTWARE WILL BE ERROR-FREE OR UNINTERRUPTED OR THAT ALLSOFTWARE ERRORS WILL BE CORRECTED. COUCHBASE INC. HEREBY DISCLAIMS ALL WARRANTIES,EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE IMPLIED WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, TITLE, AND ANYWARRANTIES ARISING OUT OF COURSE OF DEALING, USAGE OR TRADE.

8. Agreement Term and Termination. The term of this Agreement shall begin on the Effective Date and will continueuntil terminated by the parties. Licensee may terminate this Agreement for any reason, or for no reason, by providing atleast ten (10) days prior written notice to Couchbase Inc. Couchbase Inc. may terminate this Agreement if Licensee ma-terially breaches its obligations hereunder and, where such breach is curable, such breach remains uncured for ten (10)days following written notice of the breach. Upon termination of this Agreement, Licensee will, at Couchbase Inc.'s op-tion, promptly return or destroy (and provide written certification of such destruction) the applicable Software and allcopies and portions thereof, in all forms and types of media. The following sections will survive termination or expira-tion of this Agreement: Sections 2, 3, 6, 7, 8, 9, 10 and 11.

9. Limitation of Liability. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENTWILL COUCHBASE INC. OR ITS LICENSORS BE LIABLE TO LICENSEE OR TO ANY THIRD PARTY FORANY INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL OR EXEMPLARY DAMAGES OR FOR THECOST OF PROCURING SUBSTITUTE PRODUCTS OR SERVICES ARISING OUT OF OR IN ANY WAY RE-LATING TO OR IN CONNECTION WITH THIS AGREEMENT OR THE USE OF OR INABILITY TO USE

Licenses

147

THE SOFTWARE OR DOCUMENTATION OR THE SERVICES PROVIDED BY COUCHBASE INC. HERE-UNDER INCLUDING, WITHOUT LIMITATION, DAMAGES OR OTHER LOSSES FOR LOSS OF USE, LOSSOF BUSINESS, LOSS OF GOODWILL, WORK STOPPAGE, LOST PROFITS, LOSS OF DATA, COMPUTERFAILURE OR ANY AND ALL OTHER COMMERCIAL DAMAGES OR LOSSES EVEN IF ADVISED OF THEPOSSIBILITY THEREOF AND REGARDLESS OF THE LEGAL OR EQUITABLE THEORY (CONTRACT, TORTOR OTHERWISE) UPON WHICH THE CLAIM IS BASED. IN NO EVENT WILL COUCHBASE INC.'S OR ITSLICENSORS' AGGREGATE LIABILITY TO LICENSEE, FROM ALL CAUSES OF ACTION AND UNDER ALLTHEORIES OF LIABILITY, EXCEED ONE THOUSAND DOLLARS (US $1,000). The parties expressly acknowl-edge and agree that Couchbase Inc. has set its prices and entered into this Agreement in reliance upon the limitations ofliability specified herein, which allocate the risk between Couchbase Inc. and Licensee and form a basis of the bargainbetween the parties.

10.General. Couchbase Inc. shall not be liable for any delay or failure in performance due to causes beyond its reasonablecontrol. Neither party will, without the other party's prior written consent, make any news release, public announce-ment, denial or confirmation of this Agreement, its value, or its terms and conditions, or in any manner advertise orpublish the fact of this Agreement. Notwithstanding the above, Couchbase Inc. may use Licensee's name and logo, con-sistent with Licensee's trademark policies, on customer lists so long as such use in no way promotes either endorsementor approval of Couchbase Inc. or any Couchbase Inc. products or services. Licensee may not assign this Agreement, inwhole or in part, by operation of law or otherwise, without Couchbase Inc.'s prior written consent. Any attempt to as-sign this Agreement, without such consent, will be null and of no effect. Subject to the foregoing, this Agreement willbind and inure to the benefit of each party's successors and permitted assigns. If for any reason a court of competent ju-risdiction finds any provision of this Agreement invalid or unenforceable, that provision of the Agreement will be en-forced to the maximum extent permissible and the other provisions of this Agreement will remain in full force and ef-fect. The failure by either party to enforce any provision of this Agreement will not constitute a waiver of future en-forcement of that or any other provision. All waivers must be in writing and signed by both parties. All notices per-mitted or required under this Agreement shall be in writing and shall be delivered in person, by confirmed facsimile,overnight courier service or mailed by first class, registered or certified mail, postage prepaid, to the address of the par-ty specified above or such other address as either party may specify in writing. Such notice shall be deemed to havebeen given upon receipt. This Agreement shall be governed by the laws of the State of California, U.S.A., excluding itsconflicts of law rules. The parties expressly agree that the UN Convention for the International Sale of Goods (CISG)will not apply. Any legal action or proceeding arising under this Agreement will be brought exclusively in the federalor state courts located in the Northern District of California and the parties hereby irrevocably consent to the person-al jurisdiction and venue therein. Any amendment or modification to the Agreement must be in writing signed by bothparties. This Agreement constitutes the entire agreement and supersedes all prior or contemporaneous oral or writtenagreements regarding the subject matter hereof. To the extent there is a conflict between this Agreement and the termsof any "shrinkwrap" or "clickwrap" license included in any package, media, or electronic version of Couchbase Inc.-furnished software, the terms and conditions of this Agreement will control. Each of the parties has caused this Agree-ment to be executed by its duly authorized representatives as of the Effective Date. Except as expressly set forth in thisAgreement, the exercise by either party of any of its remedies under this Agreement will be without prejudice to itsother remedies under this Agreement or otherwise. The parties to this Agreement are independent contractors and thisAgreement will not establish any relationship of partnership, joint venture, employment, franchise, or agency betweenthe parties. Neither party will have the power to bind the other or incur obligations on the other's behalf without theother's prior written consent.

11.Definitions. Capitalized terms used herein shall have the following definitions: "Confidential Information" means anyproprietary information received by the other party during, or prior to entering into, this Agreement that a party shouldknow is confidential or proprietary based on the circumstances surrounding the disclosure including, without limitation,the Software and any non-public technical and business information. Confidential Information does not include infor-mation that (a) is or becomes generally known to the public through no fault of or breach of this Agreement by the re-ceiving party; (b) is rightfully known by the receiving party at the time of disclosure without an obligation of confiden-tiality; (c) is independently developed by the receiving party without use of the disclosing party's Confidential Infor-mation; or (d) the receiving party rightfully obtains from a third party without restriction on use or disclosure. "Docu-mentation" means any technical user guides or manuals provided by Couchbase Inc. related to the Software. "LicensedServer" means an instance of the Software running on one (1) operating system. Each operating system instance may be

Licenses

148

running directly on physical hardware, in a virtual machine, or on a cloud server. "Couchbase" means Couchbase, Inc."Couchbase Website" means www.couchbase.com. "Software" means the object code version of the applicable elasticdata management server software provided by Couchbase Inc. and ordered by Licensee during the ordering process onthe Couchbase Website.

If you have any questions regarding this Agreement, please contact [email protected].

Couchbase Manual 1.8

Documents

Transcript of Couchbase Manual 1.8