Riak CS in Cloudstack
-
Upload
giles-sirett -
Category
Technology
-
view
4.745 -
download
2
description
Transcript of Riak CS in Cloudstack
Simple, Available Cloud Storage For Cloudstack
Overview
On March 27, 2012
Bashoannounced a newproduct called
Riak CS
On September 5, 2012
BASHOjoined Apache Cloudstack
On March 20, 2013
Riak CSbecame open source
Riak CS is...
enterprise cloud storage
RiakS3-compatibility
multi-tenancy
per user reporting
offeri
ngbuilton topof
large object storage
Enabling you to host your own
PUBLICPRIVATE
&CLOUDS
or….Reliable Storage Behind Apps
Basho's Commits
@john_burwell 's contribution:@john_burwell 's contribution:S3-backed secondary storage feature in 4.1.0S3-backed secondary storage feature in 4.1.0Uses S3 to sync secondary storage across zonesUses S3 to sync secondary storage across zones
Long term: (shhhhhh!)Native S3 SupportFederated authentication and authorization
DataPipeblog.datapipe.com/datapipe-cloudstack
“Riak CS provides the high-performance,
distributed datastore
we need to deliver a sound foundation for
our cloud storage needs now
and for many years into the future”
- Ed Laczynski, VP Cloud Strategy, Datapipe.
Yahoo!“Today, Yahoo! leverages Riak CS Enterprise to offer an
S3-compatible public cloud storage service,
as well as dedicated hosting options ...
Yahoo! is highly supportive of open source software
and we view Basho’s (OSS) announcement as
a positive move that will work
to accelerate its ability to innovate
and ultimately strengthen our cloud platform.”
- Shingo Saito, cloud product manager, Yahoo!
About Riak
RiakDynamo-inspired key/value store
Written in Erlang with C/C++
Open source under Apache 2 license
Thousands of production deployments
RiakHigh availability
Low-latency
Horizontal scalability
Fault-tolerance
Ops friendliness
RiakMasterless• No master/slave or different roles• All nodes are equal• Write availability and scalability • All nodes can accept/route requests
RiakNo Sharding• Consistent hashing• Prevents “hot spots”• Lowers operational burden of scale• Data rebalanced automatically
RiakAvailability and Fault-Tolerance • Automatically replicates
data• Read and write data during
hardware failure and network partition
• Hinted handoff
How It Works
Riak CS
RiakStanchion
Riak CS nodefor every node of Riak1
RiakNode
RiakNode
RiakNode
RiakNode
RiakNode
Large Object
Riak CS
S3API
ReportingAPI
Riak CS
S3API
ReportingAPI
Riak CS
S3API
ReportingAPI
Riak CS
S3API
ReportingAPI
Riak CS
S3API
ReportingAPI
1. User uploads an object
1 MB
2. Riak CSbreaks object
into 1 MB chunks
1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB
3. Riak CSstreams chunksto Riak nodes
4. Riak replicatesand stores
chunks
USERSmulti-tenancy:
Riak CS will trackindividual usage/stats
BASIC
CONCEP
TS
access_key secret_key
users identified byusers authenticated by
BUCKETSusers create buckets.
buckets are like folders.
store objects in buckets.
names are globally unique.
BASIC
CONCEP
TS
OBJECTSstored in buckets.
objects are opaque.
store any file type.
BASIC
CONCEP
TS
Features
Riak CS Large Object Support• Started with 5GB / object• Now have multipart upload• Content agnostic
Riak CS S3-Compatible API• Use existing S3 libraries
and tools• RESTful operations• Multipart upload • S3-style ACLs for
object/bucket permissions• S3 authentication scheme
Riak CS Administration and Users• Interface for user creation,
deletion, and credentials • Configure so only admins can
create users
Riak CS New Stuff in Riak 1.3• Multipart upload: parts
between 5MB and 5GB • Support for GET range
queries• Restrict access to buckets
based on source IP
Riak CS
Riak CS Packages• Debian• Ubuntu• FreeBSD• Mac• Red Hat Enterprise• Fedora• SmartOS• Solaris • Source
Operations
statsDTrace
&built-in
support
track access &storage per user
monitor totalcluster ops
inspect ops with DTrace probes
OPERATIONAL STATSexposed via HTTP resource: /riak-cs/stats
block bucket object
GET, PUT, DELETELIST KEYS, CREATE,DELETE, GET/PUT ACL
GET, PUT, DELETEHEAD, GET/PUT ACL
HISTOGRAMS & COUNTERS
TRACK INDIVIDUAL USER’S
THE “USAGE”BUCKET
ACCESS STORAGE
QUERY USAGE STATS
Storage and access statistics tracked on per-user basis, as rollups for slices of time
•Operations, Count, BytesIn, BytesOut, + system and user error•Objects, Bytes
Enterprise
Multi-Datacenter Replication
• For active backups, availability zones, disaster recovery, global traffic
• Real-time or full-sync• 24/7 support• Per-node or storage-based
pricing
basho.com
SIGN UP FOR AN ENTERPRISE DEVELOPER
TRIAL http://docs.basho.com/
Riak LondonA distributed systems
meet/drink up
www.meetup.com/riak-london
github.com/bashotwitter.com/
bashodocs.basho.com
Q&A@_stu_