27th July 2012Yosuke Hara

You can rely on whenever you need.

Yosuke HaraJuly 28th, 2012

We launched LeoFS as OSS on July 4th.LeoFS - http://www.leofs.org

Motivation

1. Low ROI2. Possibility of SPOF3. Storage Expansion is difficult during increasing data

?

Motivation

NFS - Very Expensive

X 3.5 762B

220B

Motivation

Object Storage Farm

S3FS-C

Object Storage Farm

Unstructured data

8

Object Storage Farm

Overview

LeoFS Overview

LeoFS-Manager

LeoFS-Gateway

LeoFS-Storage

REST over HTTP

RPC

Request from Web Application(s) or Browser

META Object Store

Storage Engine/Router

META Object Store META Object Store

RPC

Storage Engine/Router Storage Engine/Router

Load Balancer

S3-API

SNMP

GUI Console

LeoFS Overview

GatewayHTTP

Request/Response Handling+

w/Object Cache(AWS S3-API)

ManagerCluster

Management

Ring WatcherNode Watcher

StorageObject Storage, Meta data Storage

+Replicator/Recoverer

LeoFS Overview

LeoFS-Manager

LeoFS-Gateway

LeoFS-Storage

REST over HTTP

RPC

Request from Web Application(s) or Browser

META Object Store

Storage Engine/Router

META Object Store META Object Store

RPC

Storage Engine/Router Storage Engine/Router

[Load Balancer]

S3-API

SNMP

GUI Console

LeoFS Overview

Minimum for Service (App) Dev

Manager x 1 Gateway x 1 Storage x 1

50TB Storage System (# of replicas = 3)

Manager x 2 Gateway x 3 .. Storage x 8 - 15

10TB .. 20TB / server

300TB Storage System (# of replicas = 3)

Manager x 2 Gateway x 4 .. Storage x 45 - 90

10TB .. 20TB / server

LeoFS Overview - Example of System layouts

XEN, VMWare,...

XEN, VMWare,...

Inside LeoFS

Manager Cluster

Storage Cluster

Gateway

Process Monitor

Erlang RPC

HTTP

Erlang RPC

Erlang RPC

LeoFS Architecture

Object Cache

Erlang/OTP Runtime

Gateway REST over HTTP (S3-API)

membership (fault-detection)

redundant-manager replicator

queue

Storage Engine

Object Storage Metadata

Storage

read-repairer

redundant-manager membership (fault-detection)

get put delete head

RPC

RPC RPC

LeoFS Architecture - Gateway / Storage

Erlang Mnesia

Gateway(s) / Storage Cluster

RING, Member / Cluster State, AuthProcess Monitor

LeoFS Architecture - Manager

LeoFS Gateway

Gateway

From Applications (Clients)

replicate when using RPC

Consistent Hashing - Horizontal Distribution

Erlang RPC

S3-API

Object Cache

Storage Nodes

LeoFS Gateway

Storage Engine

...

LeoFS Storage

Object Storage

LeoFS Storage Engine

Metadata : Keeps an in-memory index of all data.Object Storage : Log structured (append-only) object store.

Request From Gateway

replicatorrepairer

queue

< META DATA >IDFilenameOffsetSizeChecksum Metadata

Data

Object Container

Header

File

Footer

< META DATA >IDFilenameOffsetSizeChecksum

LeoFS Storage Engine

LeoFS Storage Engine - Retrieve an object from the storage

Log Structured File System

Append an object

Data

Add a Metadata

LeoFS Storage Engine - Insert an object into the storage

Metadata

Log Structured File System

Compaction

OLD Object Container

NEW Object Container

LeoFS Storage Engine - Reduce unnecessary objects from the storage

Log Structured File System

LeoFS Manager

Manager

For AdministratorsLeoFS Manager

monitor

operate

RING, Node State

status, suspend,resume, detach, whereis, ...

Future Works

Future Works - Leo Tamer

Operation Center

Log Analyzer / Searcher

Future Works - Integrated HBase with LeoFS

+“Cloud DWH” =

Wrap Up

LeoFS can realize 3-HIGHs - HIGH Cost performanceHIGH ReliabilityHIGH Scalability

We set sail for our goal.

LeoFS - http://www.leofs.org