Large-Scale Decentralized Storage Systems used by Volunteer ComputingFinal Presentation

G7: Arinto, Diego, Enkhjin, Iuliia

Outline

● Problem Definition● Volunteer Computing (VC)● Decentralized Storage Systems

(DSS)● Systems Review● State-of-the-Art DSS used by VC● Conclusions

Problem Definition ● Decentralized Storage Systems ● Volunteer Computing ● DSS + VC = ?

Problem Definition

Problem Definition

Volunteer Computing● Group of computing resources● Voluntary share the resource● Incentive volunteers!!● Trust between peers● Example:

○ SETI@home○ Storage@home○ Folding@home

● However, data is centrally stored

Volunteer ComputingBOINC example

Decentralized Storage Systems

Decentralized Storage Systems ReviewFocus Criteria:

AV = availabilitySC = scalabilityECO = eventual consistencyP = performanceSE = security

Decentralized Storage Systems Review

System Description Focus

Farsite Large scale persistent P2P storage system SE

Ivy P2P storage system based on Dhash table logs

ECO, AV

OverNet P2P DHT-based storage system that used XOR-based metrics for routing

ECO

PAST Large scale persistent P2P storage system SC, SE, AV

Pastis Highly scalable P2P storage system based on trust

SC, ECO

Voldemort Big, distributed, fault tolerant hashtable AV

OceanStore Globally persistent DSS for any-file-anywhere SC, SE

Decentralized Storage Systems Review

System Description Focus

Glacier Highly durable DSS for archiving, no remote write SC, AV, P

Total Recall P2P storage system. Automatically manage its availability in dynamic changing environment

AV, SC

Cassandra DSS with no point of failure to store structured data

SC, AV, ECO

Riak Dynamo-inspired NOSQL storage system. SC, AV

Dynamo Large scale DSS developed by Amazon P, AV, ECO

Attic Secure DSS and BitTorrent-like file swarming techniques to serve data and manage load.

SE, AV

Squirrel Decentralized Web Cache P, ECO

State-of-The-Art DSS- Characteristics● Read and Write Access

● Fault Tolerance and Replication Techniques

● Availability and Symmetry

● Incentives

● Video/Images

State-of-The-Art DSS- Characteristics

State-of-The-Art DSS- Suggestions for IncentivesIncentives based on amount of storage dedicated, computing resources volunteered: ● Credit based system● Printable Certificates ● Government tax break, possible in countries

with significant amount of nodes

State-of-The-Art DSS- Challenges

● Providing the right Incentive ● Security ● Integration into current VC environment

Conclusions● VC data storage/distribution issues.

● Survey on DSS○ Classification

● Integration VC systems - DSS○ Identify characteristics ○ Valid and Useful / Define state-art○ Challenges

● Tradeoffs ○ functionality and complexity

Work Effort Arinto Murdopo = 100%Diego Montero = 100%Enkhjin Bayarsaikhan = 100%Iuliia Proskurnia = 100%

Questions

...???

Decentralized Storage Systems

● Decentralization

● Availability○ Fault-Tolerance

● Consistency○ Eventual consistency

● Scalability

● Consistent Hashing● Active/Passive

Replication● Failure-handling

(gossip-based)● Logging● Ring locality