[D25] 分散Key-Valueストア「okuyama」&「Riak」の同時書込み性能検証 by Yusuke...

49
© Hitachi Solutions, Ltd. 2014. All rights reserved. 分散Key-Valueストア okuyama& Riak書込み性能検証 株式会社 日立ソリューションズ オープンソース技術開発センタ 2014/6/19 倉又 裕輔

description

 

Transcript of [D25] 分散Key-Valueストア「okuyama」&「Riak」の同時書込み性能検証 by Yusuke...

  • 1. Hitachi Solutions, Ltd. 2014. All rights reserved. Key-Value okuyama & Riak 2014/6/19
  • 2. Hitachi Solutions, Ltd. 2014. All rights reserved. 1 NoSQL
  • 3. Hitachi Solutions, Ltd. 2014. All rights reserved. . . . Contents .
  • 4. Hitachi Solutions, Ltd. 2014. All rights reserved. . okuyamaRiak NoSQLKey-Value
  • 5. Hitachi Solutions, Ltd. 2014. All rights reserved. . IT 4 IoT 201720% 2014417 http://www.gartner.co.jp/press/html/pr20140417-01.html
  • 6. Hitachi Solutions, Ltd. 2014. All rights reserved. .. IoT 5 / A B C
  • 7. Hitachi Solutions, Ltd. 2014. All rights reserved. .. NoSQL 6 RDB SQL NoSQLNot only SQL
  • 8. Hitachi Solutions, Ltd. 2014. All rights reserved. .. NoSQL1/2 7 NoSQL MongoDB CouchDB Couchbase Server Key-Value KVS okuyama Riak memcached Redis Cassandra Hbase Hypertable Key Value Key Value KVS
  • 9. Hitachi Solutions, Ltd. 2014. All rights reserved. KVS .. NoSQL2/2 K V S 8 okuyama Riak memcached Redis
  • 10. Hitachi Solutions, Ltd. 2014. All rights reserved. KVS .. NoSQL2/2 K V S 9 okuyama Riak memcached Redis KVS okuyamaRiak
  • 11. Hitachi Solutions, Ltd. 2014. All rights reserved. .. okuyama 10 okuyama GPL v3 Java - EC ( http://okuyama-project.com/ja/index.html ) -
  • 12. Hitachi Solutions, Ltd. 2014. All rights reserved. .. Riak 11 Riak Basho Technologies, Inc. Apache License, Version 2.0 Erlang/C P2P EC Basho Technologies ( http://basho.co.jp/riak/ ) P2P
  • 13. Hitachi Solutions, Ltd. 2014. All rights reserved. .
  • 14. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 13 KVS
  • 15. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 14 / KVS Key Value Insert
  • 16. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 15 okuyamaRiak Disk Key Value Disk ValueKey Key Value KVS KVS KVS
  • 17. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 16 Disk Key Value Disk ValueKey KVS KVS
  • 18. Hitachi Solutions, Ltd. 2014. All rights reserved. okuyama v0.9.5 1/ .. 17 Riak v1.4.6 1/ CPU Intel(R) Core(TM) i5 CPU 520 4GB HDDSATA 750GB, 5400RPM OS CentOS 6.5 64bit
  • 19. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 18 https://github.com/brianfrankcooper/YCSB/wiki Yahoo! Cloud Serving Benchmark YCSB Yahoo! Inc. Apache License, Version 2.0 Java 0.1.4 NoSQL Yahoo! Cloud Serving Benchmark YCSB
  • 20. Hitachi Solutions, Ltd. 2014. All rights reserved. .. YCSB 19 START NoSQL Key Value YCSB NoSQL END KeyValue 25 okuyamaRiak 80
  • 21. Hitachi Solutions, Ltd. 2014. All rights reserved. . okuyama Riak okuyama Riak
  • 22. Hitachi Solutions, Ltd. 2014. All rights reserved. okuyama .. - okuyama 21 Disk Key Value -
  • 23. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - okuyama 22 okuyama - 0 10 20 30 40 50 60 70 80 90 0 50 100 150 200 250 300 [ops/sec] 84ops/sec
  • 24. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - okuyama 23 okuyama - 0 2000 4000 6000 8000 10000 12000 0 50 100 150 200 250 300 [ms]
  • 25. Hitachi Solutions, Ltd. 2014. All rights reserved. Riak .. - Riak 24 Disk Key Value P2P
  • 26. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - Riak 25 Riak - 0 50 100 150 200 250 0 50 100 150 200 250 300 350 400 [ops/sec] 220ops/sec
  • 27. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - Riak 26 Riak - 0 500 1000 1500 2000 2500 0 50 100 150 200 250 300 350 400 [ms]
  • 28. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - 27 I/O 100200 ops/sec okuyama 84ops/sec Riak 220 ops/sec
  • 29. Hitachi Solutions, Ltd. 2014. All rights reserved. okuyama .. - okuyama 28 Disk Value Key -
  • 30. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - okuyama 29 okuyama - 0 5000 10000 15000 20000 25000 30000 0 100 200 300 400 500 600 700 ops/sec sec
  • 31. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - okuyama 30 okuyama - 0.1 1 10 100 0 100 200 300 400 500 600 700 ms sec
  • 32. Hitachi Solutions, Ltd. 2014. All rights reserved. 0 5000 10000 15000 20000 25000 0 2000 4000 6000 8000 10000 12000 ops/sec sec .. - okuyama 31 okuyama - 6585ops/sec 10,0006,000ops/sec
  • 33. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - okuyama 32 okuyama - 1 10 100 1000 10000 0 2000 4000 6000 8000 10000 12000 ms sec 20ms 1,000ms
  • 34. Hitachi Solutions, Ltd. 2014. All rights reserved. Riak .. - Riak 33 Disk Value Key P2P
  • 35. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - Riak 34 Riak - 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 50 100 150 200 250 300 ops/sec sec Riak
  • 36. Hitachi Solutions, Ltd. 2014. All rights reserved. Riak .. - Riak 35 25% 25% 25% Disk Value Key 25% P2P
  • 37. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - Riak 36 Riak - 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0 20 40 60 80 100 120 140 ops/sec sec
  • 38. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - Riak 37 Riak - 1 10 100 10 30 50 70 90 110 130 150 ms sec
  • 39. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - Riak 38 Riak - 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0 2000 4000 6000 8000 10000 12000 14000 16000 ops/sec sec 2,200ops/sec 8,000ops/sec
  • 40. Hitachi Solutions, Ltd. 2014. All rights reserved. 1 10 100 1000 10000 0 2000 4000 6000 8000 10000 12000 14000 16000 ms sec .. - Riak 39 Riak - 15ms 50ms
  • 41. Hitachi Solutions, Ltd. 2014. All rights reserved. .. - 40 okuyama10,000 ops/sec 6,000 ops/sec Riak 8,000 ops/sec
  • 42. Hitachi Solutions, Ltd. 2014. All rights reserved. .
  • 43. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 42 okuyama10,0006,000 ops/sec Riak 8,000 ops/sec I/O 100200 ops/sec okuyama 84 ops/sec Riak 220 ops/sec
  • 44. Hitachi Solutions, Ltd. 2014. All rights reserved. .. 43 I/O 1) 2) KVS 1) 2) KVS
  • 45. okuyama Riak Basho Technologies, Inc. Cassandra Apache Software Foundation HBase Apache Software Foundation MongoDB MongoDB, Inc. CouchDB Apache Software Foundation IntelIntel Core Intel Corporation OracleJavaOracle Corporation Yahoo! Yahoo! Inc.
  • 46. Hitachi Solutions, Ltd. 2014. All rights reserved. 45 Disk Disk Value Key KeyValue Key Value Key Value KeyValue KVS KVS KVS Disk Key Value
  • 47. Hitachi Solutions, Ltd. 2014. All rights reserved. YCSB 46http://labs.yahoo.com/files/ycsb-v4.pdf YCSB NoSQL CRUD (IP) NoSQL NoSQL CRUD MongoDB HBase okuyama Riak okuyamaRiak
  • 48. Hitachi Solutions, Ltd. 2014. All rights reserved. KVSJava okuyama, Riak 47 public int insert(String table, String key, HashMap values) { com.mongodb.DB db = null; try { db = mongo.getDB(database); db.requestStart(); DBCollection collection = db.getCollection(table); DBObject r = new BasicDBObject().append("_id", key); for(String k: values.keySet()) { r.put(k, values.get(k).toArray()); } WriteResult res = collection.insert(r,writeConcern); return res.getError() == null ? 0 : 1; } catch (Exception e) { System.err.println(e.toString()); return 1; } finally { if (db!=null) { db.requestDone(); } } } public int insert(String table, String key, HashMap values) { try { boolean setResult = okuyamaClient.setValue(key, key); if (setResult) { System.out.println("setKey = "+key); } else { System.out.println("setValue Result = [Server Error]"); System.exit(1); } } catch(OkuyamaClientException oc) { oc.printStackTrace(); } return 0; } NoSQL okuyamaRiak KVSInsert okuyamaRiak80 KeyValue25 okuyama okuyama Riak Java https://github.com/basho/riak-java-client/tree/1.4.2
  • 49. Hitachi Solutions, Ltd. 2014. All rights reserved. 48 YCSBKVSsar KVS KVS KVS