HBASE Performane Test

www.vmcd.org

HBASE Performance test by YCSB

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Bulid YCSB From github

We use thumbtack modify version https://github.com/thumbtack-technology/ycsb

Modify pom.xml – using CDH platform version:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>

<groupId>com.yahoo.ycsb</groupId>

<artifactId>root</artifactId>

<version>0.1.4</version>

</parent>

<artifactId>hbase-binding</artifactId>

https://github.com/thumbtack-technology/ycsb

www.vmcd.org

<name>HBase DB Binding</name>

<repositories>

<repository>

<id>central</id>

<name>Maven Repository Switchboard</name>

<url>http://repo1.maven.org/maven2</url>

</repository>

<repository>

<id>maven-hadoop</id>

<name>Hadoop Releases</name>

<url>https://repository.cloudera.com/content/repositories/releases/</url>

</repository>

<repository>

<id>cloudera-repos</id>

<name>Cloudera Repos</name>

<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>

</repository>

</repositories>

<dependencies>

<dependency>

<groupId>org.apache.hbase</groupId>

<artifactId>hbase-client</artifactId>

<version>0.96.1.1-cdh5.0.0</version>

www.vmcd.org

</dependency>

<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-hdfs</artifactId>

<version>2.3.0-cdh5.0.0</version>

</dependency>

<dependency>

<groupId>com.yahoo.ycsb</groupId>

<artifactId>core</artifactId>

<version>${project.version}</version>

</dependency>

</dependencies>

<build>

<plugins>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-assembly-plugin</artifactId>

<version>${maven.assembly.version}</version>

<configuration>

<descriptorRefs>

<descriptorRef>jar-with-dependencies</descriptorRef>

</descriptorRefs>

<appendAssemblyId>false</appendAssemblyId>

</configuration>

<executions>

www.vmcd.org

<execution>

<phase>package</phase>

<goals>

<goal>single</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

</project>

Run mvn clean package – build HBase DB Binding successfully

Copy core-site.xml hbase-site.xml hdfs-site.xml to /home/hadoop/ycsb/hbase/src/main/conf

Load 50 millions data to usertable:

YCSB Client 0.1

Command line: -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family -p

recordcount=50000000 -s -threads 10 -p measurementtype=timeseries -p timeseries.granularity=2000 –load

[OVERALL], Reconnections, 0.0

[OVERALL], RunTime(ms), 823765.0

[OVERALL], Operations, 4.9999968E7

[OVERALL], Throughput(ops/sec), 60696.883213052264

www.vmcd.org

Run test scripts:

$ls -l |grep .sh

for mode in a b c d e f g h i j k

do

for thread in `seq 1 128`

do

/home/hadoop/ycsb/bin/ycsb run hbase -P workloads/workload"$mode" -p columnfamily=f1 -p table=test2 -s -threads "$thread" |tee

workload"$mode"_t"$thread"

done

sleep 60

done

-rw-rw-r-- 1 hadoop hadoop 58901 Sep 10 23:27 workloada_t1

………

-rw-rw-r-- 1 hadoop hadoop 58901 Sep 10 23:27 workloadk_t120









-rw-rw-r-- 1 hadoop hadoop 54592 Sep 11 09:43 load50m.data

www.vmcd.org

Workload Summary

Total 11 modes to run test script on 1~128 threads

Workload all (ops/sec a-k mode)

0

50000

100000

150000

200000

250000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

mode_a

mode_b

mode_c

mode_d

mode_e

mode_f

mode_g

mode_h

mode_i

mode_j

mode_k

www.vmcd.org

Workload A AverageLatency (us) : read50% update50%

0

5000

10000

15000

20000

25000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read_50%

update_50%

www.vmcd.org

Workload B AverageLatency (us) : read95% update5%

0

5000

10000

15000

20000

25000

30000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read_95%

update_5%

www.vmcd.org

Workload C AverageLatency (us) : read100%

0

5000

10000

15000

20000

25000

30000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read100%

read100%

www.vmcd.org

Workload D AverageLatency (us) : read95% insert5%

0

1000

2000

3000

4000

5000

6000

7000

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101105109113117121125

read 95%

insert5%

www.vmcd.org

Workload E AverageLatency (us) : scan95% insert5%

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

scan95%

insert5%

www.vmcd.org

Workload F AverageLatency (us) : read50% readmodify50%

0

2000

4000

6000

8000

10000

12000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read50%

readmodify50%

www.vmcd.org

Workload G AverageLatency (us) : read5% update95%

0

2000

4000

6000

8000

10000

12000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read5%

update95%

www.vmcd.org

Workload H AverageLatency (us) : update50% insert50%

0

500

1000

1500

2000

2500

3000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

update50%

insert50%

www.vmcd.org

Workload I AverageLatency (us) : read30% insert70%

0

2000

4000

6000

8000

10000

12000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read30%

insert70%

www.vmcd.org

Workload J AverageLatency (us) : read20% update30% insert50%

0

2000

4000

6000

8000

10000

12000

14000

16000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

read20%

update30%

insert50%

www.vmcd.org

Workload K AverageLatency (us) : update20% insert80%

0

1000

2000

3000

4000

5000

6000

7000

8000

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

81

85

89

93

97

10

1

10

5

10

9

11

3

11

7

12

1

12

5

update20%

insert80%

www.vmcd.org

HBASE ENV:

3 regionservers and 10 hdfs datanodes

#hbase version

2014-09-03 22:46:34,973 INFO [main] util.VersionInfo: HBase 0.96.1.1-cdh5.0.0

2014-09-03 22:46:34,973 INFO [main] util.VersionInfo: Subversion file:

///var/lib/jenkins/workspace/generic-binary-tarball-and-maven-deploy/CDH5.0.0-Packaging-HBase-2014-03-27_22-53-27/hbase-0.96.1.1-cdh5

.0.0 -r Unknown

2014-09-03 22:46:34,973 INFO [main] util.VersionInfo: Compiled by jenkins on Thu Mar 27 23:02:13 PDT 2014

export HBASE_OPTS="-Xloggc:/home/hadoop/logs/gc -XX:+UseConcMarkSweepGC -XX:PermSize=96m -XX:MaxPermSize=256m -Xss1m"

export HBASE_REGIONSERVER_OPTS="$HBASE_OPTS -XX:+HeapDumpOnOutOfMemoryError -Xms32g -Xmx32g -Xmn16g -XX:SurvivorRatio=2

-XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=80"

#export HBASE_REGIONSERVER_OPTS="$HBASE_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:+UseParNewGC -Xms32g -Xmx32g

-Xmn12g -XX:SurvivorRatio=10 -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+CMSClassUnloadingEnabled

-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=80 -XX:CMSFullGCsBeforeCompaction=0 -XX:+DisableExplicitGC

-Dsun.net.client.defaultConnectTimeout=10000 -Dsun.net.client.defaultReadTimeout=30000"

export HBASE_MASTER_OPTS="-Xms4g -Xmx4g"

export HBASE_LOG_DIR=/home/hadoop/logs/hbase

export HBASE_MANAGES_ZK=false

www.vmcd.org

Datanode OS system information :

#cat /etc/issue

CentOS release 6.5 (Final)

#cat /proc/cpuinfo |grep processor |wc -l

24

#cat /proc/meminfo |more

MemTotal: 99035868 kB

Mount system info:

/dev/sdb1 on /data1 type ext4 (rw)

/dev/sdc1 on /data2 type ext4 (rw)

/dev/sdd1 on /data3 type ext4 (rw)

/dev/sde1 on /data4 type ext4 (rw)

/dev/sdf1 on /data5 type ext4 (rw)

/dev/sdg1 on /data6 type ext4 (rw)

/dev/sdh1 on /data7 type ext4 (rw)

/dev/sdi1 on /data8 type ext4 (rw)

/dev/sdj1 on /data9 type ext4 (rw)

/dev/sdk1 on /data10 type ext4 (rw)

/dev/sdl1 on /data11 type ext4 (rw)

/dev/sdm1 on /data12 type ext4 (rw)

#sg_vpd /dev/sde --page=0xb1

www.vmcd.org

Block device characteristics VPD page (SBC):

Nominal rotation rate: 7200 rpm

: 3.5 inch

=== START OF INFORMATION SECTION ===

Device Model: ST2000NM0033-9ZM175

Serial Number: Z1X1YV4Q

LU WWN Device Id: 5 000c50 06672b0c2

Firmware Version: SN03

User Capacity: 2,000,398,934,016 bytes [2.00 TB]

Sector Size: 512 bytes logical/physical

Device is: Not in smartctl database [for details use: -P showall]

ATA Version is: 8

ATA Standard is: ACS-2 (revision not indicated)

Local Time is: Wed Sep 3 15:53:55 2014 CST

SMART support is: Available - device has SMART capability.

SMART support is: Enabled

HBASE Performane Test

Technology

Transcript of HBASE Performane Test