ZFS test with VM on PC

Alignment test

• Commands– cp ~/r100609_1_thy__coreRag1__Rag1_E1.fastq .– ~/src/bowtie2-2.1.0/bowtie2 -x ~/indexes/mm9 -U

r100609_1_thy__coreRag1__Rag1_E1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.sam --threads 2

• Check the process status (execution time ,CPU, memory) with “ps” command for each 10 seconds

Alignment testUncompressed Compressed

resident set size, the non-swapped physical memory that a task has used (in kiloBytes)

% CPU

Memory

I/O performance

• Copy 1GB divided by 1 – 10 blocks• Command– for count in $(seq 1 10); do for directory in

uncompressed compressed; do dd if=/dev/zero of=/mnt/$directory/foovar count=$count bs=`perl -e "print int(1000000000 / $count - 0.5) + 1"` 2>&1 | tail -n1 | sed 's/^.*, //'; done; done | paste - - > dd.txt &

I/O performanceBlocks Uncompressed Compressed

1 50.1 MB/s 59.1 MB/s

2 56.0 MB/s 54.5 MB/s

3 56.0 MB/s 57.8 MB/s

4 57.6 MB/s 52.7 MB/s

5 53.9 MB/s 58.9 MB/s

6 57.2 MB/s 61.4 MB/s

7 52.3 MB/s 60.0 MB/s

8 43.9 MB/s 60.7 MB/s

9 58.2 MB/s 57.3 MB/s

10 48.1 MB/s 61.3 MB/s

File system disk space usage (df)Filesystem 1K-blocks Used Available Use% Mounted on/dev/mapper/fedora-root 64247652 25341508 35635900 42% /devtmpfs 1017952 0 1017952 0% /devtmpfs 1025732 80 1025652 1% /dev/shmtmpfs 1025732 708 1025024 1% /runtmpfs 1025732 0 1025732 0% /sys/fs/cgrouptmpfs 1025732 12 1025720 1% /tmp/dev/sda1 487652 67503 394549 15% /boot129.112.138.227:/mnt/zfs1 203341312 8697920 194643392 5% /mnt/uncompressed129.112.138.227:/mnt/zfs2 203341376 3823808 199517568 2% /mnt/compressed

File system disk space usage (df)Filesystem 1K-blocks Used Available Use% Mounted on/dev/mapper/fedora-root 64247652 25341508 35635900 42% /devtmpfs 1017952 0 1017952 0% /devtmpfs 1025732 80 1025652 1% /dev/shmtmpfs 1025732 708 1025024 1% /runtmpfs 1025732 0 1025732 0% /sys/fs/cgrouptmpfs 1025732 12 1025720 1% /tmp/dev/sda1 487652 67503 394549 15% /boot129.112.138.227:/mnt/zfs1 203341312 2181504 201159808 2% /mnt/uncompressed129.112.138.227:/mnt/zfs2 203341376 2181376 201160000 2% /mnt/compressed

After gzipping of data files

ZFS test with Amazon EC2

Alignment test 1

• Commands (4 processes using 4 CPU cores)– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U

r100609_1_thy__coreRag1__Rag1_E1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.1.sam --threads 1 &

– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U r100609_1_thy__coreRag1__Rag1_E1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.2.sam --threads 1 &

– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U r100609_1_thy__coreRag1__Rag1_E1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.3.sam --threads 1 &

– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U r100609_1_thy__coreRag1__Rag1_E1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.4.sam --threads 1 &

Alignment test 1

• The test processes were terminated with memory error.Out of memory allocating the offs[] array for the Bowtie index.Please try again on a computer with more memory.

Error: Encountered internal Bowtie 2 exception (#1)Command: /home/ec2-user/src/bowtie2-2.1.0/bowtie2-align --wrapper basic-0 -x indexes/mm9 -S r100609_1_thy__coreRag1__Rag1_E1.1.sam --threads 1 -U r100609_1_thy__coreRag1__Rag1_E1.fastqbowtie2-align exited with value 1

Alignment test 2

• Commands (2 processes using 4 CPU cores)– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U

r100609_1_thy__coreRag1__Rag1_E1.1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.1.sam --threads 2 &

– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U r100609_1_thy__coreRag1__Rag1_E1.2.fastq -S r100609_1_thy__coreRag1__Rag1_E1.2.sam --threads 2 &

Alignment test 2Uncompressed

resident set size, the non-swapped physical memory that a task has used (in kiloBytes)

% CPU

Memory

Alignment test 2Compressed

resident set size, the non-swapped physical memory that a task has used (in kiloBytes)

% CPU

Memory

Alignment test 3

• Checking the total memory usage• Commands (1 process using 2 CPU cores)– free -t -s 1– ~/src/bowtie2-2.1.0/bowtie2 -x indexes/mm9 -U

r100609_1_thy__coreRag1__Rag1_E1.fastq -S r100609_1_thy__coreRag1__Rag1_E1.sam --threads 4 &

Alignment test 3

Uncompressed Compressed

I/O performance (Read rate)Blocks Uncompressed Compressed

1 1.1 GB/s 1.1 GB/s

2 1.6 GB/s 1.6 GB/s

3 1.9 GB/s 2.0 GB/s

4 2.2 GB/s 2.2 GB/s

5 2.3 GB/s 2.4 GB/s

6 2.4 GB/s 2.4 GB/s

7 2.5 GB/s 2.5 GB/s

8 2.6 GB/s 2.6 GB/s

9 2.6 GB/s 2.6 GB/s

10 2.6 GB/s 2.7 GB/s

I/O performance (Write rate 1)Blocks Uncompressed Compressed

1 43.6 MB/s 324 MB/s

2 1.1 GB/s 1.1 GB/s

3 1.2 GB/s 43.9 MB/s

4 42.8 MB/s 1.3 GB/s

5 46.0 MB/s 165 MB/s

6 57.1 MB/s 46.3 MB/s

7 374 MB/s 1.4 GB/s

8 1.1 GB/s 45.4 MB/s

9 42.5 MB/s 44.8 MB/s

10 510 MB/s 1.4 GB/s

I/O performance (Write rate 2)Blocks Uncompressed Compressed

1 814 MB/s 812 MB/s

2 1.1 GB/s 45.5 MB/s

3 43.3 MB/s 50.4 MB/s

4 233 MB/s 1.3 GB/s

5 1.3 GB/s 45.5 MB/s

6 41.8 MB/s 1.3 GB/s

7 98.6 MB/s 176 MB/s

8 60.4 MB/s 599 MB/s

9 47.3 MB/s 38.0 MB/s

10 1.5 GB/s 1.4 GB/s

File system disk space usage (df)Filesystem 1K-blocks Used Available Use% Mounted on/dev/xvde1 6193088 2393636 3736840 40% /none 7619740 0 7619740 0% /dev/shmtank/compressed 79563136 12784384 66778752 17% /mnt/compressedtank/uncompressed 89918080 23139328 66778752 26% /mnt/uncompressedtank 66778752 0 66778752 0% /tank

100GB disk pooling (64G + 13G + 23G)

Conclusion

• We adopted a short-read alignment step with bowtie2 to the tests of ZFS, as it would mainly utilize the computational resource including storage disk.

• The tests show that there is no considerable difference of resource usage between on uncompressed and compressed disk partitions of ZFS.– Execution time, CPU, memory, I/O performance– But it may need more tests in more computational burden to

recognize the stability of ZFS.• The adoption of compressed system makes more efficient

usage of our storage, especially with unzipped data files. (refer slide 6-7)