Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and...
Transcript of Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and...
![Page 1: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/1.jpg)
FFRI,Inc.
Fourteenforty Research Institute, Inc.
FFRI,Inc. http://www.ffri.jp
Automated on-execute test using VirtualBox
Junichi Murakami Executive Officer, Director of Advanced Development Division
Ver2.00.01
![Page 2: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/2.jpg)
FFRI,Inc.
1. Background and motivation
2. Overview of a test
– automated on-execute test
– virtualization software and automation methods
– Oracle VM VirtualBox and its automation
– example of VBoxManage
3. Automation script
– FFRI AutoMonkey
– design concept
– throughput
– performance
4. References
5. Contact information
Agenda
2
![Page 3: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/3.jpg)
FFRI,Inc.
• Automated test against a large amount of malware is required to evaluate a malware detection engine
• Testing methods are classified into on-demand and on-access testing
• on-execute test which is a kind of on-access test has to execute malware one by one
• Therefore automation based on virtualization is required
• This slides describes automated on-execute test method using VirtualBox
1.Background and motivation
3
malware scanning on-demand
on-access on-read/write
on-execute
* type of malware scanning
![Page 4: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/4.jpg)
FFRI,Inc.
• Basic steps are following
1. Copy malware into a guest
2. Execute copied malware in the guest
3. Analyze or detect malware in the guest
4. Preserve the result after execution is terminated
5. Revert the guest back to original condition
6. Go to 1.
• Required functions to execute above are following
a. Copying a file to a guest from a host (copy-to)
b. Executing arbitrary a program in a guest from a host(exec)
c. Copying a file from guest to a host (copy-from)
d. Reverting a guest condition based on a snapshot(revert)
2.1.Automated on-execute testing
4
→ All functions can be achieved by making a communication interface between a host
and a guest using TCP/IP. We considered the way we do not need to involve
developing software as possible as we could
![Page 5: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/5.jpg)
FFRI,Inc.
software Licence copy-to copy-from exec revert method
VMware Workstation Proprietary ○ ○ ○ ○ VIX API
VMware ESX(#1) Proprietary
○ ○ ○ ○ VIX API
Oracle VM VirtualBox GPL2 ○ ○ ○ ○ VBoxManage
QEMU + KVM GPL2(#2) × × × ○ Libvirt
2.2.virtualization software and automation methods
5
#1 ESXi can also use VIX API for 60days by registering a evaluation license.
#2 KVM's parts are licensed under various GNU licenses(GPL, GPL2, LGPL2, etc.)
• Use functions which virtualization software has natively
• VMware(licensed) and VritualBox have all the features we need → We considered using VirtualBox because of the cost advantage
• QEMU+KVM can be used by 3rd party software(ex: libguestfs + winexe)
– “Malware Analysis: Collaboration, Automation & Tuning”, Shmoocon 2013 http://www.slideshare.net/xabean/malware-analysis-16674048
![Page 6: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/6.jpg)
FFRI,Inc.
• A kind of x86 virtualization software, currently developed by Oracle
• Version 4.0 and later, fully open source software (GPL2)
• Supporting various host and guest environments
– HostOS:Windows, Linux, Mac OS X, Solaris
– GuestOS:Windows, Linux, FreeBSD, OpenBSD, Mac OS X Server, Solaris,etc.
• CLI is available (VBoxManage), friendly to automation
– startvm , pause, resume, poweroff, clonevm, showvinfo
– copyto, copyfrom, exec
– taking snapshot and reverting
– control virtual machine devices status, etc.
2.3.Oracle VM VirtualBox and its automation
6
![Page 7: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/7.jpg)
FFRI,Inc.
2.4.Example of VBoxManage
7
% vboxmanage startvm vm
% vboxmanage controlvm vm poweroff
% vboxmanage snapshot vm restore snapshot-1
% vboxmanage guestcontrol exec vm --image “c:/windows/system32/calc.exe” ¥
--username admin --timeout 60000 --wait-exit
% vboxmanage guestcontrol vm copyto “/some/file” “c:/file.txt” --username admin
* starting a guest
* power off a guest
* reverting a guest based on a snapshot
* execute a program in a guest from a host
* copying a file to a guest from a host
![Page 8: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/8.jpg)
FFRI,Inc.
• Automation script using VBoxManage, just a shell script
– auto-monkey.sh:automation for copy, exec, copy, revert steps
– watch-monkey.sh:watch dog script for the monkey
• It can execute multiple test simultaneously, works individually
• Published at our website below, see README for the detail (License: BSD)
– http://www.ffri.jp/research/freeware.htm
3.1.FFRI AutoMonkey
8
VirtualBox(guest)
malware
log
auto-monkey.sh
watch-monkey.sh
VirtualBox(guest)
malware
log
auto-monkey.sh
watch-monkey.sh
image
snapshot
image
snapshot
![Page 9: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/9.jpg)
FFRI,Inc.
• conform to KISS principle
• Estimation of remaining time is important for this kind of test
– we cannot determine when it would finish if the script hangs up
• Stability of VBoxManage (and VIX API) is the lifeline for the automation
• In fact, error occurs when it runs long time
– Failure by error
• exits immediately
• resumed a test automatically by watch-monkey.sh
– Hanging up(stuck) by error
• watch-monkey.sh monitors lifetime of a VirtualBox process
• if it is stuck, kill and resume
3.2.Design concept
9
![Page 10: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/10.jpg)
FFRI,Inc.
• Testing under 1host and 7guest environment
• Processed 20,000 malware, each execution time was 60 seconds
– total elapsed time: 37h15m
– throughput:8.95 malware/minute # if malware execution terminated less than 60 seconds, the script processes next item.
• Host and guest environment is following
3.3.Throughput
10
Hardware CPU: Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz
Memory: 8GB
HDD: 1.8TB x 1
Host OS Ubuntu 13.04 + VirtualBox 4.2
Guest OS Windows XP SP3(x86) + FFR yarai 2.3
CPU:1 CPU
Memory:750MB
![Page 11: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/11.jpg)
FFRI,Inc.
• About 70% of total processor is idle state(each core also indicates same trend)
3.4.performance - processor
11
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0:0
5
0:5
5
1:4
5
2:3
5
3:2
5
4:1
5
5:0
5
5:5
5
6:4
5
7:3
5
8:2
5
9:1
5
10
:05
10
:55
11
:45
12
:35
13
:25
14
:15
15
:05
15
:55
16
:45
17
:35
18
:25
19
:15
20
:05
20
:55
21
:45
22
:35
23
:25
load
all processsor statistics
%idle
%steal
%iowait
%system
%nice
%user
![Page 12: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/12.jpg)
FFRI,Inc.
• Consuming about 80% - 90% memory steadily
– real memory usage is between 2.5 and 4.0GB
3.4.performance - memory
12
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
4000000
4500000
5000000
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0:0
5
1:0
5
2:0
5
3:0
5
4:0
5
5:0
5
6:0
5
7:0
5
8:0
5
9:0
5
10
:05
11
:05
12
:05
13
:05
14
:05
15
:05
16
:05
17
:05
18
:05
19
:05
20
:05
21
:05
22
:05
23
:05
memory utilization statistics
kbmemused
kbmemfree
kbmemused-(kbbuffers+kbcached)
![Page 13: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/13.jpg)
FFRI,Inc.
• Disk busy ratio(%util) stays around 30% steadily
• The number of queued requests is between 4 and 8
3.4.performance – Disk IO
13
0
10
20
30
40
50
60
70
80
90
0
2
4
6
8
10
12
0:0
5
0:5
5
1:4
5
2:3
5
3:2
5
4:1
5
5:0
5
5:5
5
6:4
5
7:3
5
8:2
5
9:1
5
10
:05
10
:55
11
:45
12
:35
13
:25
14
:15
15
:05
15
:55
16
:45
17
:35
18
:25
19
:15
20
:05
20
:55
21
:45
22
:35
23
:25
0:0
0
Usage of block device(/dev/sda)
avgqu-sz
%util
![Page 14: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/14.jpg)
FFRI,Inc.
• None of CPU, memory and IO wasn’t bottleneck under 1host and 7guest environment
• It seems we can add some more guests up to around 10 VMs according to memory usage
• However, we have to consider requirement of a process which is executed in a guest (cpu, memory)
3.4.performance - consideration
14
![Page 15: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/15.jpg)
FFRI,Inc.
• http://www.ffri.jp/assets/files/research/freeware/FFRIAutoMonkey-1.0.tgz
• https://www.virtualbox.org/manual/UserManual.html
• http://www.slideshare.net/xabean/malware-analysis-16674048
• http://www.youtube.com/watch?v=peHdyUlchSM
• http://libguestfs.org/
• http://sourceforge.net/projects/winexe/files/
References
15
![Page 16: Fourteenforty Research Institute, Inc. Automated on ... · 2.2.virtualization software and automation methods 5 #1 ESXi can also use VIX API for 60days by registering a evaluation](https://reader034.fdocuments.net/reader034/viewer/2022050416/5f8c6b9742f168032261ee8c/html5/thumbnails/16.jpg)
FFRI,Inc.
– @FFRI_Research
Contact Information
16