Automated Detection and Classification for Packed Android...

23
Automated Detection and Classification for Packed Android Applications Yibin Liao, Jiakuan Li, Bo Li, Guodong Zhu, Yue Yin, Ruoyan Cai Network System and Security (NSS) Lab University of Georgia (UGA)

Transcript of Automated Detection and Classification for Packed Android...

Page 1: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

AutomatedDetectionandClassificationforPackedAndroid

Applications

YibinLiao,Jiakuan Li,BoLi,Guodong Zhu,YueYin,Ruoyan Cai

NetworkSystemandSecurity(NSS)LabUniversityofGeorgia(UGA)

Page 2: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Goal

• Detection– AutomaticallyidentifypackedAndroidapps

• Classification– Automaticallyclassifydifferentpackersintodifferentgroups

Page 3: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

How Android App is Built and Run

JavaSourceCode(.java)Java

Compiler

JavaBytecode(.class)Dex

Compiler

Dalvik Bytecode(.dex)

Dalvik VM

Dalvik Executable

Page 4: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Packed Android App

JavaSourceCode(.java)Java

Compiler

JavaBytecode(.class)Dex

Compiler

Dalvik Bytecode(.dex)

Dalvik VM

Dalvik Executable

Page 5: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Code Tree of Decompiled Dex

Original ijiami Bangcle Ali

Page 6: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Currentproblems

• PackedAndroidMalware• Manualeffortforanalysis– Tedious

• Packersareevolving.– Unpackingapproachesonlyworksforalimitedtime,orparticulartypeofpackers.

Page 7: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Currentpackingtechniques

• Codeobfuscation• Anti-debugging• Bytecode hiding• Dynamiccodemodification• Dynamicloading

Page 8: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Ourapproach

Detection• CombinedStaticand

Dynamicanalysis– Static

• Staticanalysistools(baksmali)

– Dynamic• DVMinstrumentation

– Compareclassesfromstaticanddynamicanalysis

Classification• Runtimeenvironment

monitoringtocapturetheexecutionbehaviorpattern– Systemcalls

• Kernelmodules

– IPCtransaction• Bindertrace

– Native-to-Javainteraction• JNItrace

Page 9: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 10: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 11: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 12: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

Static Analysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 13: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 14: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 15: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 16: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 17: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 18: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 19: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 20: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

InstrumentedDVM

Baksmali

Comparer

Classes

Smalifiles Classes

DynamicAnalysis

StaticAnalysis

AndroidApps

PackedApps

SystemCallMonitor

Native-to-JavaMonitor

BinderTransactionMonitor

BehaviorPatterns Classifier

PackedAppsSet1

DetectionModule ClassificationModule

PackedAppsSet2

PackedAppsSet3

PackedAppsSetN

Overview

Page 21: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

VMinstrumentation(DVM)

Unzipapk ->dex files

Openandreaddexfile

InitializeClasses

LoadclassesInsertcodestocapturetheclassinformation(names,methods,,fields,etc.)

Page 22: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Conclusion

• ImplementedadetectionmoduletoidentifypackedAndroidApps

• Proposedapproachestoextracttheexecutionbehaviorfromdifferentpackers

Page 23: Automated Detection and Classification for Packed Android ...cobweb.cs.uga.edu/~liao/MS2016-3035.pdf · Automated Detection and Classification for Packed Android Applications Yibin

Thankyou

QA?