A First Look at Deep Learning Apps on Smartphones
Mengwei Xu, Jiawei Liu, Yuanqiang Liu, Felix Xiaozhu Lin,
Yunxin Liu,Xuanzhe Liu
Presented: Rahul Bagchi
Content
▶ Introduction
▶ Motivation, background, and the problem
▶ Solution
❖ Core idea and the technique used
❖ The workflow of the analyzing tool
❖ Characteristics of early adopter apps using Deep Learning
❖ Deep Learning Frameworks used by the apps
❖ How are the Deep Learning capabilities used in the apps
❖ Deep Learning models used by the apps
❖ Model Resource Footprint
❖ Security and optimization of the Deep Learning models
▶ Findings and Conclusion
2
Introduction
▶ This presentation explains in a simple way the Smart-Device research
article - A First Look at Deep Learning Apps on Smartphones
▶ The article analyses 16,500 Android apps dataset and focuses on apps
that embrace Deep Learning, and the Deep Learning frameworks and
models used in practice
▶ The article aims to bridge the knowledge gap between research and
practice to find out:
❖ Characteristics of early adopter apps using Deep Learning
❖ Deep Learning Frameworks used by the apps
❖ How are the Deep Learning capabilities used in the apps
❖ Deep Learning models used by the apps
❖ Security and optimization of the Deep Learning models3
Introduction
▶ The article findings have direct implications to:
❖ App developers,
❖ Mobile Deep Learning framework developers and hardware
engineers/ vendors
❖ Mobile Deep Learning researchers
▶ To find answer to the article problem the authors uses a software tool
built by them
▶ The tool indirectly sniffs Deep Learning usage of mobile apps by
detecting the use of Deep Learning frameworks
4
Motivation, background, and the problem
▶ Deep Learning has revolutionized many Artificial Intelligence tasks
particularly in computer vision and Natural Language Processing (NLP)
▶ The huge smartphones market provides very promising platform for Deep
Learning based apps
▶ Year 2017 marked noticeable increased use of Deep Learning for
smartphones
▶ Around the same time major vendors like Google, Facebook, Apple and
Baidu launched their mobile Deep Learning frameworks like TFLite,
Caffe2, Core ML and MDL
▶ These frameworks provided affordable on-device rendering of the Deep
Learning inference
5
Motivation, background, and the problem
▶ To bridge the knowledge gap between research and practice, the article
presents the first empirical study on 16,500 popular Android apps to find
out how Deep Learning is used by smartphone apps based on the queries
below:
❖ Characteristics of early adopter apps using Deep Learning
❖ Deep Learning Frameworks used by the apps
❖ How are Deep Learning capabilities used in the apps
❖ What are the popular Deep Learning models used by the apps
❖ Security and optimization of the Deep Learning models
6
Solution
Core idea and the technique used
▶ To automate the analysis of numerous Android apps, a new analyzer tool
was built by the authors
▶ The tool inspects app installation packages, identifies the apps that use
Deep Learning, identify Deep Learning functions and the usage of known
Deep Learning frameworks, and extracts the Deep Learning models from
these apps for inspection
▶ The analysis focuses on apps that embrace Deep Learning, and the Deep
Learning frameworks and models used
▶ The overall workflow of the analyzing tool is depicted in the next slide7
Solution
▶
8
The workflow of the analyzing tool
Characteristics of early adopter apps using Deep Learning
9
Solution
Deep Learning (DL) based apps are significantly more popular than non DL apps
Deep Learning Frameworks used by the apps
10
Solution
How are the Deep Learning capabilities used in the apps
11
Solution
Deep Learning models used by the apps
Convolution Neural Network containing convolution and pooling layers
12
Solution
▶ Among the DL models extracted
❖ 87.7% models are CNN models
❖ 7.8% models are RNN models
❖ Other models were not confirmed at that time
Deep Learning models used by the apps
Recurrent Neural Network
▶ Recurrent neural network processing sequential inputs
Wikipedia
13
Solution
Model Resource Footprint
14
Solution
▶ Deep Learning models are very lightweight on memory usage and computation
complexity, with median value of 2.47 MB and 10M FLOPs respectively
▶ Running such models on mobile processors is inexpensive
▶ Findings of dominant lightweight Deep Learning models on smartphones:
❖ Deep Learning inference can be as cheap as a few MBs of memory overhead and
❖ Tens of ms execution delay
Optimization of the Deep Learning models
15
Solution
▶ Most DL models lacks optimizations despite availability of well-known
optimizations, e.g. quantization
▶ Quantization can significantly reduce Deep Learning cost with little accuracy loss
▶ The study finds only 6% of models has such optimizations
Security of the Deep Learning models
16
Solution
▶ DL models are poorly protected
▶ The study finds that only 39.2% discovered models are obfuscated and
only about 19.2% models are encrypted
▶ The remaining models were easy to extract and therefore
vulnerable to unauthorized reuse
Findings and Conclusion
▶ The authors carried out the first large-scale study of 16,500 Android
mobile apps to find out how the apps exploit Deep Learning
▶ Findings of the empirical study projects a promising picture of Deep
Learning for smartphones:
❖ showing the prosperity of mobile Deep Learning frameworks available
❖ the prosperity of apps building their cores on top of Deep Learning
❖ on-device Deep Learning inference is fast becoming popular due to stronger
privacy protection, resilient against poor Internet connectivity, and lower
cloud computing cost
❖ massive scope for optimizations on Deep Learning models
❖ huge potential for expanding the footprint of protection for Deep Learning
models
❖ validation of research ideas on these models17
Thank You
18
Top Related