Post on 03-Jun-2015
description
Tue-4-Mar, 11:15am, Khasim Syed Mohammed
LCA14-205: Optimizing SQLite for Android mobile
www.linaro.org
Boot Time Reduction investigation
Back ground of this work
FutureQ1 Q220132014
12-DEC-2013
Android Optimizations
Cortex-A7/A15 Investigation
Q3 Q4
Power Optimization investigation
fdlibm
sqlite
Migrate AOSP to latest
zlib
guava
LLVM / Clang Android
explore libart
system start-upA7/A15 Optimized bionic
LLVM wrapper
LLVM patches
profile and benchmark LLVM and GCC
NEON MM libraries
Open Crypto librariesOptimize Speech compcompiler-rt
Q4
Closed
Development
Drafting
Community/External
Upstream
Approved
Scheduled
Ongoing
www.linaro.org
Agenda
Progress update■ What’s the approach■ Our findings so far■ Remaining work
NOTE : This is still a work in progress - estimated date of closure end of April 2014
www.linaro.org
What is SQLite ?
■ SQLite is a software library that implements a self-contained, serverless, zero-configuration,transactional SQL database engine.
■ SQLite is the most widely deployed SQL database engine in the world.
■ The source code for SQLite and more information available on sqlite.org
www.linaro.org
Do we really need database
in
Mobile devices ?
Do we need database in Mobile device ?
Source : http://gigaom.com/2013/05/11/welcome-to-the-new-and-fast-growing-ecosystem-of-mobile-business-apps/
- Mobile apps are transforming the business software market- Business / Enterprise apps are getting built for “mobile first”
- Most of these applications need data base management system on mobile
-basic requirement: Large/Small amount of data should be stored and important data should be retrieved from a large database in the least possible time.
• Explore the folder external/sqlite in Android sources
• Finding possible ways of improving the overall performance of SQlite
• Ensure that we don’t break any compatibility
Expectation
• Understand how SQLite operates and find it’s dependencies.
Our approach
Commands
Use CPU to run Queries
eMMC access to store or retrieve information
• Possible optimizations areas : • Data : Improve data throughput• CPU : Utilize CPU efficiently to run queries faster
and reduce CPU utilization by offloading the SQL queries to other cores if possible.
• RL Benchmark is an SQLite benchmark that runs thousands of insert, select, update, and delete functions - the same functions Android uses to store data.
• The resulting number is the total amount of time (in seconds) that it took for your device to run through all the tests.
• The faster it completed it, the faster your device stores and retrieves data; thus, lower is better.
Tools / Measurement
Also looked at SQL Bench, Andro Bench, but this one was most widely used by app developers.
• The following functions from cortex-strings optimizations from Linaro perform better than AOSP
• strlen• memchr• memcpy
Learnings
Test with out optimization with optimization
500000000 * strlen(15*a) 8.563289 7.3658144
1000000 * strlen(15000*a) 4.795874 4.4586328
10000000 * memcpy of 15000 bytes 14.0750624 12.276912
900000000 * memcpy of 15 bytes 3.1883678 3.1854456
https://wiki.linaro.org/Platform/Android/CortexStringsInBionic
Thanks to Bero @ Linaro.org
NOTE: SQLite source has around a. 240 “memcpy”b. 60 “strlen”
• Accelerating SQL Database Operations on a GPU is possible (Example with CUDA) - Thanks to Tom Gall @ Linaro.org for sharing the information
• http://pbbakkum.com/db/
• Nexus 10 device can be prevented from suspending or entering lower power modes by
echo benchmark >/sys/power/wake_lockcd /sys/devices/system/cpu/cpu0/cpufreqecho userspace >scaling_governorecho 1700000 >scaling_max_freqecho 1700000 >scaling_min_freqecho 1700000 >scaling_setspeed
• From 4.3 onwards Android supports FSTRIM, that basically calls DISCARD command - This marks all the unused blocks on the underlying flash device as unused, so the device performs better.
Learnings
• On Nexus 10 with AOSP master + Linaro patches for BIONIC optimizations
• Preventing the device to enter lower power states
Performance improved by 15 %
Observation
BIONIC (memcpy, strlen) optimizationsTest AOSP
4.4.2Master + BIONIC optimizations
Master + BIONIC optimization + suspend disabled
1000 INSERTs 8.3 7.328 7.027
25000 INSERTs in a transaction 2.1 2.026 1.997
25000 INSERTs into an indexed table
2.5 2.012 1.957
100 SELECTs without an index 0.042 0.012 0.008
100 SELECTs on a string comparison
0.012 0.008 0.015
Creating an index 0.428 0.305 0.312
5000 SELECTs with an index 0.718 0.556 0.515
1000 UPDATEs without an index 2.38 2.138 2.068
BIONIC (memcpy, strlen) optimizationsTest AOSP
4.4.2Master + BIONIC optimizations
Master + BIONIC optimization + suspend disabled
25000 UPDATEs with an index 3.144 3.197 2.965
INSERTs from a SELECT 0.58 0.635 0.552
DELETE without an index 0.63 0.509 0.509
DELETE with an index 0.5 0.468 0.48
DROP TABLE 0.29 0.309 0.292
OVER ALL 21.747 19.503 18.697
• On Nexus 7 with AOSP master
• Enabling ART instead of Dalvik
Performance improved by 11 % (surprising ? despite SQLite is a native library, the Java API imposes an significant overhead)
HOWEVER, for some reason ART on Nexus 10 failed to give any better performance - maybe ART on Nexus 10 is still not optimized efficiently
Observation
Lib ART (Nexus 7)
Test Dalvik ART
1000 INSERTs 10.594 10.369
25000 INSERTs in a transaction 3.96 3.36
25000 INSERTs into an indexed table 3.93 3.336
100 SELECTs without an index 0.026 0.016
100 SELECTs on a string comparison 0.029 0.016
Creating an index 0.602 0.566
5000 SELECTs with an index 1.222 0.81
1000 UPDATEs without an index 4.357 3.648
Lib ART (Nexus 7)Test Dalvik ART
25000 UPDATEs with an index 6.079 5.038
INSERTs from a SELECT 0.935 0.96
DELETE without an index 0.886 0.915
DELETE with an index 0.776 0.809
DROP TABLE 0.462 0.433
OVER ALL 33.867 30.279
● Android uses 3.7.11 and SQLite.org is on 3.8.3.1 ○ Lot of bug fixes and optimizations - trying since yesterday …○ Will file a AOSP bug once we try it out to migrate SQLite in AOSP to latest.
● libsql depends on following libraries - need to find if we could optimize any of these routines.○ libdl.so○ liblog.so○ libicuuc.so○ libicui18n.so○ libutils.so○ libc.so○ libstdc++.so○ libm.so
● Android provides many APIs and FLAGS for APP developers like WRITE_AHEAD (basically buffers multiple I/O access into one and reads/writes in one chunk)○ Many books, tutorials available to app developers to refer to.
Next steps
• Accelerating SQL Database Operations on a GPU - Should we explore this ?• The research paper uses CUDA• Linaro GWG team has a card to implement the same with OpenCL• Android supports only renderscript (not compatible with OpenCL or CUDA)• Renderscript is mainly used in JAVA layers to leverage hardware
optimizations, but SQL code is already a “C” code, introducing renderscript calls is something we haven’t done before.
• Is this worth of an effort ? DEBATE …• May be wait till we get some numbers to compare with OpenCL.
Next steps - “Need a new slide for this one”
Please feel free to mail us on
linaro-android@lists.linaro.org
or IRC (freenode)
#linaro-android
Any questions ?
More about Linaro Connect: http://connect.linaro.orgMore about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/Linaro members: www.linaro.org/members