Android System Architecture And Pen-testing of Android applications
Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java...
Transcript of Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java...
![Page 2: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/2.jpg)
Introduction
• How to write perfect code?
• Familiar with Unit testing in Java?
Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 226/10/2018
![Page 3: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/3.jpg)
Overview
• Conventional Java testing
• What makes mobile/Android testing special?
• Further concepts: Continuous Integration
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 3
![Page 4: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/4.jpg)
Unit tests
• Fundamental tests in a software testing strategy• Test smallest possible units of code
• Isolation*• Use mocking framework to isolate an unit from its
dependencies
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 4
https://developer.android.com/training/testing/unit-testing/
![Page 5: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/5.jpg)
Typical Java Unit testing
Junit framework offers convenient ways to perform reusable setup, teardown and assertion operations
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 5
https://developer.android.com/training/testing/start/index.html#junit
import static org.junit.Assert.assertEquals;
import org.junit.Test;
public class CalculatorTest {
@Test
public void evaluatesExpression() {
Calculator calculator = new Calculator();
int sum = calculator.evaluate("1+2+3");
assertEquals(6, sum);
}
}
![Page 6: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/6.jpg)
Android Unit testing
There are two types:
1. Local Unit tests• Run on local JVM
• No access to Android framework API• You could mock some of the API
2. Instrumented tests• Run on an Android device (physical/emulator)
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 6
https://developer.android.com/training/testing/start/index.html#test-types
![Page 7: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/7.jpg)
• JVM execution – fast!
• Can’t use Android framework dependencies• Can use mock objects
instead
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 7
https://developer.android.com/training/testing/start/index.html#test-types
• Access to instrumentation info (e.g. app Context)
• Use if you can’t easily mock the framework objects
![Page 8: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/8.jpg)
Local Unit Tests
Set-up (done by default in Android Studio):
- Test files in module-name/src/test/java
- JUnit 4 dependency in your app’s build.gradle
Let’s write our first unit test!
Demo – Simple Unit test using Android Studio
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 8
https://developer.android.com/training/testing/unit-testing/local-unit-tests.html
dependencies {
testImplementation 'junit:junit:4.12'
}
![Page 9: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/9.jpg)
Test Subject: Shopping List App• ShoppingItem
• Name, price, quantity
• Price determined by no. of letters!
• ShoppingBasket:• Adding items
• Item names are unique, re-adding same item updates items quantity
• Calculating total price of items in basket
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 9
![Page 10: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/10.jpg)
• Course Exercise:• Simple Unit tests (Follow lecturer example)
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 10
![Page 11: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/11.jpg)
Unit test isolation discussion
• Unit tests are executed using a modified android.jar library, which is empty!
• Helps encourage the “test-in-isolation” mindset
So any time you find your test not working because of this, you can either:
1) Rethink your tests- are you really testing independent units?
2) Isolate by using mock objects
3) Use something like Roboelectric*
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 11
* http://robolectric.org/
![Page 12: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/12.jpg)
Local Unit Testing: Mocking with MockitoAdd dependency:
Annotate your test class with:
Creating a mock Android object:- Add the @Mock annotation to the field declaration- Define the behaviour:
- when(X).thenReturn(Y) methods
- Matchers:- anyString(), eq(), …
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 12
https://developer.android.com/training/testing/unit-testing/local-unit-tests.html#mocking-dependencies
testImplementation 'org.mockito:mockito-core:1.10.19'
@RunWith(MockitoJUnitRunner.class)
![Page 13: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/13.jpg)
Code Coverage
• See if you can get 100% coverage for ShoppingBasket class
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 13
COURSE EXERCISE:
GENERATE A HTML CODE COVERAGE REPORT
(INCLUDE IN YOUR PROJECT SUBMISSION .ZIP)
![Page 14: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/14.jpg)
Moving on to Instrumented tests…
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 14
https://developer.android.com/training/testing/start/index.html#test-types
![Page 15: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/15.jpg)
Android Testing Support Library (ATSL)Framework that helps building and running app test code
Includes:
• AndroidJUnitRunner
• Espresso
• UI Automator
• ATSL is included with the Android Support Repository which you can obtain using the SDK manager
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 15
https://google.github.io/android-testing-support-library/
![Page 16: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/16.jpg)
Instrumented Unit Tests
• Make sure you have Android Support Repository installed in SDK Manager!
• Dependencies:
• Set AndroidJUnitRunner as the default test instrumentation runner:
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 16
https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests#setup
androidTestImplementation 'com.android.support:support-annotations:28.0.0'
androidTestImplementation 'com.android.support.test:runner:1.0.2'
androidTestImplementation 'com.android.support.test:rules:1.0.2'
defaultConfig {
…
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner
}
![Page 17: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/17.jpg)
Demo
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 17
![Page 18: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/18.jpg)
Test suites, Firebase Test Lab
• Instrumented tests can be grouped together to form test suites
Additionally, Android Studio provides means for deploying tests to Google’s Firebase Test Lab
Firebase Test Lab runs your testson physical devices with various
configurations
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 18
![Page 19: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/19.jpg)
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 19
![Page 20: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/20.jpg)
Single App UI testing with Espresso Framework• Programmatically simulates
user interactions• Click, swipe, text input, …
• Run on Android 2.3.3 (API v10) and up
• Espresso manages synchronization of UI actions• Can be a huge challenge in
automated UI testing
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 20
![Page 21: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/21.jpg)
Setting up Espresso
• Dependencies:
• Android Developer docs advise turning off animations on test device
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 21
androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
// Optional -- Hamcrest library
androidTestImplementation 'org.hamcrest:hamcrest-library:1.3'
![Page 22: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/22.jpg)
Espresso Test Basics
1. Select a View to work with:a) onView (targetView ) (for Activities)
b) onData() (for AdapterViews)
2. Simulate specific interaction:a) perform( action)
3. Repeat previous steps to simulate user flow
4. Use ViewAssertion methods to verify if actual behaviour matches expected
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 22
https://developer.android.com/training/testing/ui-testing/espresso-testing.html
Espresso Cheat Sheet
HamCrest cheat sheet
![Page 23: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/23.jpg)
Espresso example with ActivityTestRule• Demo
Important: use the following to reduce boilerplate code:
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 23
@Rule
public ActivityTestRule<MyActivity> mActivityRule =
new ActivityTestRule<>( MyActivity.class );
![Page 24: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/24.jpg)
Further Espresso concepts
E.g. Espresso Intents:
- Validate and stub (mock) intents sent out by the app
- Intercept outgoing intents, stub the result, send back the result to the component under test
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 24
![Page 25: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/25.jpg)
UI testing with multiple apps
• It’s not uncommon for user flow to span multiple apps• Remember HA1? The contacts app opened the e-mail
app, and then returned to the original application
• This type of flow can be handled using the UI Automator APIs
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 25
https://developer.android.com/training/testing/ui-testing/uiautomator-testing.html
![Page 26: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/26.jpg)
UI Automator basic approach
1. Get the UiDevice object, this reflects the device being tested
2. Get the UiObject object, by calling findObject()
3. Simulate user interaction on UiObject, similar to Espresso
4. Check that the behaviour matches expected one.
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 26
![Page 27: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/27.jpg)
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 27
![Page 28: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/28.jpg)
Continuous Integration
Now that you have your app + tests, you can start thinking about automating other aspects of your software project
Continuous Integration
- Practice of merging code changes across developers, as frequently as possible
- Validate changes with automated builds & tests- Includes static code analysis, testing and configuration
sets
- Can include Continuous Deployment:- Making the newest version available somewhere
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 28
![Page 29: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/29.jpg)
Continuous Integration
1. You update your application code2. Push to VCS (such as GitHub)3. CI system sees the new changes, pulls the latest code,
builds and runs tests- Perhaps on different target API levels
4. Based on build & test result:- Failure: Notify developers via E-mail, Slack, …- Success: Deploy to Play Store, to Test Audience, Staging
Environment, etc
- CircleCi, - NeverCode - company with roots in Tartu, support for
iOS and Android
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 29
![Page 30: Testing on Android - Arvutiteaduse instituut · 2018. 10. 26. · Overview •Conventional Java testing •What makes mobile/Android testing special? •Further concepts: Continuous](https://reader036.fdocuments.net/reader036/viewer/2022071002/5fbecf03eff92336d406da8f/html5/thumbnails/30.jpg)
Summary
• Local Tests vs Instrumented Tests
• Unit Testing
• UI testing• Espresso• UIAutomator
• Testing philosophy• Isolation of components• Mocking objects
• Additionally:• Firebase Test Lab• Continuous Integration
26/10/2018 Mobile & Cloud Lab. Institute of Computer Science, University Of Tartu 30