Testdroid: Different android test automation frameworks - what works you the best?
Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud
-
Upload
applitools -
Category
Mobile
-
view
6.189 -
download
0
Transcript of Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud
![Page 1: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/1.jpg)
![Page 2: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/2.jpg)
Public Device Cloud
on-demand devices
(multitenant)
Hosted by Bitbar
Mobile app testing on literally any real Android and iOS devices
Private / Reserved Device Cloud
Hosted by Bitbar in the US and/or
Europe
Devices chosen by and reserved only for the Customer
On-Premise Device Cloud
Automated mobile app testing devices
hosted by the customer, usually 30-500 devices
TESTDROID – 3 DEPLOYMENT OPTIONS
Testdroid Cloud Testdroid Enterprise Testdroid PrivateCloud
![Page 3: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/3.jpg)
![Page 4: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/4.jpg)
The Value of ‘Cloud Testing’• Devices / Hardware
- No need to handle shipping, logistics, handling of new devices and other new HW used in the infrastructure monthly
• Infrastructure- Ready-to-Go infrastructure means no need for investments,
significantly speeds things up & saves time and money
• Support- SW devs, SW devs in test (SDET), QA engineers – all need different
type of support when using the infrastructure
• Operations- Updating infrastructure, devices, networks, additional SW
• Tools Integration - Support for multiple test tools gives the users/developers
freedom of choice. No vendor, tech, tools lock-ins!
![Page 5: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/5.jpg)
Testing ‘in the Cloud’
1
2
3
![Page 6: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/6.jpg)
Android Fragmentation 2015
by Open Signal report August 2015
Testdroid Cloud
>95%
![Page 7: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/7.jpg)
Android Fragmentation 2015
by Open Signal report August 2015
Testdroid Cloud
>95%
![Page 8: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/8.jpg)
Android & iOS OS Flakiness
iOS 8.0iOS 8.0.2iOS 8.1iOS 8.1.1iOS 8.2iOS 8 ALLKitKat 4.4KitKat 4.4.2KitKat 4.4.3KitKat 4.4.4KitKat ALLLollipop 5.0Lollipop 5.0.1Lollipop 5.0.2Lollipop ALLAndroid M
0 5 10 15 20 25 30 35Failed test runs. Percentage (%).
![Page 9: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/9.jpg)
Why Apps Fail?
![Page 10: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/10.jpg)
Why Real Devices Are Must-to-Have
• Emulators cannot help you to test...
• User Experience • Usability• Hardware• Software• Infrastructure
0 % = the percentage of your app users that use emulator to run your app!
![Page 11: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/11.jpg)
Top 5 Pain PointsFragmentationDiversity of mobile platforms creates unique challenges for mobile
app, game and web devs.
Low Quality Apps Lose MoneyUsers uninstall and abandon apps/services if the apps do not work.
Manual Testing on Real DevicesExpensive, slow and error-prone process
Device AcquisitionAcquiring devices for devand testing can be expensive and
practically impossible
InefficienciesDistributed Teams, duplications of HW resources, app diversity,
fragmented tool support.
![Page 12: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/12.jpg)
Manual vs. AutomationManual Automation
![Page 13: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/13.jpg)
Manual vs. Automation
Smaller coverage, More money burnt & time wasted, Error-prone
Manual AutomationLarge coverage,
quickly completed,
Less money & time wasted, Exact results.
![Page 14: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/14.jpg)
Test Automation FrameworksRobotium uiautomato
rEspresso Appium Calabash
Android Yes Yes Yes Yes Yes
iOS No No No Yes Yes
Mobile web Yes (Android)
Limited to x.y clicks
No Yes (Android & iOS)
Yes (Android)
Scripting Language
Java Java Java Almost any Ruby
Test creation tools
Testdroid Recorder
UI Automator viewer
Hierarchy Viewer
Appium.app CLI
Supported API levels
All 16 => 8, 10, 15- All All
Community Contributors Google Google Active Pretty quiet
![Page 15: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/15.jpg)
APPIUM / SELENIUM IN THE CLOUD WITH REAL DEVICES & REAL BROWSERS
![Page 16: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/16.jpg)
![Page 17: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/17.jpg)
Client Side Appium at Testdroid Cloud
Test Script
Test Case
Desired Capabilities
{ “device”: “Android”, “app”: “/Users/user/ApiDemos.apk” “app-package”: “com.example.android.apis” “app-activity”: “.ApiDemos”}
![Page 18: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/18.jpg)
Test Script
Test Case
Desired Capabilities
WebDriverhttp://localhost:4723/wd/hub
Appium Server
4723
Device
Localhost(DesiredCaps)
![Page 19: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/19.jpg)
http://localhost:4723/wd/hub
*Testdroid Caps
http://appium.testdroid.com/wd/hub
(DesiredCaps)
Test Script
Test Case
Desired Capabilities
WebDriver
From Localhost to Testdroid
![Page 20: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/20.jpg)
Client Side ExecutionGo to
cloud.testdroid.com
![Page 21: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/21.jpg)
Client Side ExecutionGet a
Device NameGo to
cloud.testdroid.com
![Page 22: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/22.jpg)
Client Side ExecutionAdd Testdroid Desired
Caps to test script
{ “testdroid_username”: “[email protected]”, “testdroid_password”: “p4s$w0rd”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “http://domain.com/app_v1.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample”}
Get a Device Name
Go tocloud.testdroid.com
![Page 23: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/23.jpg)
Client Side Execution
driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps);
Point the Webdriver to http://appium.testdroid.com/wd/hub
Add Testdroid Desired Caps to test script
Get a Device Name
Go tocloud.testdroid.com
![Page 24: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/24.jpg)
Client Side Execution
Run the Test ScriptGet Results from Testdroid Cloud
Point the Webdriver to http://appium.testdroid.com/wd/hub
Add Testdroid Desired Caps to test script
Get a Device Name
Go tocloud.testdroid.com
ScreenshotsLogs
Detailed results
![Page 25: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/25.jpg)
Client Side Execution
Pull the Results from the Result URL
driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps);
Run the Test ScriptGet Results from Testdroid Cloud
Point the Webdriver to http://appium.testdroid.com/wd/hub
Add Testdroid Desired Caps to test script
Get a Device Name
Go tocloud.testdroid.com
![Page 26: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/26.jpg)
Multiple Devices – Client Side
python testscript.py –device <devicename>1
python testscript.py –device <devicename>2
python testscript.py –device <devicename>n
Test Script
Test Cases
Instigator Script
deviceArray=[ “iPad 4 6.0.1 A1458”, “iPad mini 7.0.4 A1432”, . . . “iPhone 4S 6.1.3 A1387”,]
![Page 27: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/27.jpg)
TESTDROID CLOUD AT GLANCE
![Page 28: Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud](https://reader036.fdocuments.net/reader036/viewer/2022081604/58892a301a28ab77528b6cc3/html5/thumbnails/28.jpg)
THANK YOU!More information about frameworks,
devices and testing at www.testdroid.com