Mobile Testing - files.meetup.com Testing.pdf · Non+Issues+ UITes6ng+ Fragmentaon+...
Transcript of Mobile Testing - files.meetup.com Testing.pdf · Non+Issues+ UITes6ng+ Fragmentaon+...
Mobile Testing Open Source Solu,ons
Top Q – Who are we? General• Established in 2005, the leading test-‐automa6on solu6ons company in Israel• More than 100 customers in major ver6cal markets, including Networking
equipment manufacturers and Java-‐based vendors Methodology• Deploys a tried-‐and-‐true project development methodology, proven to build
successful and enduring test-‐automa6on projects Tools• Exper6se in all leading test-‐automa6on plaHorms and tools, including commercial
to open source Services• Complete life-‐cycle of a test-‐automa6on solu6on: from ini6al requirement phase,
analysis and planning, to implementa6on, maintenance and audit
Customers
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies We’re not going to talk about
Tes6ng framework Programming language
BDD, TDD, ATDD,KDT etc.
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies
+
Image Based Approach
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Tes,ng
UI
Integra6on
Unit
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Tes,ng
• Func6onal black box tes6ng – no need of internal implementa6on details of the app, only its expected output when a user performs a specific ac6on or enters a specific input.
• test the behavior of your applica6on’s user interface (UI) when it is running on a device
• Usually 6me-‐consuming, tedious, and error-‐prone when done manually
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Fragmenta,on Issues
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Fragmenta,on Issues
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Fragmenta,on Issues
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Tes,ng – Commercial
ConsProsPlaIormAPIFrameworkWorks with all devices Works with IOS on PC
Android/IOSJava / C# / Perl/Python
SeeTest
Selenium API Works with IOS on PC A lot of tes6ng devices
Android/IOSRecording Selenium API
Perfecto Mobile
DDT supports web and
desktop app
Android/ios/ windows mobile
.netRanorex
Good KDT Excellent repor6ng
Supports DB,Web,Desktop apps
etc.
AndroidC#/Java Script
TestComplete
Works with other Mobile Automa6on
Framworks
AndroidVB.netQTP
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Tes,ng – Open Source
ConsProsPlaIormAPIFrameworkSinging
Instrumenta6ons App domain only
Works with all Android versions
Hybrid Apps
AndroidJava/C#/ Python etc.
Robo6um
Android API 16 and above
Selenium API Works with IOS Hybrid Apps
Android/IOSJava/C#/ python etc. Selenium API
Appium
Android API 16 and above
No WebView control
No need of special permission
Controls the whole UI Hybrid Apps
AndroidJava/C#/ python etc.
UI Uiautomator
not enough documenta6on
Easy to useIOSJava/ cucumber
Frank
BDDBDDIOS/AndroidCucumber+ruby/java
Calabash (Xamarin)
No mobile webGoogleAndroidJavaEspresso
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Tes,ng – Commercial
ConsProsPlaIormAPIFrameworkSelenium API
Works with IOS on PC A lot of tes6ng devices
Android/IOSRecording Selenium API
Perfecto Mobile
Based on Appium Runs in the cloud
Android/IOSJava/Ruby/ PHP/JS/ Python
SauceLabs
Excellent UI tes6ng toolAndroidJava/Ruby/ PHP/JS/ Python
Applitools
Real devices on cloudAndroid/IOSJava/ Cucamber
TestDroid
Based on CalabashAndroid/IOSC#Xamarin Test Cloud
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies
• A service provided by the different opera6ng systems
• Designed for unit tes6ng • Interfaces with the applica6on UI objects
Instrumenta,on Based Approach
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Instrumenta,on Based Approach
• The tes6ng applica6on must have instrumenta6on permissions on the applica6on under test package
• In Android we need to use the same signature in the DUT and the test applica6on
• Provided differently in each OS • Enable to test only the applica6on context
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Instrumenta,on Based Approach
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies
• Open source library extending JUnit • Black-‐box test cases for Android apps • Supports na6ve, hybrid and mobile web tes6ng
• Semi-‐ac6ve open source community with monthly releases
Robo,um
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Robo,um – example
public class NotePadTest extends ActivityInstrumentationTestCase2<NotesList>{ private Solo solo;
@Override public void setUp() throws Exception {
//setUp() is run before a test case is started. //This is where the solo object is created.
solo = new Solo(getInstrumentation(), getActivity()); {
public void testAddNote() throws Exception { solo.clickOnMenuItem("Add note"); //Assert that NoteEditor activity is opened solo.assertCurrentActivity("Expected NoteEditor activity", "NoteEditor"); solo.enterText(0, "Note 1"); solo.goBack(); solo.clickOnMenuItem("Add note"); solo.enterText(0, "Note 2"); solo.goBackToActivity("NotesList"); //Takes a screenshot and saves it in "/sdcard/Robotium-‐Screenshots/". solo.takeScreenshot(); boolean expected = true; boolean actual = solo.searchText("Note 1") && solo.searchText("Note 2");
assertEquals("Note 1 and/or Note 2 are not found", expected, actual); }
}
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies
• Robo6um remote control allows execu6on of Robo6um tests from the deskop
• No need to deploy tests with the applica6on • Java and Python binders
Robo,um Remote Control
Robo6um RC
Robo6um Client
Instrumenta6on
Json over HTTP
Test Code
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Calabash
• Behavior driven test framework for na6ve Android, na6ve iOS and mobile web (Android)
• Calabash installs an HTTP server as an instrumenta6on package that listens commands from Calabash server
• New controllers can be implemented in Ruby or Java • Tests are executed on server side • Each test scenario is described in Cucumber • Ruby Client library converts Cucumber commands to
either Robo6um or Frank method calls
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Calabash
iOS Android
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Calabash -‐ Gherkin
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UiAutomator
• A Java library, created by Google, containing APIs to create automated customized func6onal UI tests.
• An execu6on engine to run the tests. • UiAutomator allows to write code based UI Tests with no screen resolu6on dependency
• UiAutomator will work on any Android device from API 16 and above
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UiAutomatorViewer
• A GUI tool to scan and analyze the UI components of an Android applica6on.
• Inspect the UI of an applica6on in order to find the layout hierarchy and view the proper6es of the individual UI components of an applica6on
• Aher the installa6on of Android SDK, the tool exists in the /tools/ folder and you can start it by typing: uiautomatorviewer from the command line.
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UiAutomatorViewer -‐ Example
• A screenshot of the Calculator” applica6on opens. • For example click on “7” bulon and look at the right bolom panel the
“resource-‐id” value
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Wri,ng a Test – code example
public class CalculatorTest extends UiAutomatorTestCase {
public void tes6ngCalculator() throws UiObjectNotFoundExcep6on { getUiDevice().pressHome(); UiObject Applica6ons = new UiObject(new UiSelector().descrip6on("Apps")); Applica6ons.clickAndWaitForNewWindow(); UiObject apps = new UiObject(new UiSelector().text("Calc")); apps.click(); UiScrollable ListOfapplica6ons = new UiScrollable(new UiSelector().scrollable(true)); UiObject Calculator = ListOfapplica6ons.getChildByText(new UiSelector().className(android.widget.TextView.class.getName()), "Calculator"); Calculator.clickAndWaitForNewWindow(); UiObject seven = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/digit7")); seven.click(); UiObject plus = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/plus")); plus.click(); UiObject one = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/digit1")); one.click(); UiObject result = new UiObject(new UiSelector().resourceId("com.android.calculator2:id/equal")); result.click(); getUiDevice().pressBack();
{{
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Running a Test
• Type <sdk_path>/tools/android create <name> -‐n -‐t 5 -‐<path> The <name> is the name of the project that contains your uiautomator test source files, the <path> is the path to the corresponding project directory.
• In order to find the correct value for the parameter -‐t, type android list targets.
• Go to your project directory and type ant build • Push the JAR file to your Android device • Run the tests by typing
adb shell uiautomator runtest <Path/Jar_name> -‐c <package.test_class> • for example:
adb shell uiautomator runtest LaunchSe>ngs.jar -‐c com.uia.tests.LaunchSe>ngs
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UiAutomator Server – Client Solu,on
• As can see from the pervious slides the UiAutomator code is compiled and pushed and executed on the DUT
• To allow more flexible test code (I.e test paramteriza6on) and to enjoy the test framework service Top Q’s solu6on is based on client-‐server architecture over HTTP implanted with JSON-‐RPC
• The server contains two compiled jars and it is running on the DUT with the regular UiAutomator execu6on command
• The server implements an interface that refelects UiAutomator API • The client is a lean Java class that uses the same interface over JSON-‐RPC
client • UiSelector, UiDevice and UiObject are implemented in both sides
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Appium
• Allows to locate UI elements and perform various opera6ons of them
• Supports Mac and Windows • Supports na6ve, hybrid and web applica6ons • Supports iOS and Android (and FirfoxOS) • Supports real devices and simulators • Uses Selenium Webdriver (W3C standard) as a scrip6ng framework
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Appium Architecture
• Appium is an HTTP server that creates and handles WebDriver sessions
• It starts an Appium server on the device that is listening commands from the main Appium process
• It receives json requests from client libraries over HTTP
• On Android Appium executes these commands as either uiautomator or Selendroid commands depending on the API level of the device
• Tests are driven from a Selenium script on Appium server
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Android Architecture
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies iOS Architecture
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Appium inspector
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies
DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability("platformName", "Android"); capabilities.setCapability("app","src/test/resources/com.twitter.android_5.5.0.apk"); capabilities.setCapability("app-‐package", "com.twitter.android"); capabilities.setCapability("app-‐activity", ".LoginActivity"); AppiumDriver driver = new AppiumDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
Appium API
//Find the element in page using a selector locating strategy WebElement element = driver.findElement(MobileBy.className("android.widget.EditText")); //Perform operation on the found element element.sendKeys("jsystemdemo");
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Addi,onal Test Tools
• Fiddler Core – Based on fiddler core DLLs – Java API
• Logcat parsing -‐ Wait for specific message -‐ Search for excep6ons
• GennyMo6on – Android Emulator • SDK tes6ng
– Tester appliac6on – Client – Server architecture with JSON-‐RPC
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies Page Object Design Pa\ern
• Within your applica6on UI there are areas that your tests
interact with.
A Page Object simply models these as objects within the
test code.
• The Page Object’s methods represent the services offered
by a par6cular page.
• The page object encapsulate the UI elements. if the UI
changes, the fix need only be applied in one place.
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Repository
• In code or in test framework GUI use repository keys instead of real locators
• Saves 6me between versions • In some cases can be used with iOS/Android versions
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Repository -‐ GUI
Non Issues UI Tes6ng
Fragmenta6on Tools Quick Review instrumenta6on
Calabash UiAutomator
Appium Case Studies UI Repository -‐ Code
public class LoginTests { private String username; private String password; @Test @TestProperties(name="Login successfully with username ${username} and password ${password}", paramsInclude = { "username","password" }) public void loginSuccess() throws Exception{ appiumSo.getActions().waitForElement(appiumSo.objectRepository.getObject("login.username"),30); appiumSo.getActions().sendText(appiumSo.objectRepository.getObject("login.username"), username); appiumSo.getActions().sendText(appiumSo.objectRepository.getObject("login.password"), password); appiumSo.getActions().click(appiumSo.objectRepository.getObject("login.btn")); appiumSo.getActions().waitForElement(appiumSo.objectRepository.getObject("fileManager.switchBtn"),30); }
Thank you For listening
Nir Armon Top Q [email protected] Find me on LinkedIn