Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with...
Transcript of Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with...
![Page 2: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/2.jpg)
Mobile Quality Challenges
Device Fragmentation
App Complexity
Fast Release Cycles
Short Sessions
High Mobile User Expectations
![Page 3: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/3.jpg)
Results! The Differences in Time to Value
19K unique
Android devices available on the market
20 iPhone and
iOS configuration combinations on the market
In the United States, a business must test on
134 different devices
80% of devices in use
to cover
![Page 4: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/4.jpg)
The Manual Approach
Limited device reach, based on user availability
Slower and less accurate
Delayed feedback collection
Inconsistent
![Page 5: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/5.jpg)
The Device Remoting Approach
Prone to human error
Limited test complexity
No simultaneous testing slows
time to identify issues
![Page 6: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/6.jpg)
The Simulator Approach
Limited real world applicability
Virtual, not real devices
Limited accuracy
![Page 7: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/7.jpg)
Xamarin Test Cloud Approach
AUTOMATED WITH MANAGED DEVICES
Consistent, scalable and accurate tests
Simultaneous testing across device platforms, operating systems and form factors
Real usage scenarios, including complex gestures
Secure with device wipe after each test run
![Page 8: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/8.jpg)
Xamarin Test Cloud
![Page 9: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/9.jpg)
Xamarin Test Cloud
2,000+ devicesAutomate your app testing and run them on over
2,000 different real devices. Test everything users do,
as well as any performance problems with step-by-
step memory and performance tracking.
Real devices, real quality• Ensure higher quality by testing on real devices
• Automate app testing on over 2,000 real devices
• Use C#, Ruby, or Cucumber to build automated
tests
• Integrates with any continuous integration system
![Page 10: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/10.jpg)
Tap Scroll Swipe Pinch Multi finger
Text Entry Rotation GPS
Examples of User Interactions
![Page 11: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/11.jpg)
Visually compare test results against
hundreds of devices at a time
• Zoom in and full screen resolution of all
screens
• Video playback coming in 2015
Review each step of a test
case and compare visual
results on all devices
![Page 12: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/12.jpg)
![Page 13: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/13.jpg)
Access precise device specifications,
full-size screenshots, and the native
device log
Understand CPU and memory usage
during every step
![Page 14: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/14.jpg)
![Page 15: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/15.jpg)
![Page 16: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/16.jpg)
Object-level user
interface testing
Tests adapt as user
interface changes
Tests Adapt to UI Changes
![Page 17: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/17.jpg)
March 2015
![Page 18: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/18.jpg)
How does it work?
![Page 19: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/19.jpg)
Xamarin Test Cloud
1. Create a Xamarin Test Cloud account
2. Prepare tests
3. Deploy to XTC
4. Integrate with your release pipeline
![Page 20: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/20.jpg)
1. Create Xamarin Test Cloud Account
free trial
– 30 days
– max. 3 concurrent devices
– max. 3 testing hours per day
![Page 21: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/21.jpg)
DEMO
Registering XTC account
![Page 22: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/22.jpg)
2. Prepare tests
UI Tests in a separate project
using NUnit
can be hand-written
can be recorded using Test Recorder
iOS tests cannot be run on Windows
![Page 23: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/23.jpg)
REPL
Read-Eval-Print-Loop
useful when defining tests
app.Repl()
– stops running test and opens a console window
tree
– lists the current view‘s structure
copy
– copies all commands to clipboard
![Page 24: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/24.jpg)
Commands (Methods)
app.Flash()
– „blinks“ an element
– app.Flash(c => c.Class(„EditText“))
app.EnterText(c => c.Marked(„creditCardNumberText“), „12345“)
– app.ClearText(...)
app.Tap(c => c.Marked(„validateButton“))
app.WaitForElement(c => c.Marked(„label“).Text(„ABC“))
– 1 minute in test cloud
– 15 seconds locally
– can be changed with a parameter
![Page 25: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/25.jpg)
![Page 26: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/26.jpg)
![Page 27: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/27.jpg)
![Page 28: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/28.jpg)
![Page 29: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/29.jpg)
DEMO
Working with Xamarin REPL
![Page 30: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/30.jpg)
Test Recorder
available as a preview extension for Visual Studio
https://developer.xamarin.com/guides/testcloud/testrecorder
/visual-studio/
![Page 31: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/31.jpg)
![Page 32: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/32.jpg)
![Page 33: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/33.jpg)
![Page 34: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/34.jpg)
![Page 35: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/35.jpg)
3. Deploy to Xamarin Test Cloud
via web portal
from Visual Studio / Xamarin
Studio
from command line
![Page 36: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/36.jpg)
DEMO
Deploy to XTC from VS
![Page 37: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/37.jpg)
4. Integrate with your release pipeline
Visual Studio Team Services
Xamarin Test Cloud as one of the build steps
![Page 38: Mobile App Testing with Xamarin Test Cloud - sec.ch9.ms · PDF fileMobile App Testing with Xamarin Test Cloud Martin Šimeček @microsoft.com @deeedx](https://reader031.fdocuments.net/reader031/viewer/2022021501/5a7fef0c7f8b9a682c8be47b/html5/thumbnails/38.jpg)
DEMO
VSTS & Xamarin Test Cloud