Automated Testing of Web Mapping Applications -...
Transcript of Automated Testing of Web Mapping Applications -...
![Page 1: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/1.jpg)
Automated Testing of Web Mapping
ApplicationsBrooks Robertson @brooksjbr
Jeremy Schneider @ymerejredienhcs
Tom Wayson @tomwayson
![Page 2: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/2.jpg)
Anatomy of a web application
• UI/client app
• API/Service Layer
• Backend/Business Logic
![Page 3: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/3.jpg)
Why test?
![Page 4: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/4.jpg)
The Test Pyramid
Unit
• Fast/Rapid
feedback
• Isolated
• Deterministic
• Run locally
• Live with code
Integration
UI
• End-to-end
• Full integration
• Longer running
• Connectivity
• Protocol
• CRUD
• Data Integrity
• Server response
time
• Page rendering
Cost
![Page 5: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/5.jpg)
Who writes what and how much?
Unit
Integration
UI
Performance
Feature Engineers
QA Engineers
![Page 6: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/6.jpg)
Where to start?
![Page 8: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/8.jpg)
Front-end vs Back-end Tests
Unit
Integration
UI
Front end Back end
![Page 9: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/9.jpg)
Unit tests
Test a ”unit” of code in isolation
![Page 10: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/10.jpg)
1.
_________________2. code is correct
3. catch regression
1. drive code structure
Primary value of unit
tests:
![Page 11: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/11.jpg)
average web
mapping
application
![Page 12: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/12.jpg)
24258698@N04/37656086
![Page 13: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/13.jpg)
4rulesforsuperunit tests
![Page 14: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/14.jpg)
writehigh value
tests
![Page 15: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/15.jpg)
test only your code
![Page 16: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/16.jpg)
test one thing at a time
![Page 17: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/17.jpg)
refactor
ruthlessly
![Page 18: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/18.jpg)
test one thing!
![Page 19: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/19.jpg)
“logic”
![Page 20: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/20.jpg)
“logic”
“action”
![Page 21: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/21.jpg)
1
2
3
4
“logic”
“action”
![Page 22: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/22.jpg)
1
2
3
4
“logic with calls”
“action functions”
1
2
3
4
![Page 23: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/23.jpg)
1
2
3
4
“action functions”
1
2
3
4
tests are high-value
when formulating an
algorithm
often (too) easy to test
write high value
tests!
![Page 24: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/24.jpg)
Don’t Test This Function
function add(a, b) {
return a + b;
}
![Page 25: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/25.jpg)
”Simple” Charts in Open Data
@tomwayson/arcgis-chart-utils
Demo: Test Driving
an Algorithm
![Page 26: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/26.jpg)
1
2
3
4
“high-value” tests
given a,b,c…
was fn 1 called with a,c?
did fn 3 call fn 4 with c?
1
2
3
4
focus on the “logic”
![Page 27: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/27.jpg)
1
2
3
4
testing logic
spies & fakes!
1
2
3
4
![Page 28: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/28.jpg)
1
2
3
4
testing logic
spies & fakes!
1
2
3
4
![Page 29: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/29.jpg)
24258698@N04/37656086
![Page 30: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/30.jpg)
Don’t Test the Map
![Page 31: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/31.jpg)
24258698@N04/37656086
![Page 32: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/32.jpg)
map access
![Page 33: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/33.jpg)
Separation of Concerns(hint: use a
framework)
![Page 34: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/34.jpg)
controllersUI components
![Page 35: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/35.jpg)
(or “map controller”)
a facade
map service
![Page 36: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/36.jpg)
mapService.on(“load”,
…)
map
mapService.newMap(…)
map service
mapService.destroyMap(
…)
mapService.showLayer(…
)
![Page 37: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/37.jpg)
spies & fakes!
map
spy:
how many times was the fn() called?
what parameters were passed?
fake (a.k.a stub):
replace implementation of a fn()
can also spy on the fake
![Page 38: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/38.jpg)
spies & fakes!
map
var stub = sinon.stub(mapService, “newMap”);
controllerUnderTest.showMap();
assert.ok(stub.calledOnce, 'newMap was called once');
![Page 39: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/39.jpg)
const stub = this.stub(mapService,
'newMap');
@tomwayson/ambitious-arcgis-app
Demo: Stubbing a
Map Service
![Page 40: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/40.jpg)
codetest frameworks
kk/7022179049
![Page 41: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/41.jpg)
![Page 42: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/42.jpg)
codeWhich is right for me?
kk/7022179049
Use whatever works with your app framework
![Page 43: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/43.jpg)
codeWhich is right for me?
kk/7022179049
Use whatever works for you
![Page 44: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/44.jpg)
codeWhich is right for me?
kk/7022179049
Use whatever
![Page 45: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/45.jpg)
Writing tests makes your code better.
![Page 46: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/46.jpg)
Writing tests makes you code better.
![Page 47: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/47.jpg)
Testable code is more robust.
![Page 48: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/48.jpg)
Testable code is more understandable.
![Page 49: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/49.jpg)
Testable code is more modular.
![Page 50: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/50.jpg)
Testable code is more reusable.
![Page 51: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/51.jpg)
Component
Tests
![Page 52: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/52.jpg)
“mini” e2e tests
![Page 53: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/53.jpg)
mimic user
interaction
![Page 54: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/54.jpg)
Only for a part of the page
![Page 55: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/55.jpg)
UIComponent
App Infrastructure
Config Util Service
API
partial isolationfake expensive
operations (async API
calls, etc)
![Page 56: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/56.jpg)
makes this
easy
![Page 57: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/57.jpg)
https://github.com/tomwayson/dev-summit-2017-front-end-testing
Thank you!
![Page 58: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/58.jpg)
![Page 59: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/59.jpg)
Browser-based
Web Mapping Application TestingJeremy Schneider
![Page 60: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/60.jpg)
Top of Testing Pyramid
![Page 61: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/61.jpg)
Browser & Visual Testing
• Selenium/WebDriver
• Screenshot Comparison
![Page 62: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/62.jpg)
Selenium TestingHuman-ish tool to
drive browser and
verify application behavior
• Hands/Fingers
• Eyes
• Brain
• Voice
![Page 63: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/63.jpg)
SeleniumHands:
• Navigate to sites – type into URL Bar
• Type stuff in fields
• Move mouse
• Click buttons
![Page 64: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/64.jpg)
SeleniumEyes:
Look at page (DOM)
through small
keyhole!
![Page 65: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/65.jpg)
SeleniumEyes:
Look at page (DOM)
through small
keyhole!
![Page 66: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/66.jpg)
SeleniumEyes:
Look at page (DOM)
through small
keyhole!
![Page 67: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/67.jpg)
Selenium
Brains/voice – Test Framework:
• RSpec, Cucumber, Junit, Test::Unit,
TestNG…
• API bindings in many languages
![Page 68: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/68.jpg)
Basic Selenium Flow
• Get driver
• Go to page
• Find element and interact with it
• Verify expected new page contents
![Page 69: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/69.jpg)
Get driver
![Page 70: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/70.jpg)
Go to page
![Page 71: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/71.jpg)
Locate elements
![Page 72: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/72.jpg)
Interact with elements
![Page 73: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/73.jpg)
Verify page changes
![Page 74: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/74.jpg)
Verify page changes
![Page 75: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/75.jpg)
Visual Regression Testing
• Create baseline screenshots
• Perform scenarios
• Capture and compare
screenshots to
known-good baseline
![Page 76: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/76.jpg)
Visual Testing
• Easy! Doesn’t know or care how page
is built
• Powerful! Lots of bang for your buck
– equivalent to many laboriously-
created DOM-based assertions
![Page 77: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/77.jpg)
Visual Testing
(example
baseline)
![Page 78: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/78.jpg)
Visual Testing
(example
pass)
![Page 79: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/79.jpg)
Visual Testing
(example
failure in
text)
![Page 80: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/80.jpg)
Visual Testing
(example
failure in
graphic)
![Page 81: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/81.jpg)
Visual Testing
• Screenshots of the same scenario
can differ slightly at the pixel level
• Captures everything in the browser window• 3/1/17 4:13:15 PM
• Image only – what about target of a link?
![Page 82: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/82.jpg)
Visual Testing
• Screenshots of the same scenario
can differ slightly at the pixel level
• Captures everything in the browser window• 3/16/2017 9:14:06 AM
• Image only – what about target of a link?
![Page 83: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/83.jpg)
Visual Testing
Drawback Mitigations:
• Tune “fuzz-factor” to allow for small pixel diffs
• Hide uninteresting elements before taking screenshot
• Pay for full-featured system (ex: Applitools Eyes)
• Use Selenium to check targets
![Page 84: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/84.jpg)
Browser and Visual Testing
Cons:
• Expensive to write
• Slow to run
• Require maintenance – “transparent” DOM
changes will break locators
![Page 85: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/85.jpg)
Browser and Visual Testing
Considerations:
• Better for regression tests
• Be selective in what to test
• Coordinate with dev for testability
• Use “page object model”
![Page 86: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/86.jpg)
Browser and Visual Testing
Pros:
• Enables testing of full stack, end-to-end
• Can test in multiple browsers
• Validates the app and deploy/env
• Exercises the web app the same way a user
does
![Page 87: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/87.jpg)
Demos
Let’s test some Web Mapping!
• Selenium in action
• With visual testing added
![Page 88: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/88.jpg)
ResourcesSelenium
• Official Selenium site: http://www.seleniumhq.org/
• Dave Haeffner: http://elementalselenium.com/
• Sauce Labs (ex: https://saucelabs.com/resources/articles/selenium-tips-css-selectors)
• https://gist.github.com/YmerejRedienhcs/28ec449f758056223076fa55d0954a72#file-
selenium-cheat-sheet-md-
• Page Object model: http://elementalselenium.com/tips/7-use-a-page-object
Visual Testing
• Spectre Open Source visual testing: https://github.com/wearefriday/spectre
• Applitools Eyes visual testing SAAS: https://applitools.com/
![Page 89: Automated Testing of Web Mapping Applications - Esriproceedings.esri.com/library/userconf/devsummit17/papers/dev_int... · Selenium Testing Human-ish tool to ... • Page Object model:](https://reader031.fdocuments.net/reader031/viewer/2022013014/5aa1b3277f8b9a436d8bf4ef/html5/thumbnails/89.jpg)