Testing Web Mapping Applications and Web Services using Python

Jáchym ČepickýOSGeo.org

● Member● Board member (2012-2015)● Secretary

Open source software developer, contributor, user

● GRASS GIS● PyWPS● OpenLayers, …● http://opengeolabs.cz

Current: Head of Internal development at Cleerio inc.

Cleerio inc.

What is software testing

“Software testing is an investigation conducted to provide stakeholders withinformation about the quality of the product or service under test”

-- Wikipedia, https://en.wikipedia.org/wiki/Software_testing

What do testers doSoftware testers do not make software.

But they make it better

-- Sergejus Bartos, http://pt.slideshare.net/SergejusBartos/

What do testers do

What they really do

What they really do

What they really do

Tester

Developer

Software testing impact

https://en.wikipedia.org/wiki/Software_testing

Cost to fix a defectTime detected

Requirements Architecture Construction System test Post-release

Time introduced

Requirements 1× 3× 5–10× 10× 10–100×

Architecture -- 1× 10× 15× 25–100×

Construction -- -- 1× 10× 10–25×

Testers

Developers

History of testingUntil 1956 – Debugging oriented

1957–1978 – Demonstration oriented

1979–1982 – Destruction oriented

1983–1987 – Evaluation oriented

1988–2000 – Prevention oriented

Testing types

Static × Dynamic

Testing types

White box × Gray box × Black box

Testing levels

Unit testing

Integration testing

System testing

...

Unit testing

Integration testing

System testing

Unit testing vs. System testing

How we test web map application at Cleerio.com

Unit tests - front-end and back-end

Integration and system tests - manual work

Deploying new version without system testing

…

Learn Python for testing!● Easy to learn● Test frameworks● Geo-positive

Testing software stack

Testing software stack● Jenkins https://jenkins.io/ ● Python http://python.org● PhandomJS http://phantomjs.org/ ● Selenium http://www.seleniumhq.org/ | http://selenium-python.readthedocs.io/

Start to lear Python, it’s easy!

Example

from selenium import webdriverfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support import expected_conditions as EC

# …

self.driver = webdriver.Chrome('chromedriver/bin/chromedriver')self.wait = WebDriverWait(self.driver, 90)

self.driver.set_window_size(1366, 768)self.driver.get(

“http://maps.foo.bar/test”)

self.wait.until( EC.presence_of_element_located( (By.CSS_SELECTOR, '.ol-viewport') ))

self.assertTrue(driver.find_element_by_class_name('gs-app')

)

self.assertTrue(driver.find_element_by_css_selector('div.gs-disclaimer')

)

self.wait.until(EC.presence_of_element_located( (By.CSS_SELECTOR, div.gs-disclaimer'))) div = self.driver.find_element_by_css_selector('div.gs-disclaimer')

agree_button = div.find_element_by_css_selector('button[data-gs-handler=agree]')

agree_button.click()

with self.assertRaises(NoSuchElementException, msg='Discalimer dismissed'):

self.driver.find_element_by_css_selector('div.gs-disclaimer')

menu_button = self.driver.find_element_by_css_selector('div.test-menu-button button[data-gs-handler=menu]')

menu_button.click()

login_item = case.driver.find_element_by_css_selector('ul.gs-main-menu li button ...') login_button = login_item.find_element_by_xpath('..') login_button.click()

loginform = case.driver.find_element_by_class_name('loginform') self.assertTrue(

loginform, "Module login main div visible"

)

user_input = loginform.find_element_by_css_selector('input[name=login_username]')passwd_input = loginform.find_element_by_css_selector('input[name=login_password]')submit_button = loginform.find_element_by_css_selector('button[data-gs-handler=submit]')

user_input.send_keys(login_username)passwd_input.send_keys(login_password)submit_button.click()

self.assertTrue(self.driver.find_elements_by_class_name('test-loader'),

"Loading indicator visible")

self.wait.until(EC.invisibility_of_element_located((By.CLASS_NAME, 'test-loader')))

with case.assertRaises(NoSuchElementException, msg="Login module div not available"):self.driver.find_element_by_css_selector('div.test-logindiv')

mapcanvas = self.driver.find_element_by_css_selector('canvas.ol-unselectable')

action = webdriver.common.action_chains.ActionChains(case.driver)action.move_to_element_with_offset(mapcanvas, 700, 400)action.click()action.move_to_element_with_offset(mapcanvas, 700, 450)action.click()

self.driver.save_screenshot('screenie.png')

mapcanvas = self.driver.find_element_by_css_selector('canvas.ol-unselectable')

action = webdriver.common.action_chains.ActionChains(case.driver)action.move_to_element_with_offset(mapcanvas, 700, 400)action.click()action.move_to_element_with_offset(mapcanvas, 700, 450)action.click()

self.driver.save_screenshot('screenie.png')

Result: Happy product owner

Jonáš: Product owner at Cleerio

IssuesTest stability - usually back-end problem

Developers complaining about tests falling down

Not so easy reporting of broken tests

======================================================================ERROR: test_datagrid (tests.map_application.servers.TestBranchComplex)Test gs.module.Datagrid----------------------------------------------------------------------Traceback (most recent call last): File "tests/map_application/__init__.py", line 153, in test_datagrid datagrid.test(self, True) File "tests/map_application/modules/datagrid.py", line 20, in test ls_show_module(case, 'datagrid', layer_id) File "tests/map_application/modules/layerswitcher.py", line 169, in show_module module_button.click() File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 75, in click self._execute(Command.CLICK_ELEMENT) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 469, in _execute return self._parent.execute(command, params) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 201, in execute self.error_handler.check_response(response) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response raise exception_class(message, screen, stacktrace)StaleElementReferenceException: Message: {"errorMessage":"Element is no longer attached to the DOM","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:46272","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"sessionId\": \"89569f10-33c8-11e6-bee5-bb4fe1c88b42\", \"id\": \":wdc:1466084714415\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/89569f10-33c8-11e6-bee5-bb4fe1c88b42/element/:wdc:1466084714415/click"}}Screenshot: available via screen

!*

* No element <cookie class=”my-cookie” data-value=”chockolate” /> found

Conclusion● Easy to setup● Easy to implement with new programmers in team - Python is easy and

makes fun!● Everybody is happy with the result - shipping well tested and integrated

application with less bugs makes sense● Python is great even for web mapping software tests.

Result 2: Happy tester

Magdalena: Tester at Cleerio

?Jachym Cepicky

@jachymc

jachym.cepicky@cleerio.com

Magdalena Kabatova

@madlenkk

magdalena.kabatova@cleerio.com