Sphinx + Robot Framework = documentation as result of functional testing

Paweł Lewicki

Goldmund, Wyldebeast & Wunderliebe

lewicki@gw20e.com@plewy

● Tools● Simple example● Useful example

Agenda

● reStructured text● Python documentation generator● Pluggable● Different outputs● Standard in Python world

Tools: Sphinx

Tools: Robot Framework

● Tests automation framework● tool for acceptance test-driven development● keyword-driven testing approach● Web testing with Selenium2 library

o headless with PhantomJS

*** Variables ***

${LOGIN_URL} https://www.python.org/accounts/login/${HOME_URL} https://www.python.org

*** Keywords ***

Open Browser To Login Page

Open Browser ${LOGIN URL}

Maximize Browser Window

Title Should Be Sign In to Python.org

test.robot(1/2)

*** Test Cases ***

Valid Login Open Browser To Login Page Input Text id_login demouser Input Text id_password demopass Click button css=.primaryAction

Location Should Be ${HOME_URL}

Page Should Contain demouser Title Should Be Welcome To Python.org [Teardown] Close Browser

test.robot(2/2)

Keywords - selenium2

● “Add Cookie”● “Click Button/Image/Link”● “Drag And Drop”● “Element Should Contain”● “Input Password”● “Mouse Down”● “Radio Button Should Be Set To”● “Set Window Size”

Keywords - selenium2screenshots

● “Add pointy note”● “Remove element”● “Update element style”● “Align elements horizontally”● “Capture viewport screenshot”● “Crop page screenshot”● jQuery based functions

● buildout created with zc.buildout● Fixture of Sphinx documentation● reST code enriched with Robot Framework

Minimal setup

[buildout]

parts = sphinx-build

[sphinx-build]

recipe = zc.recipe.egg

eggs =

Sphinx

sphinxcontrib-robotframework [docs]

sphinx-rtd-theme

scripts =

sphinx-build

# Usage:

# bin/sphinx-build -b html docs build

Minimal setupMinimal setup: buildout.cfg

+---desktop \---_screenshots \---index.rst

+---tablet \---_screenshots \---index.rst

+---mobile \---_screenshots \---index.rst

conf.py

index.rstrobot.rst_frontpage.rst_topmenu.rst

Minimal setup: docs/

==========================

PyGrunn site documentation

==========================

.. include:: robot.rst

.. toctree::

:maxdepth: 2

:numbered:

desktop/index

tablet/index

mobile/index

docs/: index.rst

.. code:: robotframework

*** Settings ***

Resource Selenium2Screenshots/keywords.robot

*** Variables ***

${BASE_URL} = http://pygrunn.org

*** Keywords ***

Suite Teardown

Close All Browsers

docs/: robot.rst

Tablet (768x)

===============

.. code:: robotframework

… (cut part)

*** Keywords ***

Suite Setup

Open Browser ${BASE_URL}

Set window size 768 900

*** Test Cases ***

.. include:: ../_frontpage.rst

.. include:: ../_topmenu.rst

docs/: tablet/index.rst

Initial viewport

----------------

Below we can see the screenshot taken from current pyGrunn.org site.

.. figure:: _screenshots/homepage.png

.. code:: robotframework

Site homepage viewport

Go to ${BASE_URL}

Capture viewport screenshot _screenshots/homepage.png

docs/: _frontpage.rst

$bin/sphinx-build -b html docs build

…and…

Cropping and annotations

------------------------

Below we may see cropped and annotated page element.

.. figure:: _screenshots/topmenu.png

.. code:: robotframework

Capture annotated top menu

${note1} = Add pointy note

... css=.headerbar

... This Robot Framework stuff is very very cool!

... width=200 position=bottom

Capture and crop page screenshot _screenshots/topmenu.png

... css=.headerbar ${note1}

docs/: topmenu.rst

Useful example

● Plone theme product● Documentation as part of testing● Jenkins as visual regression tester

Plone product (package/egg)

● plone.app.testingo fixture of fully working Plone site

● plone.app.robotframeworko Plone-related keywords

Plone setup: testing.pyfrom plone.app.testing import FunctionalTestingfrom plone.app.robotframework.testing import REMOTE_LIBRARY_BUNDLE_FIXTURE

EXAMPLE_THEME_ROBOT_TESTING = FunctionalTesting( bases=(EXAMPLE_THEME_ACCEPTANCE_FIXTURE, REMOTE_LIBRARY_BUNDLE_FIXTURE, z2.ZSERVER_FIXTURE), name="ExamplethemeLayer:Robot")

$bin/robot-server example.theme.testing.EXAMPLE_THEME_ROBOT_TESTING

Edit document

.............

.. figure:: _screenshots/document-edit.png

.. code:: robotframework

Capture Document Edit Screenshot

${uid} = Create content type=Document

... id=new-document

... title=${DUMMY_TEXT_LINE}

... description=${DUMMY_TEXT_SHORT} text=${DUMMY_TEXT}

Fire transition ${uid} publish

Go to ${PLONE_URL}/new-document/edit

Capture page screenshot _screenshots/document-edit.png

Plone setup: Robot content creation

Keywords - plone.app.robotframework

● “Debug”● “Log in as test user”● “Trigger Workflow Transition”● “Add content”● “Click Action by id”

● Git repository● Bash script

virtualenv .

bin/pip install distribute

bin/python bootstrap.py

bin/buildout

bin/test

bin/sphinx-build -b html docs build

● Documentation (build/index.html)

Jenkins setup

Useful areas

● Customer documentation● Visual coverage of theme development

o commit hooks trigger new version● Multiple versions

o Screen sizes (responsiveness)o Language versions

http://www.youtube.com/watch?v=VN9FROZO5AY

o the same application many themes

● FUNCTIONAL TESTING

References

● https://github.com/lewicki/pygrunn_robot_demo

Thank you!Questions?

Paweł Lewicki

Goldmund, Wyldebeast & Wunderliebe

lewicki@gw20e.com@plewy