Sphinx autodoc - automated api documentation - PyCon.KR 2015
-
Upload
takayuki-shimizukawa -
Category
Technology
-
view
1.562 -
download
2
Transcript of Sphinx autodoc - automated api documentation - PyCon.KR 2015
Sphinx autodocautomated API documentation
Takayuki Shimizukawa
Sphinx co-maintainer
Sphinx-users.jp
1PyCon Korea 2015
안녕하세요 .annyeonghaseyo.나는 시미즈 강입니다 .
naneun shimizukawa ibnida.
2
こんにちはKon-Nichi-Wa
3
Who am I
@shimizukawa
1. Sphinx co-maintainer
2. Sphinx-users.jp
3. PyCon JP committee
BePROUD co, ltd.4
http://pycon.jp/
5
Sphinx autodoc
6
Do you knowthe Sphinx?
7
Are you usingthe Sphinx?
8
Do you know the docstring?
9
1. def dumps(obj, ensure_ascii=True):2. """Serialize ``obj`` to a JSON formatted
``str``.3. """4. 5. ...6. return ...
Line 2,3 is a docstring
You can see the string by "help(dumps)"
Docstring
10
Have you writtenAPI docs
using docstrings?
11
What is the reason you do not write docstrings.
I don't know what/where should I write.
Are there some docstring format spec?
It's not beneficial.
I'll tell you a good way to write the docstrings.
12
Goal of this session
How to generate a doc from Python source code.
re-discovering the meaning of docstrings.
13
Sphinx introduction& How to setup it
14
What is Sphinx?
15
Sphinx is a documentation generator
Sphinx generates doc as several output formats from the reST text
markup
Sphinx
reSTreSTreStructuredText
(reST)reST
Parser
HTML Builder
ePub Builder
LaTeX Builder texlive
HTMLtheme
Favorite Editor
The history of Sphinx (short ver.)
16
The father of Sphinx
Too hard to maintenance
~2007
Easy to writeEasy to
maintenance2007~
Sphinx Before and After
Before
There was no standard ways to write documentsSometime, we need converting markups into other
formats
After
Generate multiple output format from single sourceIntegrated html themes make read docs easierAPI references can be integrated into narrative docsAutomated doc building and hosting by ReadTheDocs
17
Many docs are written by Sphinx
For examples
Python libraries/tools:Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …
Non python libraries/tools:Chef, CakePHP(2.x), MathJax, Selenium, Varnish
18
Many docs are written by Sphinx
For examples
Python libraries/tools:Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …
Non python libraries/tools:Chef, CakePHP(2.x), MathJax, Selenium, Varnish
19
Sphinx extensions (built-in)
Sphinx provides these extensions to support automated API
documentation.
sphinx.ext.autodocsphinx.ext.autosummarysphinx.ext.doctestsphinx.ext.coverage
Sphinx
reST Parser
HTML Builder
ePub Builder
LaTeX Builder
docutils
auto
sum
ma
ry
auto
doc
doctest
coverage
20
library code example
1. "utility functions"2. 3. def dumps(obj, ensure_ascii=True):4. """Serialize ``obj`` to a JSON formatted
``str``.5. """6. ...
deep_thought/utils.py
$ tree /path/to/your-code+- deep_thought| +- __init__.py| +- api.py| +- calc.py| +- utils.py+- setup.py
21
$ pip install sphinx
Your code and sphinx should be in single python environment.
Python version is also important.
How to install Sphinx
22
$ cd /path/to/your-code$ sphinx-quickstart doc -m...Project name: Deep thoughtAuthor name(s): MiceProject version: 0.7.5......Finished
"-m" to generate minimal Makefile/make.bat-m is important to introduce this session easily.
How to start a Sphinx project
Keep pressing ENTER key
23
Create a doc directory
$ cd doc$ make html...Build finished. The HTML pages are in _build/html.
"make html" command generates html files into _build/html.
make html once
24
Current files structure
$ tree /path/to/your-code+- deep_thought| +- __init__.py| +- api.py| +- calc.py| +- utils.py+- doc| +- _build/| | +- html/ | +- _static/| +- _template/| +- conf.py| +- index.rst| +- make.bat| +- Makefile+- setup.py
Scaffold files
Build output
Library files
25
Generate API docs from your python source code
26
$ tree /path/to/your-code+- deep_thought| +- __init__.py| +- api.py| +- calc.py| +- utils.py+- doc| +- _build/| | +- html/ | +- _static/| +- _template/| +- conf.py| +- index.rst| +- make.bat| +- Makefile+- setup.py
1. import os2. import sys3. sys.path.insert(0,
os.path.abspath('..'))4. extensions = [5. 'sphinx.ext.autodoc',6. ]
setup autodoc extensiondoc/conf.py
27
Line-3: add your library path to import them from Sphinx autodoc.
Line-5: add 'sphinx.ext.autodoc' to use the extension.
Add automodule directive to your doc
1. Deep thought API2. ================3. 4. .. automodule:: deep_thought.utils5. :members:6.
1. "utility functions"2. 3. def dumps(obj, ensure_ascii=True):4. """Serialize ``obj`` to a JSON formatted
``str``.5. """6. ...
doc/index.rst
28
deep_thought/utils.pyLine-4: automodule directive import
specified module and inspect the module.
Line-5: :members: option will inspects all members of module not only module docstring.
$ cd doc$ make html...Build finished. The HTML pages are in _build/html.
make html
29
How does it work?
autodoc directive generates intermediate reST internally:1. Deep thought API2. ================3. 4. .. py:module:: deep_thought.utils5. 6. utility functions7. 8. .. py:function:: dumps(obj, ensure_ascii=True)9. :module: deep_thought.utils10. 11. Serialize ``obj`` to a JSON formatted :class:`str`.
doc/index.rst
IntermediatereST
30
$ make html SPHINXOPTS=-vvv......[autodoc] output:
.. py:module:: deep_thought.utils
utility functions
.. py:function:: dumps(obj, ensure_ascii=True) :module: deep_thought.utils
Serialize ``obj`` to a JSON formatted :class:`str`.
You can see the reST with -vvv option
31
Take care!
Sphinx autodoc import your code
to get docstrings.
It means autodoc will execute code at module global level.
32
Danger code
1. import os2. 3. def delete_world():4. os.system('sudo rm -Rf /')5. 6. delete_world() # will be executed at "make html"
danger.py
33
execution guard on import
1. import os2. 3. def delete_world():4. os.system('sudo rm -Rf /')5. 6. delete_world() # will be executed at "make html"
danger.py
1. import os2. 3. def delete_world():4. os.system('sudo rm -Rf /')5. 6. if __name__ == '__main__':7. delete_world() # doesn't execute at "make
html"
safer.py
Execution guard
34
execution guard on import
1. import os2. 3. def delete_world():4. os.system('sudo rm -Rf /')5. 6. delete_world() # will be executed at "make html"
danger.py
1. import os2. 3. def delete_world():4. os.system('sudo rm -Rf /')5. 6. if __name__ == '__main__':7. delete_world() # doesn't execute at "make
html"
safer.py
Execution guard
35
"Oh, I can't understand the type of arguments and meanings even reading this!"
36
Lacking necessary information
1. def dumps(obj, ensure_ascii=True):2. """Serialize ``obj`` to a JSON formatted3. :class:`str`.4. 5. :param dict obj: dict type object to
serialize.6. :param bool ensure_ascii: Default is True. If7. False, all non-ASCII characters are
not ...8. :return: JSON formatted string9. :rtype: str10. """http://sphinx-doc.org/domains.html#info-field-
lists
"info field lists" for arguments
deep_thought/utils.py
37
def dumps(obj, ensure_ascii=True): """Serialize ``obj`` to a JSON formatted :class:`str`.
:param dict obj: dict type object to serialize. :param bool ensure_ascii: Default is True. If False, all non-ASCII characters are not ... :return: JSON formatted string :rtype: str """ ...
"info field lists" for argumentsdeep_thought/utils.py
38
Cross-reference to functions
1. Examples2. ==========3. 4. This is a usage
of :func:`deep_thought.utils.dumps` blah blah blah. ...
examples.py
reference(hyper link)
39
Detect deviations of the impl and doc
40
Code example in a docstring
1. def dumps(obj, ensure_ascii=True):2. """Serialize ``obj`` to a JSON formatted3. :class:`str`.4. 5. For example:6. 7. >>> from deep_thought.utils import dumps8. >>> data = dict(spam=1, ham='egg')9. >>> dumps(data)10. '{spam: 1, ham: "egg"}'11. 12. :param dict obj: dict type object to
serialize.13. :param bool ensure_ascii: Default is True. If14. False, all non-ASCII characters are
not ...
deep_thought/utils.py
41
doctestblock
You can copy & paste the red lines from python interactive
shell.
Syntax highlighted output
$ make html...
rendereddoctestblock
42
You can expect that developers will update code examples when the interface is changed.
We expect ...
1. def dumps(obj, ensure_ascii=True):2. """Serialize ``obj`` to a JSON formatted3. :class:`str`.4. 5. For example:6. 7. >>> from deep_thought.utils import dumps8. >>> data = dict(spam=1, ham='egg')9. >>> dumps(data)10. '{spam: 1, ham: "egg"}'
The code example is very close from implementation!!
deep_thought/utils.py
43
 ̄\ _( ツ )_/  ̄
44
doctest builder
45
1. ...2. 3. extensions = [4. 'sphinx.ext.autodoc',5. 'sphinx.ext.doctest',6. ]7.
Line-5: add 'sphinx.ext.doctest' extension.
setup doctest extension
doc/conf.py
append
46
$ make doctest...Document: api-------------********************************************************File "api.rst", line 11, in defaultFailed example: dumps(data)Expected: '{spam: 1, ham: "egg"}'Got: 'to-be-implemented'...make: *** [doctest] Error 1
Result of "make doctest"
47
Result of doctest
Listing APIs automatically
48
Individual pages for each modulesapi.html
calc.html
utils.html
49
1. deep_thought.utils2. ===================3. .. automodule:: deep_thought.utils4. :members:
doc/deep_thought.utils.rst
1. deep_thought.utils2. ===================3. .. automodule:: deep_thought.utils4. :members:
Add automodule directive to your doc
doc/deep_thought.utils.rst
1. deep_thought.calc2. ==================3. .. automodule:: deep_thought.calc4. :members:
1. deep_thought.api2. ==================3. .. automodule:: deep_thought.api4. :members:
doc/deep_thought.calc.rst
doc/deep_thought.api.rst
And many many reST files ... 50
 ̄\ _( ツ )_/  ̄
51
autosummary extension
52
1. ...2. 3. extensions = [4. 'sphinx.ext.autodoc',5. 'sphinx.ext.doctest',6. 'sphinx.ext.autosummary',7. ]8. autodoc_default_flags = ['members']9. autosummary_generate = True10.
Line-9: autosummary_generate = True generates reST files what you will specify with using autosummary directive.
setup autosummary extension
doc/conf.py
append
appendappend
53
1. Deep thought API2. ================3. 4. .. autosummary::5. :toctree: generated6. 7. deep_thought.api8. deep_thought.calc9. deep_thought.utils10.
Replace automodule with autosummary
doc/index.rst
$ make html...
54
Output of autosummary
autosummary directive become TOC for each module pages.
55
Discovering undocumented APIs
56
1. def spam():2. ...3. return res4. 5. def ham():6. ...7. return res8. 9. def egg():10. ...11. return res
How to find undocumented funcs?
doc/nodoc.py
57
coverage extension
58
1. ...2. 3. extensions = [4. 'sphinx.ext.autodoc',5. 'sphinx.ext.doctest',6. 'sphinx.ext.autosummary',7. 'sphinx.ext.coverage',8. ]9. autodoc_default_flags = ['members']10.autosummary_generate = True
setup coverage extension
doc/conf.py
append
Line-7: add 'sphinx.ext.coverage' extension.
59
make coverage and check the result
$ make coverage...Testing of coverage in the sources finished, look at the results in _build\coverage.
$ ls _build/coveragec.txt python.txt undoc.pickle
1. Undocumented Python objects2. ===========================3. deep_thought.utils4. ------------------5. Functions:6. * egg
_build/coverage/python.txt
This function doesn't have a doc!60
CAUTION!
1. Undocumented Python objects2. ===========================3. deep_thought.utils4. ------------------5. Functions:6. * egg
python.txt
$ make coverage...Testing of coverage in the sources finished, look at the results in _build\coverage.
$ ls _build/coveragec.txt python.txt undoc.pickle
The command always return ZERO
coverage.xml is not exist
reST format for whom?
61
Let's make Pull-Request!
We are waiting for your contribution
to solve the problem.(bow)
62
Wrap-up
63
Why don't you write docstrings?
I don't know what/where should I write.Let's write a description, arguments and
doctest blocks at the next line of function signature.
Are there some docstring format spec?Yes, you can use "info field list" for argument
spec and you can use doctest block for code example.
It's not beneficial.You can use autodoc, autosummary,
doctest and coverage to make it beneficial.
64
Options, Tips
65
1. Options
66
Options for autodoc
:members: blahTo document just specified members. Empty is ALL.
:undoc-members: ...To document members which doesn't have docstring.
:private-members: ...To document private members which name starts with
underscore.
:special-members: ...To document starts with underscore underscore.
:inherited-members: ...To document inherited from super class.
67
2. Directives for Web API
sphinxcontrib-httpdomain
68
sphinxcontrib-httpdomain extension
http domain's get directive
render
page
render routing table (index)
http highlighter
It also contains sphinxcontrib.autohttp.flask, .bottle, .tornado extensions
Listing
New Index
69
3. Directives for Diagram
sphinxcontrib-blockdiagsphinxcontrib-sqdiagsphinxcontrib-actdiagsphinxcontrib-nwdiag
70
blockdiag series
71
http://blockdiag.com/
blockdiag seqdiag (sequence)
actdiag (activity) nwdiag (network)
packetdiag (packet) is included in nwdiag
class Request(object): """ .. seqdiag::
{ code [label="Your Code"]; lib [label="Library"]; site [label="Web Site"];
code -> lib [label="Request(url)"]; code <-- lib [label="req"]; code -> lib [label="req.open(auth_cb)"]; lib -> site [label="GET"]; lib <-- site [label="status=401"]; lib => code [label="auth_cb(realm)", ... lib -> site [label="GET/auth header"]; lib <-- site [label="status=200"]; code <-- lib [label="status"]; } """
sphinxcontrib-seqdiag extension
72
deep_thought/api.py
extensions = [ 'sphinxcontrib.seqdiag',]
conf.py
$ pip install sphinxcontrib-seqdiag
4. Document translation
73
Translation into other languages
$ make gettext...Build finished. The message catalogs are in _build/gettext.
$ sphinx-intl update -p _build/gettext -l es
#: ../../../deep_thought/utils.pydocstring of deep_thought.msgid "Serialize ``obj`` to a JSON formatted :class:`str`."msgstr "Serializar ``obj`` a un formato JSON :class:`str`."
msgid "For example:"msgstr "Por ejemplo:"
locale/es/LC_MESSAGES/generated.po
language = 'es'locale_dirs = ['locale']
conf.py
$ make html...Build finished. The HTML pages are in _build/html. 74
Questions?@shimizukawa
Grab me at Office Hour!(After this session)
75
Thanks :)
76