Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type...
Transcript of Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type...
![Page 1: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/1.jpg)
Using Type Annotations in Pythonby Philippe Fremy / IDEMIA
![Page 2: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/2.jpg)
Python code can be obscure
• You do not know the types of the arguments
• The function may accept multiple types and you don’t know it
• Docstrings (when present) may not be accurate or useful
• You may break production code just be providing an unexpected type and you will only notice it at run-time.
def validate(form, data):"""Validates the input data"""return form.validate(data)
![Page 3: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/3.jpg)
Very brief history of type annotations
• Function argument annotations introduced for Python 3.0 in 2006 by Guido van Rossum
• Type annotations introduced in Python 3.5 (2015)
• Further improved in Python 3.6 (2016) and Python 3.7 (2018)
![Page 4: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/4.jpg)
Syntax for type annotation
# a functiondef my_func(a : int, b : str = "") -> bool:
# ...
# a methodclass A:
def my_method(self, a : bool, b : int = 0) -> None: # ...
![Page 5: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/5.jpg)
# Variables (only with python 3.6 and later)a: int = 0 b: str
class MyClass:
c: float # type of the instance variable# (only with python 3.6 and later)
def __init__(self) -> None: self.c = 33.17 self.d: str = "I am a string"
Syntax for type annotation
![Page 6: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/6.jpg)
Available types for annotations
# List defines a general content typemy_list_int: List[int] = [1,2,3]
# multiple types content requires Unionmy_multi_type_list: List[ Union[bool, int] ] = [ True, 33 ]
# Tuple usually define precisely all their membersmy_tuple: Tuple[int, str, float] = (1, "abc", 3.14)
# Tuple can also declare a general content typemy_float_tuple: Tuple[float, ...] = (11.14, 20.18, 0.1)
![Page 7: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/7.jpg)
Available types for annotations
# Dict defines keys and content typemy_dict: Dict[str, int] = { "33": 17 }
# Containers may be combinedschool_coords: Dict[ str, Tuple[int, int] ] school_coords = {"Epita": (10, 20)}
![Page 8: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/8.jpg)
# None is a valid type annotationdef f(a: None) -> int:
...
# None is always used in a Union:def f(a: Union[None, int]) -> int:
...
# Union[None, int] may be spelled as Optional[int]def f(a: Optional[int] = None) -> int:
...
Available types for annotations
![Page 9: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/9.jpg)
And there is more…
The typing module also offers :
• Duck typing with types such as Sequence, Mapping, Iterable, Sized, ...
• Type aliasing, type generics, subtyping, typing joker with Any, …
• Conversion between types with cast
Please check the typing module documentation and the Mypy tool
![Page 10: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/10.jpg)
How does Python handle type annotations ?
• Annotations are valid expressions evaluated during module loading
• Result is stored in the function object
• And then … they are totally ignored by Python
Type annotations are verified by external tools : Mypy, Pyre, …
![Page 11: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/11.jpg)
Type Annotations verification tools
Tools to verify static type information:
• PyCharm IDE along with inspection mode
• Mypy : Open Source, written in Python, maintained by Dropbox team on GitHub
• Pyre : Open Source, written in OCaml, maintained by Facebook team on GitHub, only for Linux and MacOs X
![Page 12: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/12.jpg)
How to get started with annotations
• On a new codebase set the rule of having annotations and be strict about it.
• On an existing codebase, start small, one module at a time.
Then improve gradually.
All the annotation tools are designed for gradual improvements.
• Put static type verification in your Continuous Integration / Nightly builds / non regression tests.
![Page 13: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/13.jpg)
Proceed one module at a timeStep 1: add annotations to my_module.py and verify them
Mypy in strict mode complains about every missing annotation.
$ mypy --strict my_module.pymy_module.py:11: error: Function is missing a return type annotation
![Page 14: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/14.jpg)
Proceed one module at a timeStep 1: add annotations to my_module.py and verify them
Mypy in strict mode complains about every missing annotation.
Step 2: when the module is fully annotated, check the whole codebase.
Mypy reports every misuse of my_module (only in annotated code).
$ mypy --strict my_module.pymy_module.py:11: error: Function is missing a return type annotation
$ mypy *.pymod2.py:5: error: Argument 1 to "my_func" has incompatible type "float"; expected "int"
![Page 15: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/15.jpg)
Proceed one module at a timeStep 1: add annotations to my_module.py and verify them
Mypy in strict mode complains about every missing annotation.
Step 2: when the module is fully annotated, check the whole codebase.
Mypy reports every misuse of my_module (only in annotated code).
Step 3: run your non-regression tests
$ mypy --strict my_module.pymy_module.py:11: error: Function is missing a return type annotation
$ mypy *.pymod2.py:5: error: Argument 1 to "my_func" has incompatible type "float"; expected "int"
![Page 16: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/16.jpg)
Where to add type annotation
# annotate all your functions and methods
# variable with value do not need type annotationvat_rate = 20 # OK, vat_rate is an int
# unless the value type is not correct…if reduced_vat:
vat_rate = 5.5 # Error from mypy, vat_rate does not accept float
vat_rate: float = 20 # OK for float and int values
![Page 17: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/17.jpg)
Where to add type annotations
# All empty containers need annotations
names = [] # Mypy can not figure out the content type
names: List[str] = [] # OK
# Dict and other empty containers need annotationsbirth_dates: Dict[str, Date]birth_dates = {}
![Page 18: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/18.jpg)
Let’s practice
Example 1
![Page 19: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/19.jpg)
class A: def use_another_a(self, a: A) -> None:
pass
def use_b(self, b: Optional[B]) -> None: pass
class B: pass
![Page 20: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/20.jpg)
$ mypy --strict ab.py$$ python ab.pyFile “ab.py", line 4, in A
def use_another_a( self, a: A ) -> None:NameError: name 'A' is not definedFile “ab.py", line 7, in A
def use_b( self, b: Optional[B] ) -> None:NameError: name 'B' is not defined
class A: def use_another_a(self, a: A) -> None:
pass
def use_b(self, b: Optional[B]) -> None: pass
class B: pass
![Page 21: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/21.jpg)
from __future__ import annotations # python 3.7 only
class A: def use_another_a(self, a: A) -> None:
pass
def use_b(self, b: Optional[B]) -> None: pass
class B: pass
$ mypy --strict ab.py$ $ python ab.py$
![Page 22: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/22.jpg)
# Other solution: put annotations inside quotes
class A: def use_another_a(self, a: "A") -> None:
pass
def use_b(self, b: Optional["B"]) -> None:pass
class B: pass
$ mypy --strict ab.py$ $ python ab.py$
![Page 23: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/23.jpg)
Let’s practice
Example 2
![Page 24: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/24.jpg)
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init
def get_step(self) -> int: return self.step + 1
![Page 25: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/25.jpg)
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init
def get_step(self) -> int: return self.step + 1
$ mypy --strict a.pya.py:6: error: Unsupported operand types for + ("Optional[int]" and "int")
![Page 26: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/26.jpg)
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init
def get_step(self) -> int: return self.step + 1
$ mypy --strict a.pya.py:6: error: Unsupported operand types for + ("Optional[int]" and "int")
Mypy found a bug !
![Page 27: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/27.jpg)
# Solution 1: prepend a check for None
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init
def get_step(self) -> int: # deal with self.step being Noneif self.step is None: return 0
# now we can proceedreturn self.step + 1
$ mypy --strict a.py$
![Page 28: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/28.jpg)
# Solution 2: default initialise with the right type
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init or 0 # self.step type is always int
def use_step(self) -> int: return self.step + 1
$ mypy --strict a.py$
![Page 29: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/29.jpg)
# Solution 3: do not use Optional, have better default
class A: def __init__(self, step_init: int = 0) -> None:
self.step = step_init
def get_step(self) -> int: return self.step + 1
$ mypy --strict a.py$
![Page 30: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/30.jpg)
# Solution 4: disable None checking in Mypy
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init
def get_step(self) -> int: return self.step + 1
$ mypy --strict --no-strict-optional a.py$
![Page 31: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/31.jpg)
# Solution 5: silence the error (not a good practice)
class A: def __init__(self, step_init: Optional[int] = None) -> None:
self.step = step_init
def get_step(self) -> int: return self.step + 1 # type: ignore
$ mypy --strict a.py$
![Page 32: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/32.jpg)
Let’s practice
Example 3
![Page 33: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/33.jpg)
# Dealing with multiple types
def upper(thing: Union[str, bytes, List[str]]) -> str:if type(thing) == list:
thing = "".join(thing)
return thing.upper()
$ mypy --strict upper.pyupper.py:5: error: Argument 1 to "join" of "str" has incompatible type "Union[str, bytes, List[str]]"; expected "Iterable[str]"upper.py:8: error: Incompatible return value type (got "Union[str, bytes, List[str]]", expected "str")
![Page 34: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/34.jpg)
# Dealing with multiple types
def upper(thing: Union[str, bytes, List[str]]) -> str:if type(thing) == list:
thing = "".join(thing)
return thing.upper()
![Page 35: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/35.jpg)
# Dealing with multiple types
def upper(thing: Union[str, bytes, List[str]]) -> str:if type(thing) == list:
thing = "".join(thing)
return thing.upper()
$ mypy --strict upper.pyupper.py:5: error: Argument 1 to "join" of "str" has incompatible type "Union[str, bytes, List[str]]"; expected "Iterable[str]"upper.py:8: error: Incompatible return value type (got "Union[str, bytes, List[str]]", expected "str")
![Page 36: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/36.jpg)
# Solution: use isinstance()
def upper(thing: Union[str, bytes, List[str]]) -> str: if isinstance(thing, list): # mypy understand isinstance()
thing = "".join(thing) # so now, join() passes fine
return thing.upper()
![Page 37: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/37.jpg)
Mypy found a bug !I forgot to deal with bytes
$ mypy --strict upper.pyupper.py:7: error: Incompatible return value type (got "Union[str, bytes]", expected "str")
# Solution: use isinstance()
def upper(thing: Union[str, bytes, List[str]]) -> str: if isinstance(thing, list): # mypy understand isinstance()
thing = "".join(thing) # so now, join() passes fine
return thing.upper()
![Page 38: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/38.jpg)
Mypy found a bug !I forgot to deal with bytes
$ mypy --strict upper.pyupper.py:7: error: Incompatible return value type (got "Union[str, bytes]", expected "str")
# Solution: use isinstance()
def upper(thing: Union[str, bytes, List[str]]) -> str: if isinstance(thing, list): # mypy understand isinstance()
thing = "".join(thing) # so now, join() passes fine
return thing.upper()
![Page 39: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/39.jpg)
$ mypy --strict upper.py$
# Solution: use isinstance() and catch all types
def upper(thing: Union[str, bytes, List[str]]) -> str: if isinstance(thing, list):
thing = "".join(thing)
elif isinstance(thing, bytes): # we also check for bytesthing = thing.decode("UTF8")
# now, all paths make thing a stringreturn thing.upper() # OK, returning a str
![Page 40: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/40.jpg)
$ mypy --strict upper.py$
# Solution: use cast and catch all types
def upper(thing: Union[str, bytes, List[str]]) -> str:if type(thing) == list:
thing = cast(List[str], thing) thing = "".join(thing)
elif type(thing) == bytes:thing = cast(bytes, thing) thing = thing.decode("UTF8")
thing = cast(str, thing) return thing.upper()
![Page 41: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/41.jpg)
Let’s practice
Example 4
![Page 42: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/42.jpg)
# file form_validator.py
def validate(form, data):# ... (do some pre-validation stuff)return form.validate(data)
class UserForm:def validate(self, data):
"""Validates the data. Data must be a list of int"""data[4] = data[1] * data[2] % data[3]return data[4] > 21
![Page 43: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/43.jpg)
# file production_code.py
def production_code():userForm = UserForm()data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# ... return validate(userForm, data)
![Page 44: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/44.jpg)
# file production_code.py
def production_code(): userForm = UserForm() # data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] data = range(10)# ...return validate(userForm, data)
![Page 45: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/45.jpg)
# file production_code.py
def production_code(): userForm = UserForm() # data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] data = range(10)# ...return validate(userForm, data)
$ python production_code.pyTraceback (most recent call last):
File "production_code.py", line 7, in <module>production_code()
File "form_validator.py", line 4, in validatereturn form.validate(data)
File "form_validator.py", line 9, in validatedata[4] = data[1] * data[2] % data[3]
TypeError: 'range' object does not support item assignment
![Page 46: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/46.jpg)
# file form_validator.py
def validate(form: UserForm, data: List[int]):# ... (do some pre-validation stuff)return form.validate(data)
class UserForm: def validate(self, data: List[int]):
"""Validates the data. Data must be a list of int""" data[4] = data[1] * data[2] % data[3] return data[4] > 21
![Page 47: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/47.jpg)
# file production_code.py
def production_code() -> bool: userForm = UserForm() # data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] data = range(10)# ...return validate(userForm, data)
![Page 48: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/48.jpg)
# file production_code.py
def production_code() -> bool: userForm = UserForm() # data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] data = range(10)# ...return validate(userForm, data)
$ mypy production_code.pyproduction_code.py:7: error: Argument 1 to "validate" has incompatible type "range"; expected "List[int]“
![Page 49: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/49.jpg)
# file production_code.py
def some_production_code() -> bool: userForm = UserForm() # data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] data = list(range(10))# ...return validate(userForm, data)
$ mypy production_code.py$$ python production_code.py…
![Page 50: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/50.jpg)
Let’s practice
Monkeytype
![Page 51: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/51.jpg)
Let the monkey find the types for you
def validate(form, data):"""Validates the input data"""return form.validate(data)
![Page 52: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/52.jpg)
Let the monkey find the types for you
$ monkeytype run all_unit_tests.py $ monkeytype run end_to_end_tests.py $ monkeytype run production_code.py $ monkeytype apply validate
def validate(form, data):"""Validates the input data"""return form.validate(data)
![Page 53: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/53.jpg)
Let the monkey find the types for you
You can also use PyAnnotate which does the same thing.
$ monkeytype run all_unit_tests.py $ monkeytype run end_to_end_tests.py $ monkeytype run production_code.py $ monkeytype apply validate
def validate(form, data):"""Validates the input data"""return form.validate(data)
def validate(form: Union[UserForm, AdminForm], data: List[int]) -> bool:
"""Validates the input data"""return form.validate(data)
![Page 54: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/54.jpg)
Conclusion• Type annotation is powerful to bug finder. Use it !
• Type annotation is also good way of documenting your code
• Feedback from developers using type annotation: “It rocks !”
• Some Python dynamic constructs are difficult to verify staticallyThat’s why you should go step-by-step when adding annotationsMypy has excellent documentation to complement this presentation
• Tools like MonkeyType or PyAnnotate can really help.
![Page 55: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/55.jpg)
Philippe Fremy / IDEMIA in Bordeaux(IDEMIA is recruiting)
Slides are online at PyParis websiteand at:
https://github.com/bluebird75/whoiam
![Page 56: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/56.jpg)
Time for questions
![Page 57: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/57.jpg)
Let’s practice - extra track
Example 5
It did not fit into the 40 minutes track
![Page 58: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/58.jpg)
# file a.py
class A:# ...
# file b.pyfrom a import A
class B(A):pass
![Page 59: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/59.jpg)
# file a.py
class A: def use_some_b(self, b):
pass
# file b.py from a import A
class B(A): pass
![Page 60: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/60.jpg)
# file b.py from a import A
class B(A): pass
# file a.py
class A: def use_some_b(self, b: B) -> None:
pass
![Page 61: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/61.jpg)
# file b.py from a import A
class B(A): pass
# file a.py
class A: def use_some_b(self, b: B) -> None:
pass
$ mypy --strict a.py b.pya.py:6: error: Name 'B' is not defined
![Page 62: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/62.jpg)
# file b.pyfrom a import A
class B(A): pass
# file a.pyfrom b import B
class A: def use_some_b(self, b: B) -> None:
pass
![Page 63: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/63.jpg)
# file b.pyfrom a import A
class B(A): pass
# file a.pyfrom b import B
class A: def use_some_b(self, b: B) -> None:
pass
$ mypy --strict a.py b.py$
![Page 64: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/64.jpg)
# file b.pyfrom a import A
class B(A): pass
# file a.pyfrom b import B
class A: def use_some_b(self, b: B) -> None:
pass
$ mypy --strict a.py b.py$$ python a.pyFile "b.py", line 2, in <module>
from a import AFile "a.py", line 2, in <module>
from b import BFile "b.py", line 2, in <module>
from a import AImportError: cannot import name 'A' from 'a'
![Page 65: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/65.jpg)
# file b.pyfrom a import A
class B(A): pass
# file a.pyfrom b import B
class A: def use_some_b(self, b: B) -> None:
pass
Ouch, import cycle !
$ mypy --strict a.py b.py$$ python a.pyFile "b.py", line 2, in <module>
from a import AFile "a.py", line 2, in <module>
from b import BFile "b.py", line 2, in <module>
from a import AImportError: cannot import name 'A' from 'a'
![Page 66: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/66.jpg)
# file b.pyfrom a import A
class B(A): pass
# file a.py# we do not import B and use quotes# like in forward references
class A: def use_some_b(self, b: "B") -> None:
pass
![Page 67: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/67.jpg)
# file b.pyfrom a import A
class B(A): pass
# file a.py# we do not import B and use quotes# like in forward references
class A: def use_some_b(self, b: "B") -> None:
pass
$ python a.py$$ mypy --strict a.py b.pya.py:6: error: Name 'B' is not defined
![Page 68: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/68.jpg)
# file a.pyfrom typing import TYPE_CHECKING if TYPE_CHECKING:
from b import B
class A: def use_some_b(self, b: "B") -> None:
pass
# file b.pyfrom a import A
class B(A): pass
![Page 69: Using Type Annotations in Pythonpyparis.org/static/slides/Philippe+Fremy-31b0239a.pdf · •Type annotations introduced in Python 3.5 (2015) •Further improved in Python 3.6 (2016)](https://reader035.fdocuments.net/reader035/viewer/2022070113/605e31a03428b647e2726f3d/html5/thumbnails/69.jpg)
# file a.pyfrom typing import TYPE_CHECKING if TYPE_CHECKING:
from b import B
class A: def use_some_b(self, b: "B") -> None:
pass
# file b.pyfrom a import A
class B(A): pass
$ python a.py$$ mypy --strict a.py b.py$