New-Style Classes

Thomas Wouters

XS4ALL

thomas@xs4all.net

Yhg1s @ #python

Overview

• Old situation– types, classes– shape of Python

• Type unification– subtyping– descriptors and properties– class- and static-methods

• Metaclasses

Old situation: types

• Python types implemented in C– C structs represent types– C function pointers define functionality– No real class data– C structs represent objects– All instance data in object structs– 'Manual' attribute retrieval– Explicit methods

Old situation: Python classes

• Implemented in terms of Python types– "bolted on top"– 'classes' and 'instances' as distinct types– C function pointers delegated to 'magic'

methods to define behaviour– tp_getattro function searches base classes– Implicit methods from Python functions– Class and instance data in '__dict__' attribute

Old situation: limitations

• Python and C as two distinct worlds

• No real accessors

• No class/static methods

• No immutable classes

• Simplistic inheritance order

• Less control over behaviour

Old situation: consequences

• Interface-based functionality– informal interfaces rather than inheritance

• Containment rather than inheritance– inheritance is not always the answer

• Simple inheritance trees

• Easy to use

Type unification

• Allow (better) mingling of Python and C types (or classes)– bring C and Python closer

• __dict__ and inheritance for C types

• descriptors, properties, class/staticmethods for Python classes

– subclassing C types in C

– subclassing C types in Python

• Correct 'warts' in classic classes• Generalize special cases

Type unification (2)

• Metaclasses: a new type of type• Explicit base class: object

– container of generic functionality

• Old-style classes for compatibility– hidden metaclass

• Translation from C function-pointers to Python __methods__ (slots)

• __dict__ for C types and __slots__ for Python classes

Subclassing

• Subclass C types from Python in the expected manner:

class mylist(list): def __getitem__(self, i): try: return list.__getitem__(self, i) except IndexError: return None

• Resricted multiple inheritance• Not always a good idea!

Subclassing (2)

• __new__, Python's constructor

– called to construct (allocate) the object

– static method, called with class as first argument

– may return an existing value

• __slots__: store data almost like C would

– no __dict__, less memory consumption

Immutable Python typesclass tristate(int):

__slots__ = []

nstates = 3

def __new__(cls, state=0):

state %= cls.nstates

return int.__new__(cls, state)

def __add__(self, o):

return tristate(int.__add__(self, o))

def __sub__(self, other):

return self.__add__(-other)

Subclassing C types in C

• Make sure base type supports subclassing– Py<type>_Check(), Py<type>_CheckExact()– PyMethodDef, PyMemberDef, PyGetSetDef– PyObject_GenericGetAttr as tp_getattro– no type-object hardcoding

• Subclass's PyType object– leave unchanged behaviour up to base type– set tp_base to base class– call base class's tp_init

• Provide compatible object struct

Type checking

• Type-checking is a necessary evil (in C)• PyObject_TypeCheck() for inheritance-aware

PythonC type check• Define Py<type>_Check() in terms of

PyObject_TypeCheck()• Define Py<type>_CheckExact() as type-pointer

comparison• Use Py<type>_CheckExact() for internal

optimizations

PyList_Check*()

#define PyList_Check(op) \

PyObject_TypeCheck(op, \

&PyList_Type)

#define PyList_CheckExact(op)\

((op)->ob_type == \

&PyList_Type)

Py*Def

• Allow subclasses to extend/override parts • PyMemberDef (tp_members) for instance data

– maps C structs to Python attributes– tp_members in type struct

• PyMethodDef (tp_methods) for all methods– wraps C functions in Python objects– specifies argument style and type of method

• PyGetSetDef (tp_getset) for accessors– maps functions to attributes and vice versa

Subclass struct

• Include base class struct in subclass struct typedef struct {

PyListObject list;

PyObject * default;

} defaultlistobject;

• No changes to original memory layout• Multiple inheritance is only possible with

'compatible memory layouts'• C subclasses subclassable in Python

Method Resolution Order

• Old MRO not suited to complex inheritance trees– base classes get queried before some of their

derived classes– base classes get queried multiple times– No convenient way to access base classes

• hardcode base class names

• guess about attributes / methods

New MRO

• Published algorithm: C3– http://www.webcom.com/haahr/dylan/linearizat

ion-oopsla96.html

• Relatively easy to explain– same order as before– eliminates all but the last occurance of the same

class

• Same order for simple inheritance

Old-style MRO: D, B, A, C, ANew-style MRO: D, B, C, A (see __mro__)

A

D(B, C)

C(A)B(A)

A

D(B, C)

C(A)B(A)

E(C, B)

F(D, E)

Old-style MRO: F, D, B, A, C, A, E, C, A, B, ANew-style MRO (2.2): F, D, E, B, C, A

super()

• Proxy'object for accessing 'base' classes

• Continues MRO where it left off– requires current class and (derived) instance

• Somewhat inconvenient to use

• Very important for consistency

• Use it anyway

super() use

class BStore(Storage):

def __init__(self, state):

Storage.__init__(self, state)

class BStore(Storage):

def __init__(self, state):

super(BStore, self).__init__(state)

Descriptors

• Generalization of class-getattr magic and C tp_getattr tricks

• Trigger functioncalls when retrieved or stored from an object (getattr/setattr)– __get__()– __set__()– __delete__()

Properties

• Accessors for Python• An application of descriptors• class R(object): def _get_random(self): return random.random() random = property(_get_random)• Also hold docstrings for attributes• 'set' and 'delete' functions don't work with old-

style classes

Caching Property

class cachingprop(object):

__slots__ = ["_name", "_fget"]

def __init__(self, name, fget):

self._name = name

self._fget = fget

def __get__(self, inst, type=None):

if inst is None:

return self

v = self._fget(inst)

inst.__dict__[self._name] = v

return v

Special method types

• classmethods– Passes class as implicit first argument– can be called through class or through

instance– allow for factory functions (or 'alternate

initializers') that create subclasses• dict.fromkeys• tarfile.TarFile.open

Special Method Types (2)

• staticmethods– Passes no special arguments

– Necessary for object.__new__ (or is it?)

– Allows for regular (non-method) Python functions as attributes

Special Method Types (3)

class Buffer(object):

def __init__(self, data):

self.data = data[:]

def fromstring(cls, s):

return cls(s.splitlines())

fromstring = classmethod(fromstring)

def send(self):

self._extern_send(self.data)

_extern_send = staticmethod(sendmodule.send)

Metaclasses

• The class of class

• Usually derives from type

• Relate to classes like classes relate to instances

• Define class behaviour

• Allow for convenient post-processing of classes

Class/instance relation

• Creating the instance passes the contents (arguments) to the class __init__:

class Send(object):

def __init__(self, what, who):

...

Send("my data", him)

Metaclass/class relation

class Meta(type):

def __init__(self, name, bases, attrs):

type.__init__(self, name, bases, attrs)

class Impl(base1, base2):

__metaclass__ = Meta

X = 1

def method(self, it):

return not it

stat = staticmethod(...)

Metaclasses• Behave like classes:

– __new__ called for class creation– __init__ called for class initialization– inheritance

• Mixing metaclasses requires compatibility– derived classes must have same or derived

metaclasses– metametaclasses can automatically derive

metaclasses

Questions ?

Slides will be on http://www.xs4all.nl/~thomas/python/