Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python...
Transcript of Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python...
![Page 1: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/1.jpg)
Mixing Python and C
2012-07-10Martin Renold
![Page 2: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/2.jpg)
Content
● MyPaint, Python and C● Profiling (demo)
● Tools to speed up a Python app● SWIG for minimalists
![Page 3: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/3.jpg)
![Page 4: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/4.jpg)
MyPaint
● Painting / Sketching● Easy to use● Graphic tablets
– Stylus pressure
– Subpixel motion
● Related Projects– Krita (full digital workflow, more complex)
– GIMP (main focus is manipulation)
![Page 5: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/5.jpg)
MyPaint
● Code– 80% Python, 20% C/C++
– 25K lines of code
– Using GTK
● Project– Started in 2004
– Quite popular today
![Page 6: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/6.jpg)
Why Python?
Python
for i in items: do_something(i)
C++
for(std::vector<std::string>::const_iterator i = items.begin(); i != items.end(); ++i){ do_something(*i);
![Page 7: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/7.jpg)
But... Python is slow!
● Press a key, wait– 0.0001 seconds, instead of
0.00001 seconds
● 90% of the code is fast enough in any language.
● Now about the 10%...
![Page 8: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/8.jpg)
Fast Enough?
● Python:– GUI
– „for each tile“
– „for each motion event“
● C/C++:– „for each pixel“
– low-level algorithms (eg. interpolation)
![Page 9: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/9.jpg)
Profiling
● Classical mistake:
1. Guess what is slow
2. Optimize the wrong code
● Measure it!
--> Tool Demo– gprof2dot.py (Python)
– perf (C, Linux)
![Page 10: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/10.jpg)
Profiling Python
● cProfile, gprof2dot.py
![Page 11: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/11.jpg)
Optimize Saving
● Use libpng directly
● Saving PNG (libpng) too slow?– Decrease compression rate
– Tell libpng not to try all possible filters!
![Page 12: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/12.jpg)
Speeding up Python
Fast code is... Tool
Pure Python PyPy
Python superset Cython
Pure C SWIG | CPython API
C++ SWIG | Boost.Python | SIP
C with GObject GObject Introspection
![Page 13: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/13.jpg)
SWIG: Code
int answer() { return 42;}
hello.hpp
%module hello%{#include "hello.hpp"%}%include "hello.hpp"
hello.i
![Page 14: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/14.jpg)
from distutils.core import setup, Extension
setup(ext_modules=[ Extension("_hello", ["hello.i"])])
setup.py
$ python setup.py build_ext -i$ python>> import hello>> hello.answer()42
SWIG: Compiling
![Page 15: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/15.jpg)
SWIG: The End.
● Do not learn more SWIG!– People have died while trying to figure out
SWIG Typemaps
● Use the Python/C API– SWIG supports this
![Page 16: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/16.jpg)
Python/C API
● Reference Counting– Py_DECREF, Py_INCREF macros
PyObject * func(PyObject * arg);
New Reference Borrowed Reference
![Page 17: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/17.jpg)
Example
class Gradient { public: float parm1;
PyObject * get_color(float x, float y) { int r, g, b; // ... return Py_BuildValue("ddd", r, g, b); }};
>> g = hello.Gradient()>> g.parm1 = 2.8>> r, g, b = g.get_color_at(0, 0)
![Page 18: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/18.jpg)
NumPyArray
GDKPixBuf
PILImage
CairoSurface
......
Memory Access („Buffer Protocol“)
C Pointer and Array Dimensions
PyObject_GetBuffer()
![Page 19: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/19.jpg)
Debug and Profile
$ gdb /usr/bin/python(gdb) run program.py
● Like a C/C++ library
![Page 20: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/20.jpg)
Memory Leaks
● Unused References (common)– Hard to find, no tools (?)
● Reference Cycles with __del__– check gc.garbage
– SWIG generates empty __del__ (disable it)
● Missing Py_DECREF (rare)
![Page 21: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/21.jpg)
Thanks
● Code Samples:
http://github.com/martinxyz/python
![Page 22: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/22.jpg)
BACKUP
![Page 23: Mixing Python and C - RMLLschedule2012.rmll.info/IMG/pdf/mixing-python-and-c.pdf · Mixing Python and C 2012-07-10 Martin Renold. Content MyPaint, Python and C ... – GIMP (main](https://reader031.fdocuments.net/reader031/viewer/2022021903/5ba03e3009d3f267388caad2/html5/thumbnails/23.jpg)
NumPy (and SciPy)
from pylab import *
pix = zeros((64, 8, 3), 'uint8')pix[:,:,0] = 255pix[:,:,1] = 128 + 60 * randn(64,8)pix[:,:,2] = 0
imshow(pix)