Guido van Rossum: patterns for fast Python

plus.google.com

34 points·joeyespo··5 comments

Discussion from 2 weeks ago: http://news.ycombinator.com/item?id=4648275

I'm coming from the scientific computing side of python, so beyond the BDFL's suggestions, I use the following protocol to make python fast. Note, that I'm doing numerical analysis and manipulations, so this isn't a general workflow for all python applications.

1. Write algorithms using numpy, attempting to not use too many crazy indexing/array tricks. You can generally avoid costly for loops using broadcasting.

2. Profile code using cProfile and Robert Kern's line_profiler

3. Judiciously optimize by re-writing computationally intensive methods in Cython. Numba also looks like a contender in this arena as well, but I haven't done a benchmarked comparison.

Can you suggest any resources for learning about profiling in python? I'm just getting into scientific computing with numpy,scipy,etc. and I'd be interested to learn more.

I used timeit (http://docs.python.org/2/library/timeit.html), cProfile (http://docs.python.org/2/library/profile.html) and pycallgraph. There is a nice Q&A on this subject: http://stackoverflow.com/q/582336/343834.

Ian Ozsvald has a nice tutorial on high performance python, from profiling through cython and some multiprocessing and parallel techniques. You can find video and other materials here: http://ianozsvald.com/2012/03/18/high-performance-python-1-f...

Reply on news.ycombinator.com