Optimize Python with closures
tech.magnetic.com
tech.magnetic.com
Also, maybe it would be worth discussing building temporary function variables in outer scope that resolve the methods before the closure is called (i.e. my_list_append = my_list.append). There was no opportunity to do this in example, but it's worth discussing.
Finally, dict access seems like a bad idea if you're this pressed for performance, especially since you only need one value -- maybe better just to encourage explicitly passing in a parameter.
(And as they care about performance, and have tested PyPy, they are hopefully using it, and so the class-based solution is just as fast as the closure-based one, and is going to allow for better decoupling of filter modalities; though, as someone who does a lot of functional programming, the closure-based approach does resonate with me.)
In particular, I wanted all 3 examples to use nearly identical inside the filter function, to isolate the differences just between the ways of accessing data in the benchmark results, and to show that closures are an easy way to gain some performance in hot spots (in CPython at least).
The API in the example is really bad. Accepting a dictionary, only to require a specific key in the dictionary, is the worst of both worlds. But then if you use .get to access the dictionary body instead of the attribute access, you'll take on additional performance penalties, and other solutions will start to compete.
I tend to try and do that in any language as soon as a single conditional means a change in behaviour in different locations, regardless of optimization concerns. It makes for (IMHO) more robust code and much easier testing (though you need more mocks).
Optimizing pure Python is a waste of time. You end up with weird, unidiomatic code that it takes ages to come up with, because you're fighting the language. And in the end you hit a wall: a point at which the code can't go any faster.
If you just write Cython, you can easily reason about what the code is doing, and how you should write it to be more performant. Ultimately you can make the code run as fast as C, if necessary.
"At Magnetic, we’ve switched from running all of our performance-sensitive Python code from CPython to PyPy, so let’s consider whether these code optimizations are still appropriate... PyPy is incredibly fast, around an order of magnitude faster than CPython in this benchmark. Because PyPy performs more advanced optimizations than CPython, including many optimizations for classes and methods, the timings for the class vs. closure implementations are a statistical tie."
The fact that that method accessors are much slower than properties, or that closures can speed things up, is something that we might not always be mindful of, and that new python programmers might not think through.
our application handles about 300,000 requests
per second at peak volumes, and responds in
under 10 milliseconds
Spot checking, you've done better than you think :)
Exciting to see python implementation doing this!
The bottleneck is usually DB work, so PyPy is not a great help there.
Bottomline: Number of requests (scalability) and response times (performance) are more a work of architecture than using a particular language. And Python can be actually quite fast (fast enough)
Where I get really frustrated with the 90s-era dynamic scripting languages is not when you are "filtering one list" or "serving JSON out of the DB", but when you're trying to do all kinds of CPU-type work in one page, with numerous (unavoidable) DB accesses and filtering and transforming etc etc. You can get into "seconds" surprisingly quickly, since it is, indeed, a fairly slow language.
It is likely that rewriting this code in C++ or something could get another 5-10x out of it even over PyPy, probably even more if we really start going crazy with optimizations, but at much larger investment of developer time. By contrast, twiddling with a couple of details and switching to PyPy is the sort of thing you can prototype in an afternoon and deploy in a week or two, even with solid testing procedures. At scale that's probably worth it (using 5-10 times fewer machines is generally a significant cost savings even in this "cloud" era), but they may not be to that scale yet. Perhaps they never will be. Who knows? Not me.
def whitelist_filter(bid_request):
return not categories.isdisjoint(bid_request["categories"])
def whitelist_filter(bid_request, categories=categories):
return not categories.isdisjoint(bid_request["categories"])Locals are the fastest lookup in cpython (by a fair bit), and since default parameters create local variables and are bound once at function creation, you can use them to aliase both nonlocals and globals in order to improve their lookup time significantly:
def func1():
a = 42
def func2(param, a=a, bool=bool):
bool(param + a)
return func2