Hacker News Reader

Comment by eska | Hacker News Reader

I appreciate that you first tried to optimize the original Python code. Idiomatic Python is unfortunately disappointingly slow and not so interesting to compare to.

View on HN

I often use the rough approximation that Python is 40-50x slower than C. This is what you'll see in the benchmarks.

The truly rough thing about Python though is that that is the speed when the code is being written to a benchmark. It is really, really easy to write Python that is multiples slower than that when not writing to a benchmark and just trying to get work done without hyperoptimizing. I did some testing of Python [1] to back some other commentary I was making that compared the time it took to set an attribute repeatedly on a particular instance of an empty class to the time it took to setting it on a subclass of a subclass of a class that had a property setter that was wrapped by a decorator. The latter was about 4.6 time slower than the direct attribute setting, which was itself already ~100x slower than an attribute setting in a static language.

And it's not like a three-deep nested class with a property wrapped by a decorator is all that absurd in Python or anything. That's a completely normal case, not some absurd example I made up to skew the test.

In practice the 40-50x number is more lower bound than what you can count on. If you are actually using Python's features I think you can easily score another order of magnitude slower without anything jumping out at you as being an obviously bad idea.

[1]: https://jerf.org/iri/post/2024/not_about_python_addendum/

A while back I had claude implement something, I don't quite remember what it was, but it chose Python. It was going to take hours. I told it to rewrite it in Rust and it was > 300000x faster. This is without any optimization or prompting particularly about performance, a short one shot lift.

echo "Python sucks, use something else when you can" >> ~/CLAUDE.md

Python was cool in 2005 in academia IT, all the rage in startup 2012. These days...

> 40-50x number is more lower bound [...] easily score another order of magnitude slower

This is about what I observe. I had a utility based on `scapy`; there were no obviously bad ideas in the python source, but porting the work loop into a cpython extension module yielded a 500x speedup.

I feel like scapy is a bit of a special case though. It's unusually dynamic because it allows kind of arbitrarily crazy things. I feel like it's maily best for prototyping. I've had 1000x speedups (100s of milliseconds to dozens of microseconds) just by removing dynamism. In our case we used scapy to work out the packet shapes we cared about and then just implemented just that subset (still in Python!). But that dynamism really helped with the early stages along with wireshark

I've generally taken the position that if you're considering doing micro optimizations in your python code you should just switch to another language instead. The gains from switching language are going to be so much higher than the gains from trying to get the python interpreter to be a little less slow (and empirically, every time I've seen it tried the code has been rewritten in the end anyway).

> not so interesting to compare to

Absolutely disagree here, something that is considered good practice is very interesting to compare to!

I mean that mostly in the sense that there is huge variance in idiomatic code. So your optimized C/Rust code might be 100-1000x faster than two idiomatic versions of writing that code

Reply on news.ycombinator.com