Hacker News Reader

Comment by IshKebab | Hacker News Reader

Yes but his point is that if a 10% performance improvement is significant then you shouldn't be using Python!

That's a very naive statement. In machine learning, for example, there's a lot of scenarios where performance isn't critical but due to the huge quantities of data you may need to process (especially during feature engineering or selection), micro optimisations like this can result in huge improvements over millions of iterations.

Not to mention Python has Cython and numba's JIT which can provide great performance when needed.

To me, it seems like there is potentially quite a good business case for writing

    l = [None] * 10_000

instead of rewriting parts of your application in C (possibly hiring new staff to do so).

Very short term maybe. Medium term you're just putting off work and building up technical debt.

I was actually waiting for this comment to pop up. Unfortunately, it's not so black and white as it seems. The service in question is deeply coupled with a Django monolith and rewriting that will be mostly a waste of time for a medium scale business like us.

However, micro-optimizations and effective refactoring quickly add up and for us, improving the service response by 100ms via micro-optimization is better than improving it by 200ms via rewriting the entire service.

200ms -> 100ms for a whole request (not just a tiny part of a request) is indeed worthwhile, I never meant to disagree with that. But for micro-optimisations, you'd be lucky to get a 10% speed increase in 10% of things you do, which would only give only a speed increase of 1% overall (obviously those are made up numbers to illustrate the idea). As I said in my first comment, usually the actual gains come from using a different data structures or making vectorised calls in to C libraries, and I assume that's what you mean by the effective refactorings in your case.

Sure, but if you can easily get that 10% improvement in your existing code, you probably should take advantage of the easy win. Even if you're eventually going to rewrite all or part of it in another language.

Reply on news.ycombinator.com