Hacker News Reader

I find the month to minute transformation rather shocking. Are you sure you weren't doing something algorithmicly wrong in python? Did you profile it? Like I would never leave something running for a month without putting a fair bit of time into making sure it was fast.

View on HN

I don't think I did anything algorithmically wrong. If I could have done it with more of the data handling being done in c functions and just using Python as outer glue? Most likely but then that would not make the framework very extensible to other models.

Not only did I profile it, put a lot of time into optimization and learned how to use cloud computing, but I also learned a new language (Julia) and ported the whole application. :)

In Julia I can use things as data oriented design (few cache misses) to further reduce runtime. I'm not sure you get much out of such efforts in Python.

This. Having tight control over memory layout + heap allocations + vector instructions can literally decrease runtume by several orders of magnitude. Performance is so much more than having the "right algorithm". The low-level details matter so much.

Reply on news.ycombinator.com