Hacker News Reader

Comment by tech2 | Hacker News Reader

I kinda felt similarly until I found one of my existing web services wasn't quite fast enough (currently serving around 1000req/s at peak). I could either go for a third rewrite in yet another framework hoping to increase performance by maybe 30%, or I could try something entirely new.

First attempt with rust more than doubled performance of the existing system on the same hardware. Now I feel a need to learn more.

View on HN

What are you serving? Some of the Rust frameworks for backend should be able to do more than 1000req/s, but it depends what you're doing.

The system's peak _usage_ is 60kreq/min (avg 1k/s), the Python code it's written in can handle maybe 850/s per instance at 100% CPU.

Rust is fine, rust is managing almost 2k/s on the same hardware. And this is when I know practically nothing about Rust, I expect I'm likely able to do better given time.

It's acting as a simple filter and cache over JSON data on Postgres.

Any reason you didn't scale up horizontally with more python instances?

We have seen production users choose to make this port to save on server costs. You can always scale horizontally, but it comes with a price tag.

Also complexity - managing an order of magnitude more instances is more hassle.

But "more hassle" by an order of magnitude that gradualy reduces when growing the number of instances.

But the cost keeps growing, especially when you need to go from 50 servers to 100

The OP only doubled their performance by rewriting in Rust though...

And depending on the amount of code, that's an awesome trade-off if OP can just have one instance instead of managing a cluster.

From what OP has said, there's probably a good change they can get more performance with some optimization.

You still need redundancy, so you’re operating multiple instances either way (or you’re playing fast and loose).

I have multiple instances to keep the latency down overall (and for security should we lose an availability zone or whatever else). But, as others have mentioned, instances cost, I'd like to be able to reduce that to the minimum if possible.

I’m shocked that Rust only gave you a 2X improvement over Python (I’ve rewritten a handful of Python services to Go and typically see 100-1000X improvement). What is the bottleneck? Was scaling horizontally an option?

I suspect the reason it's not faster is one of the following:

  1. backend DB at its query limit
  2. this is basically my first rust program and I've done something stupid
  3. ab/wrk aren't scaling properly either (I know, not likely, but worth considering) when I'm testing it
  4. bandwidth/IO limits on the AWS instance types I'm running this on

There are other possibilities too I guess. I was mostly doing this as a means to learn Rust though, I'd only go live with it once I had decent tests and everything else.

What web framework are you using?

Are you using a database? If so, which one? Are you using a cache like Redis?

I agree that a 2x speedup is a terribly low for a Python to Rust rewrite and shows that Python is unlikely to be the relevant bottleneck here.

actix/r2d2 on the Rust side (I haven't worked out bb8 yet), aiohttp/uvloop/asyncpg/gunicorn on the Python side (Postgres for the DB). This kinda _is_ the cache at present, it's a distillation of a lot of other data into a given form. Rust's async story re: databases still looks a little up-in-the-air right now from an outside perspective though, so I guess I'll wait and see how it pans out.

It's definitely a work in progress and I didn't expect this much interest in my offhand "I tried Rust even though I couldn't see myself using it for real, but..." comment.

> What web framework are you using?

I can't imagine the web framework is the bottleneck...

It might be because if it's a tokio-core based one OP might be blocking the event loop too much.

Interestingly, I was able to get an amazing speed improvement without going completely to Rust.

I didn't want to port APM code as well, so I kept node.js/express for routing. Very simple middleware that immediately passed the request and body buffer to Rust for handling. Rust returned a buffer back to express for sending the response.

It's easily able to hit 100k RPM on a single instance before hitting a CPU bottleneck in the Rust code (validating an RSA signature). It only needs to handle 90k RPM total.

Reply on news.ycombinator.com