I’m shocked that Rust only gave you a 2X improvement over Python (I’ve rewritten a handful of Python services to Go and typically see 100-1000X improvement). What is the bottleneck? Was scaling horizontally an option?
1. backend DB at its query limit
2. this is basically my first rust program and I've done something stupid
3. ab/wrk aren't scaling properly either (I know, not likely, but worth considering) when I'm testing it
4. bandwidth/IO limits on the AWS instance types I'm running this on
Are you using a database? If so, which one? Are you using a cache like Redis?
I agree that a 2x speedup is a terribly low for a Python to Rust rewrite and shows that Python is unlikely to be the relevant bottleneck here.
It's definitely a work in progress and I didn't expect this much interest in my offhand "I tried Rust even though I couldn't see myself using it for real, but..." comment.
I can't imagine the web framework is the bottleneck...
I didn't want to port APM code as well, so I kept node.js/express for routing. Very simple middleware that immediately passed the request and body buffer to Rust for handling. Rust returned a buffer back to express for sending the response.
It's easily able to hit 100k RPM on a single instance before hitting a CPU bottleneck in the Rust code (validating an RSA signature). It only needs to handle 90k RPM total.