Hacker News Reader

2M qps in production is legit. Curious how much RAM and CPU that takes on average per deployment though

Depends. Only pooling, very little. Load balancing/sharding needs to parse queries, so a bit more. Could go up to a GB per pod, sometimes more if you have a lot of unique SQL queries (unique by text, not by parameters). We cache query ASTs to avoid parsing them on each request - that's the bulk of memory usage.

Semi related question - I have always wondered, how do you tackle OOM issues at the proxy layer, i.e. let's say a particular SQL query requires proxy to fan out the query to multiple shards, which return a pretty large dataset. I'm assuming you would need to load this dataset in the ram to perform certain operations. What happens if the resulting dataset causes the proxy pod to go OOM?

Two schools of thought:

1. Let it crash. Increase the RAM, try again.

2. Page to disk (swap), make it slow but ultimately work.

Both have their trade-offs. There is no free lunch here.

Reply on news.ycombinator.com