I’m shocked that a 256 core Epyc can’t do millions of requests per second at a minimum. Is it limited by the net connection or is there still this much inefficiency?
[1]: https://www.amd.com/content/dam/amd/en/documents/products/et...
There's probably going to be some overhead, but it seems like you could do 1M, if you have the bandwidth.