Hacker News Reader

Comment by jgrahamc | Hacker News Reader

We constantly think about the possibilities of using exotic hardware for acceleration. So far we've got a very, very long way with "commodity" hardware and the Linux kernel. One day, we'll probably do something exotic.

https://blog.cloudflare.com/cloudflares-gen-x-servers-for-an...

https://blog.cloudflare.com/tubular-fixing-the-socket-api-wi...

View on HN

I don't think the hardware need be very exotic - most network hardware already has firmware capable of various types of acceleration, and all you would have to do is partner with the network card vendor and have a joint team write firmware that has all the features you need.

With shared source, the network card vendor can even repackage some of the types of acceleration you write for their other clients - and I bet they're already looking into acceleration of HTTP/3.

It's a win-win - because cloudflare gets massive compute savings, and the network card vendor knows they've built quite a high moat to buying someone else's hardware.

Why not using the same kind of solution as Netflix does where they offload TLS and the like in hardware?

https://www.nvidia.com/en-us/networking/ethernet/connectx-6-...

Pretty sure they analyse TLS handshakes for their Bot Management product and probably other things.

Have you tried out io_uring? My impression is that these Rust libs end up making different syscalls that are less great.

Tokio supports io_uring (https://github.com/tokio-rs/tokio-uring), so perhaps when it's mature and battle-tested, it'd be easier to transition to it if Cloudflare aren't using it already.

Don't assume io_uring is an automatic win.

Last time I benchmarked io_uring, it was good enough, but I was able to get measurably faster from syscalls and a well-crafted userspace thread pool for storage to NVMe.

(Note: the I/O thread pool I wrote came out noticably faster than fio, the benchmark tool, so don't assume fio results are the best possible.)

For that reason, I have a small application which can be configured for either io_uring or the thread pool, and the thread pool is preferred for performance.

Just curious, is there a high cost overhead (aside from the cost of actual units) of e.g. custom silicon?

It’s just a question of “is it worth it?”. How much does this cool accelerator thing cost? How much does it save?

Reply on news.ycombinator.com