Hacker News Reader

Comment by ericvolp12 | Hacker News Reader

Jetstream isn't an official change to the Protocol, it's an optimization I made for my own services that I realized a lot of other devs would appreciate. The major driving force behind it was both the bandwidth savings but also making the Firehose a lot easier to use for devs that aren't familiar with AT Proto and MSTs. Jetstream is a much more approachable way for people to dip their toe into my favorite part of AT Proto: the public event stream.

View on HN

As I understand article, there are new API (unofficial , for now)

- Jetstream(1) (no compression) and

- Jetstream(2) (zstd compression).

And my comment means (1) not really needed, except some specific scenarios

It's impossible to use the compressed version of the stream without using a client that has the baked-in ZSTD dictionary. This is a usability issue for folks using languages without a Jetstream client who just want to consume the websocket as JSON. It also makes things like using websocat and unix pipes to build some kind of automation a lot harder (though probably not impossible).

FWIW the default mode is uncompressed unless the client explicitly requests compression with a custom header. I tried using per-message-deflate but the support for it in the websocket libraries I was using was very poor and it has the same problem as streaming compression in terms of CPU usage on the Jetstream server.

> It also makes things like using websocat and unix pipes to build some kind of automation a lot harder

Would anybody realistically be using those tools with this volume of data, for anything but testing?

It's not that much data. Certainly nothing zstdcat can't handle.

Reply on news.ycombinator.com