Hacker News Reader

Comment by samwillis | Hacker News Reader

I have used SSEs extensively, I think they are brilliant and massively underused.

The one thing I wish they supported was a binary event data type (mixed in with text events), effectively being able to send in my case image data as an event. The only way to do it currently is as a Base64 string.

View on HN

SSE supports gzip compression, and a gzip-ed base64 is almost as small as the original jpg:

$ ls -l PXL_20210926_231226615.*

-rw-rw-r-- 1 derek derek 8322217 Feb 12 09:20 PXL_20210926_231226615.base64

-rw-rw-r-- 1 derek derek 6296892 Feb 12 09:21 PXL_20210926_231226615.base64.gz

-rw-rw-r-- 1 derek derek 6160600 Oct 3 15:31 PXL_20210926_231226615.jpg

Quite true, however from memory Django doesn’t (or didn’t) support gzip on streaming responses and as we host on Heroku we didn’t want to introduce another http server such as Nginx into the Heroku Dyno.

As an aside, Django with Gevent/Gunicorn does SSE well from our experience.

Send an event that tells the browser to request the binary image.

In my case I was aiming for low latency with a dynamically generated image. To send a url to a saved image, I would have to save it first to a location for the browser to download it form. That would add at least 400ms, probably more.

Ultimately what I did was run an SSE request and long polling image request in parallel, but that wasn’t ideal as I had to coordinate that on the backend.

I'm curious if you could have kept the image in memory (or in Redis) and served it that way

That’s actually not too far from what we do. The image is created by a backend service with communication (queue and responses) to the front end servers via Redis. However rather than saving the image in its entirety to Redis, it’s streamed via it in chunks using LPUSH and BLPOP.

This lets us then stream the image as a steaming http response from the front end, potentially before the jpg has finished being generated on the backend.

So from the SSE we know the url the image is going to be at before it’s ready, and effectively long poll with a ‘new Image()’.

Reply on news.ycombinator.com