Hacker News Reader

The big problem i've run into with SSE is the connection limit [1]:

> Warning: When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be specially painful when opening various tabs as the limit is per browser and set to a very low number (6). The issue has been marked as "Won't fix" in Chrome and Firefox. This limit is per browser + domain, so that means that you can open 6 SSE connections across all of the tabs to www.example1.com and another 6 SSE connections to www.example2.com. (from Stackoverflow). When using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100).

The natural way to use SSE is to create an EventSource on each page. If you do this, and your user opens six tabs, they can now no longer make HTTP requests to your site. Not just SSE requests, any HTTP requests at all!

It's also quite natural, i think, to want to use multiple separate streams. It's very convenient to write separate endpoints for various kinds of data that a page might want, for example one for streaming numerical data, one for control messages, and one for descriptive events; or one for each panel on a dashboard. Here, you can hit the limit on a single page.

I wrote a thin layer over my HTTP server's APIs which lets me write a single handler which can send events via either SSE or a websocket. Then, when i build a page, i use websockets, but if i want to debug, i can still use curl to make a normal HTTP request. Also, other apps can use SSE to pull data, which only requires an HTTP client, not a websocket client.

[1] https://developer.mozilla.org/en-US/docs/Web/API/EventSource

> It's also quite natural, i think, to want to use multiple separate streams.

i dont think i'd call it natural per se - i'd say it's just "easy" to do so, and especially if each of these separate streams are managed by different teams, and it makes business sense to separate the responsibility.

There should be a framework, perhaps, to allow "multiplexing" of multiple streams of events into one actual tcp connection. Of course, this requires some work, but i dont think it's actually that much extra work. It just needed to be planned out as a need, as it might be a tad hard to tack it on after the fact (tho that's not impossible either).

> The natural way to use SSE is to create an EventSource on each page. If you do this, and your user opens six tabs, they can now no longer make HTTP requests to your site. Not just SSE requests, any HTTP requests at all!

You can use EventSource in a singleton worker and use postMessage to forward messages to all open browser tabs for the same origin.

The original comment stands: it's harder.

SharedWorker (the thing needed for a "singleton worker") only just made it into Safari the end of last year...

plus you now have to handle multiplexing and demultiplexing for different tabs on both server and client.

By the time that arrived and HTTP/2 had wide enough adoption, everyone's already used to building atop Websocket...

Also the EventSource in the browser is heavily gimped. GET only, No custom headers, and last I checked you can forget about configuring any CORS stuff for credential passing.

Not sure if that's what you are asking but there's a `withCredentials` option (which IIRC is the only possible option)

I was looking at : https://stackoverflow.com/questions/61041389/managing-server...

Shared worker would actually allow for SSE in a service worker ! That's nice.

EventSource is bad. We were using it to connect to GPT 4 via our server acting as a middleware (we hosted the code for returning the LLM response on our server since our company's OpenAI API key is supposed to be private).

However, previous developers had written a basic middleware to check if a req.body exists and if it contains a userId (since our server was handling multipler other things apart from this one component of the business logic).

However, any such manipulation of the request body isn't possible with EventSource (and I felt that using a workaround such as passing the userId in the querystring could cause some security issue), so I ended up using the fetch() method to connect to our server for interacting with GPT 4.

This SO answer seems to say that it wouldn't work : https://stackoverflow.com/questions/61041389/managing-server...

If you have an answer to that answer I'd be very much interested

You can use BroadcastChannel [1] to communicate between tabs. It works well and does not require service workers. It's used in Tracim [2], for some reason we could not move to HTTP/2 at the time I was working on this project so we indeed learned about this connection limit soon enough…

Even on HTTP/2, I believe it's best avoid sending 6 times the same message to the user just because they opened 6 tabs. It's less wasteful both for the server and the client.

[1] https://developer.mozilla.org/en-US/docs/Web/API/BroadcastCh...

[2] https://github.com/tracim/tracim/blob/develop/frontend_lib/s...

Thanks very much for the examples ! I get your point but it does seem quite more involved to do all that management and broadcasting by hand rather than having a shared worker with a EventSource singleton for all, don't you think ? : https://stackoverflow.com/a/61636606/10469162

EDIT: Someone posted this very relevant and interesting post : https://greenvitriol.com/posts/browser-leader

Just create more subdomains if you really need more sockets.

Or move away from the browser. Browsers are toxic now.

In short, it reads to me as "HTTP/2 or don't bother".

Not an unreasonable requirement for a serious modern web app.

Yes, but weirdly enough the most popular javascript for handling http, express, does not support HTTP2 ! Fastify does

One major issue I encountered with SSE is dealing with reconnection. WebSocket makes it very easy to detect differences between a loss of network connectivity, a server dying, and a client exiting properly (eg: closing the tab), on either side of the socket.

Maybe there are facilities to do so with SSE that I don't know of, but my experience in reliable server-to-client-only comms with it has been a bit rough.

You pretty much just have to listen for and handle premature closing of the underlying TCP connection. It's the only form of dead peer detection you'll get if someone just abrubtly closes their browser tab.

Do server frameworks really require you to think of the TCP connection?

High performance servers and frameworks will always have to think about the underlying sockets

Not TCP specifically, but ASP.NET Core binds your controller/action methods' CancellationToken parameter (if present) to HttpContext.RequestAborted - so assuming one uses it correctly (e.g. tying it to things like SQL TRANSACTION/ROLLBACK) then that helps prevent inadvertent changes if the user clicked their browser's Stop button after a POST, or gracefully handle long file upload failures, and so on.

This also matters for load balancers. With SSE or chunked encoding long polling, it's possible to lose the client connection without the load balancer closing the connection to the server, at least for a while.

WebSockets make it easy to do periodic heartbeats from the server, which puts an upper bound on how long a client connection can be in an uncertain state.

This also enables client presence data, even if the client only ever receives messages.

I don't think I understand. Both WebSockets and server-sent events are just a long lived TCP socket, right? Why would SSE be lower overhead? Why wouldn't it be more overhead to go through the whole HTTP request handling machinery for EVRY chat message everyone sends? You already have an underlying 2-way connection in the TCP socket, why not use it?

And why would latency be lower with HTTP requests + SSE than with websockets?

It's not. SSE has higher overhead and there are browser limits as to how many concurrent channels a single client can consume. Also, SSE encourages you to shift complexity to the back end where it would be more appropriate on the front end.

For example, the client knows which channels it needs, so why not let the client decide which channels to subscribe to over a single WebSocket connection? With SSE, you end up having to do this channel management and keep track of which user is associated with what SSE connections on the back end since the user cannot just open any number of new channels on-demand (due to limits in number of concurrent SSE connections; it's better to multiplex over a single SSE connection). If your system runs on multiple processes or hosts, figuring out which process/host a particular user is connected to is a nightmare and forces you to create clients on the back end to aggregate messages for specific users and then publish to the correct SSE connection which uses extra memory and exposes your server to potential DoS vulnerabilities. It creates the need for a bunch of complex user-connection-matching and cleanup logic on the back end and it complicates access control.

I have no idea why this "let's micromanage data streams for each user individually on the back end" approach became so popular while the alternative approach of end-to-end pub/sub is rarely used when it results in a much simpler, more secure system.

With end-to-end pub/sub, you can multiplex multiple channels over a single bidirectional connection in a way which conveniently and cleanly allows you to couple multiple subscriptions over that single WebSocket connection... No complex cleanup on the back end on disconnection, no risk of stale channels on the back end, no need to poll for user activity to check for subscription liveness. As soon as the WebSocket is disconnected, the client can be fully unsubscribed from all their channels on the server-side.

In NestJs it is quite easy to manage in the backend, there's a SSE decorator for you SSE service and then you just return an observable on a subject. (See RxJs) Sending message events sends to all observers on the subject and when the `observed` property` becomes false, you just close the connection.

Slack does use websocket for “someone is typing” event: https://slack.engineering/real-time-messaging/

What was the challenge in typing indicators? My naive idea is that it's basically sending `typing_started` and `typing_ended` messages.

I'm guessing that since it's not a full-duplex connection you have to send those messages out-of-band, ie, with a separate POST request API

But messages themselves are already sent out-of-band as a POST request, if I understand correctly. Don't see why that couldn't be extended to include typing indicators.

You can skip the "typing ended". You're gonna need a shortish timeout anyway, may as well just let the "typing is happening" messages all terminate by timeout.

That really depends on the UX you want.

What do you have in mind? I can't think of a case where you'd want "X is typing" to persist more than a second or maybe two past the last-received message that affirmed X is, in fact, still typing. I can think of cases in which a short timeout might break down in bad ways (making the notice really jittery on high-latency connections, say) but I think those are probably just situations in which any "... is typing" message is simply gonna be bad in one way or another.

You don't want to clear it after only a second. It should persist much longer since people pause and think between words.

It's common that someone starts typing but immediately change their mind and clears the input after only a couple of characters. So if the timeout is longer you need an "ended" event.

(the "ended" event is the message being sent.)

Except for the scenario the parent is talking about:

> It's common that someone starts typing but immediately change their mind and clears the input after only a couple of characters.

I used both WebSockets and SSE. In one project, I even support WebSockets and a fallback working like what you describe.

I'm sold on using SSE for pushed updates. But how is regular requests + SSE easier on the server compared to WebSockets? It seems like an open connection to maintain in both cases plus additional http request for the SSE+requests case.

You need to multi-thread your server on shared memory to build many-to-many solutions. The simplest way to get that working is Java + NIO with concurrent package, I have done it for you: http://github.com/tinspin/rupy

My web server outperforms everything on the planet for multiplayer.

Websockets are notoriously hard to parallelize if you want shared memory which you need for many-to-many, mostly because JavaScript and C++ have very poor atomic concurrency, or atleast I never found any Websocket server that could do it.

> You need to multi-thread your server on shared memory to build many-to-many solutions

Not necessarily, it depends on the needs, but in any case I would not expect SSE and WebSockets to be different in this regard.

I'm not sure what's specifically hard to parallelize with WebSockets. Shared memory may be hard to handle but that does not seem specific to WebSockets which are just regular two way communication channels that happen to start with an HTTP request from the client.

Websockets use one TCP socket for starters with SE you use separate sockets from ingress and egress which is much simpler to scale from an infrastructural point of view.

I clearly said many-to-many, everything else is trivial to scale because only many-to-many requires strictly shared memory atomic parallelization.

You could write a better server in Java than javascript/c(++) but I haven't found any, and to be honest the extra complication of the binary protocol doesn't even make me curious enough to waste 5 seconds to google it.

SSE over HTTP/1.1 is the final solution. Everything else is a waste of time for eternity.