Comment by babuskov | Hacker News Reader

Comment by babuskov | Hacker News Reader

Parent Full thread

The "over double digit" claim is simply not true. I'm running three multiplayer games for about two years now, and it starts to have problems once if goes over 3500 concurrent connections. It could be that I just hit the right combination of node+socketio versions, though, as I had problems with newer ones. My setup is:

node.js 0.6 socket.io 0.9.6

This version of socket.io has a nasty bug in jsonp protocol handling which I patched manually by merging some change from a newer version, but it does work fine.

To make it scale, I run multiple node processes. In the last 300+ days each of those has between 1000 and 2000 concurrent connections all the time and it works fine. Except that there is some memory leak (I'm not sure if it's related to socket.io or some other lib), so I restart each server once a week (it's done automatically each Sunday at 2am).

I had my eye on SockJS since 2012, and it looks like newer versions are really good, but I'd need upgrade to newer nodejs and this is simply not possible for the existing projects as many used modules are not available.

However, if you're starting a new project, it's worth testing and see which performs better.

We also do had a memory leak because of socket.io 0.9.6

take a look at this discussion https://github.com/LearnBoost/socket.io/issues/1303#issuecom...

Interesting, but I'm not using Redis nor MemoryStore. I gave each machine behind HAProxy a different hostname (mapped to the same IP address), and I route the clients in roundrobin fashion to each of those before they open the page with socket.io javascript. It works like this:

- client opens a page on HAProxy load-balancer picked server

- the actual node server that receives the request, redirects the browser to its custom hostname using the same URI

- client opens the same page on the particular server and socket.io connects to that subdomain

This ensures that all future requests (if socket.io connection is broken) will go to the same server without Redis, MemoryStore, cookies or whatever.

Drawbacks:

- one additional HTTP redirect when page that contains socket.io is loaded (luckly, in games there aren't many of those)

- if you want to retire some sub-domain hostname, you have to be careful to allow existing clients to re-connect somewhere else. Not really a problem for my setup, but YMMV.

How does this work for non-socket transports? You have to be saving the broadcasts in case someone is polling, right?

I'm not sure what you mean by "saving the broadcasts". Can you give an example?

Some 11% of clients are using XHR or JSONP protocols and it it Just Works(tm), I did not add anything special.

You said you don't use Redis/MemoryStore, but when you broadcast something it has to be saved somewhere until all of the polling sessions get it. Where is it being saved?

There's a memory leak that only got fixed in 0.10.22 I believe. Erin Hammer and the whole Node Black Friday team really publicized it when it was biting them[1]. I'm not sure if the leak was present in 0.6 and given that you rely on older modules the cost/benefit of switching is probably too high.

You're the first person I've heard that was able to run vanilla Socket.io at such scale. I know Trello and some other shops use Socket.io, but they've forked & changed it to something that looks nothing like the original[2].

1 - http://www.joyent.com/blog/walmart-node-js-memory-leak

2 - http://nodeup.com/fortytwo

Reply on news.ycombinator.com