Hacker News Reader

Comment by babuskov | Hacker News Reader

I use Node in production. The main thing I like about it is that looking at system usage graphs while number of users grow, only thing that is going UP is bandwidth ;)

I'd really like to see a story of someone really having 100k connected browsers. My online game currently peaks at about 1000 concurrent connections, and node process rarely lasts longer than 2 hours before it crashes. Of course, using a db like Redis to keep users sessions makes the problem almost invisible to users, as restart is instantaneous. I'm using socket.io, express, crypto module, etc.

I'd really like to see real figures for node process uptime from someone having 5000+ concurrent connections.

View on HN

I'm using C# for my game Tribal Hero (www.tribalhero.com). It's still in early beta so I've only had 450 concurrent users . Our CPU usage and memory usage barely moved from 0 to 450 users. We're using socket selects and not even async sockets which would have even better performance. It's also backed by MySQL though we want to eventually move to Redis. Why is Node breaking at 1k connections? Doesn't seem like much at all.

I also use MySQL as backlog, it's practically write-only as I keep the whole state in javascript objects. Only time when data is read from MySQL is at program startup. However, having SQL database enables me to run various complex SQL queries for reporting.

However, I do use Redis for one thing: user sessions. I turned persistence off as Redis seems to be rock-stable, and I really don't need sessions to persist. I was using a modified version of Node's MemoryStore, to which I added clean garbage collection, but with often restarts I mentioned earlier it has become pain for users to have to login again when in the middle of the game. Having a separate, dedicated Redis instance to handle the sessions made restarts completely seamless, as the cookie sent by user's browser remains valid between node restarts.

I was not willing to learn new db technology, but there wasn't really much to learn with Redis. You can set it up in minutes and it just works(tm). I highly recommend you try it.

I'll be switching over the entire game state to Redis. It'll be a bit of work but what I like the most is that it maps more naturally to objects. My db is mainly writes as well.

I do about 300,000 - 500,000 concurrent connections on nodejs but it's all short lived web requests.

It took a while to iron out most cases that can crash, right now I have:

web.1: up for 12h

web.2: up for 12h

web.3: up for 12h

web.4: up for 12h

web.5: up for 12h

web.6: up for 4h

web.7: up for 1h

web.8: up for 12h

web.9: up for 12h

web.10: up for 12h

web.11: up for 32m

web.12: up for 12h

web.13: up for 7h

web.14: up for 7h

Interesting. What's the system's uptime? Close to 12h or rather not? If the latter, this still means 1-2 restarts per day.

Regarding crashes, do you know of any special things to look out for? I do crash dumps and log uncaught exceptions, but sometimes node simply dies without any trace in the log files.

The highest I've seen on a dyno (this is all on Heroku) is over a day, web #13 and #14 are on 22 hours now. I think they're actually eventually running out of memory or being retired and replaced by Heroku rather than crashing but I'm not sure, it's not being caught in the exception catching.

Most of the crashes come down to stupid things, it's so easy to make a mistake when you don't have a compiler watching your back. External dependencies can hurt if they're laggy or unavailable. Unterminated requests are a really easy accident as well.

At this point I just use exception catching and dump the results into Redis unless I'm specifically hunting down a bug and want the crash to occur:

http://api.playtomic.com/load.html

Wow, I like your real-time monitors :) I'm about to build mine, and this gives me some great ideas.

Thanks for sharing.

I could never get my socket.io instance to max out, is there a good way to load test socket.io and web sockets?

Reply on news.ycombinator.com