Hacker News Reader

Comment by ww520 | Hacker News Reader

I don't understand why he has to stop the project just because ONE node.js process cannot handle more than 140K concurrent connections. If the goal is to handle millions of concurrent connections, spreading the connection load out to multiple servers is a must.

I'm doing something similar with node.js now to handle millions of concurrent connections. I use multiple node.js servers to handle it. Not only it helps in even out the load, it has the nice failover property. One node down doesn't mean the whole app is down. Clients of the down node migrate their connections to the rest of the nodes.

View on HN

If you have 140M active users, handling a fleet of 1000 servers is not an easy operation, considering you need to manage inter-cluster communications (i.e. when sender and receiver sockets fell on different servers).

I've built Erlang/OTP websockets clustered server, which can handle 3M per node (giving you have enough RAM). Here you handle all your users with "only" 47 servers.

There is a big operational and scaling difference between cluster of 1000 servers and cluster of 47 servers.

I'm a bit curious about this. If you have 140M concurrent users (since this was the original complaint) and you are NOT prepared or capable of servicing/monitoring/maintaining 1000 servers, that seems like a fatal flaw in your server management and analytics processes.

Certainly 47 servers vs. 1000 is far nicer. But at 140M concurrent users levels (there must only be a few handfuls of sites with these types of concerns), not having a team prepared to oversee 1000 servers seems like folly.

The problem is not only operational, but also a technical one:

Nodes in the cluster need to communicate with each other and with other systems, like databases, message queues, monitoring servers, etc.

You can aggregate data per node, so the less servers in the front-end cluster you have, the less load on the back-end servers.

There is also financial problem: many organizations can afford 50 servers. But not many can afford 1000 servers.

There's no reason why you can't have many node processes per server. This also raises the fault tolerance per server.

Facebook chat has only 38 gateway servers. I guess, that only possible b/c not every active facebook user, uses Chat.

Perhaps they should consider adding more, it's permanently screwed up in some way. Messages not being sent or the other person not receiving them.

You're spot on. I think a lot of people can't wrap their head around node: It's all about scaling horizontally (lots of processes, many servers) rather than vertically (one process with lots of threads on one big server). Once you realise that, it makes a lot of sense.

Exactly. I pick Node.js not because one process can handle lots of connections, but because one process can handle lots of connections with low resource utilization. It needs less servers to get the job done. It's more of a cost-based decision.

Reply on news.ycombinator.com