Comment by majke | Hacker News Reader

Comment by majke | Hacker News Reader

Parent Full thread

> First, what does it mean to "slow down the pace" of accepting connections?

In C, if you don't call 'accept' new connections will:

- first: get queued in a kernel queue of SYN+ACKed connections

- second: when queue is full Linux stops sending SYN+ACK, thus forcing the client to re-send SYN.

Not calling 'accept' pushes pressure back to the clients, without sending RST, without introducing any state on the server.

> and then immediately drop new connections as they're made

That's not the same as pushing back the pressure. Dropping connections means: "go away, I'm broken". Not calling 'accept' means: "I'm busy right now, wait".

> And what does c.write being non-blocking have to do with crashing the server?

If 'write' is non-blocking _and_ always succeeds (as in Node) the node.js process may be asked to 'write' plenty of data. As it can't send it to the kernel, it'll get buffered in the node process, and eventually run out of memory. This is the case if you don't have any flow-control mechanisms: you'll accept all the requests and buffer in memory all the responses, possibly leading to a crash.

> but even if you're doing everything synchronously, you've still got to manage all that stuff.

Yes indeed! But at least it's obvious that you're spawning too many threads, or that all the threads are waiting on particular slow thing (disk for example). The state still need to be handled, but it's much easier to reason about it! (it's easier to count threads / coroutines than to count how many times a callback X was called and haven't yet called back the next callback on the path)

> Maybe someone can enlighten me?

I wanted to avoid the discussion about credit based flow control. It's possible to write flow-control aware code using callbacks, but it's generally harder. And requires a programmer to be aware of the problem.

Also, IMO flow control in callbacks style introduce spaghetti, as after the last thing ('write') succeeds you need to inform the first thing ('read') to proceed. But that's aesthetics and another discussion.

Thanks for replying. I still don't ultimately see that much of a difference -- if the Node server wants to accept the connections in a queue instead of disconnecting them (but not turn on echoing yet), that's still pretty easy to do with a couple of lines of code.

And looking up socket.write, Node actually tells you with the return value, if the data was written immediately or queued in user memory, plus there's a callback when it gets written. So you certainly do have flow-control mechanisms.

In the end, maybe it takes a few more lines of code with Node (since you have to take care of the queues yourself), but you're also getting a lot of flexibility in handling things exactly the way you want, it seems.

> if the Node server wants to accept the connections in a queue instead of disconnecting them, that's still pretty easy to do

But then the queued connections are using file descriptors, and this is a limited resource, right? In real world sometimes not being able to handle all the traffic is an error condition, and disconnecting the client is okay. In other circumstances you'd like to be able to slow down the clients if the server can't cope. I claim this particular API in node can't express the latter.

> So you certainly do have flow-control mechanisms.

Yes you do! And Streams API is a great attempt to unify flow control mechanisms. The problem is: those mechanisms are a secondary citizen, invented as node matured. And in fact, they are pretty complex, see the documentation of Streams. I think most node.js users can't be bothered with this - and that's the point: flow control requires thinking in node, and you get it for free if using "blocking" coroutines / threads.

> maybe it takes a few more lines of code with Node

Agreed. A decent programmer can write a good program in any paradigm. But I claim that callback API's require more thought to use correctly (due to flow control). Thus, most users will write poor programs using callbacks.

File descriptors and RAM for the buffers.

> Not calling 'accept' pushes pressure back to the clients

To stop calling accept() you could remove or pause READ event for server's FD from event loop once the limit is reached and unpause once it goes down. Not a big deal.

> If 'write' is non-blocking _and_ always succeeds

Why would it always succeed? Obviously it should call back on completion.

Combined with node-toobusy, pausing READ events sounds perfect, but I can't figure out how to do that. Can you explain?

Sorry, don't know much about nodejs. But it seems that node does have a way to remove/setup events, and pause read events: http://nodejs.org/api/events.html#events_emitter_removeliste... http://nodejs.org/api/net.html#net_socket_pause

Ahh ok. I thought you meant at the OS level.

Well, kind of. Pause should instruct event loop to stop waiting for read events from the OS for particular fd, i.e. remove from epoll/kqueue, so new connections would stuck in backlog at the OS level.

I wonder if you could handle backpressure this way, somewhat analogously to a thread-per-connection with a bounded number of threads. I don't think node.js API exposes the necessary functionality, but the point is that it's possible with callbacks:

  var MAX = 100,
      connections = 0;

  function continue() {
    server.once('connection', function(c) {
      if (++connections < MAX) continue();
      process(c, function() {
        if (--connections == MAX - 1) continue();
      });
    });
  }

  continue();

Node.js doesn't have threads so you wouldn't need atomics or locking. I don't think it's fair to say this is spaghetti, but it has some complexity, yes.

'write' could also transparently fail (not that that is usually advisable).

Reply on news.ycombinator.com