Instrumenting HTTP requests in Node

Timeouts in Node.js HTTP requests I think are one of the most confusing aspects of all of Node.js, in terms of technical details that are critical to successful production operations. The core documentation is poorly worded in some places, and some of the helper libraries make it very difficult to figure out exactly what timeouts are being set. For example, node-fetch (mentioned in the article) had/has a pretty non-sense timeout on the entire request duration [0], and I've seen that blindly propagated into other libraries [1]. When timeouts only happen under error conditions (flaky servers), it's a nightmare to debug.

Glad to see this article explain the various timeouts (although I don't see the idle socket timeout mentioned). Huge +1 from me for using node's core http library only and adding explicit, readable timeouts in your own code.

[0] https://github.com/bitinn/node-fetch/issues/446

[1] https://github.com/googleapis/nodejs-storage/issues/27 (two different timeout bugs)

Very nice article. Valuable conclusion as well.

Somebody can think that almost every nodejs developer knows what's in https://nodejs.org/api/index.html - but I don't think that's actually true. One of the better tips on how to be a better nodejs developer is to simply read it from top to bottom. It's really not that long and gives valuable insights.

Another insight, which I'm exploring recently in node projects, that is somehow related to this, is "debloating" node npms/projects. After looking at dependency tree used by some of my own npms/projects I was surprised (after few seconds of dependencies recursively unfolding themselves), then shocked (even well before they fully resolved).

There is something good about many npms being available but there also seems to be something wrong about how dependency tree spreads out - in many cases duplicating functionality, bringing completly unnecessary dependencies and their friends and a lot of "garbage" in general (ie. thousands of test files/fixtures/what not are not necessary but they come to the node_modules party anyway) etc.

I'm not trying to say that `npm i` is bad and should be avoided because that would be nonsense - but I'm trying to say that being concious about what it actually does in your npm/project is something valuable, especially for npm authors where those decisions will have multipled effects.

Another option is to run the requests through a sidecar process so you have one place for implementing these best practices across your stack.