I'm trying to work out what you'd want this for. I suppose as a quick "yes this does actually work" but I can't think of many less efficient ways of spidering things - whole headless browsers? Ouch.

If you needed to do this, you'd use something that already exists - if you just wanted to do it in Node, for some particular reason, why not just use plain requests and an HTTP parser?

I don't mean this to be overly critical, but just in case anyone's thinking this would be a good idea - there are better ones. Just to learn from, fair enough.