Personally I prefer that build artifacts end up somewhere under my current working directory. That means if I want to clear everything out related to a project, it's easy to do so without interfering with anything else.
Yes, as frustrating as it is to need on occassion to rm -rf /node_modules and start again to fix things, at least that's an option.
Don't forget the git rm -r --cached ./node_modules!
Of course it still would be an option if these were stored out of tree, but it means I need to know where that is. I also have the potential to affect other unrelated projects.
I suppose this could be addressed by the tooling, if there was an easy way to get it to echo the target, and it made sure to always keep things separate.