The problem comes when you need to do something bespoke and custom, like building a C dependency so you can link it into your Python (or whatever language) library. Sometimes your options are "run a makefile" or "reimplement an entire library from scratch". I'm not saying that this isn't a problem; it is. I think the better solution is transparent sandboxing for dev environments.