The main issue is you can accidentally nuke your own source directory and it's possible to do so in non-obvious ways. This has happened to me a few times (though thankfully had backups elsewhere).