The issue is that an exploit could write a file in ~/bin/ and trick you in to executing it. But, if something malicious is writing files in your home directory, you've already lost.