I think its as important to make it "hard to do the wrong thing" as "easy to do the right thing". In this case having to explicitly exclude a file containing passwords from being deployed would fail that rule of thumb.

The Azure Key Vault is a good solution that so far seems easy to work with (I've only just started using it though) and it can make the storage of secrets easier to secure but you still have the issue of storing the credentials to the key vault, so its difficult to fully get rid of the problem although you can definitely isolate access to your passwords that way and then have a more limited number of credentials to secure.