I do the same. I've yet to hear a convincing argument against this practice. Everyone seems okay with passwordless docker and you can use that to privilege escalate too.