That's a bit disingenuous. The fact that privilege escalations exist doesn't mean one has to trust all the software one uses completely and unconditionally; it means there are risks that should be taken into account.
Given we're talking about the choice between an authentication method where you can trivially capture someone's password, vs an authentication method where you can't (and where you would need to successfully employ some sort of exploit to do so), then the level of risk involved in each is quite different.
(Not only that, but I'd argue people will tend to trust software more when it appears to be making best efforts with regards to security. Any obvious security-related flaws will erode that trust rather quickly.)
(Edit: typos)