This would be a disaster for those living in repressive/dictatorial regimes.
Essentially handing them the keys to your e-mail.
Police and secret service certainly have no qualms and little oversight when it comes to intercept a text message to access your accounts even in 'civilized' countries.