If we're moving into the realm of the ultra paranoid, high security over convenience focused user - sure. In fact I dare say such a user might prefer a Linux phone with physical keyboard, or no phone at all. But if we're talking average iPhone or Android users here, I can't imagine the scenario would change much at gunpoint.