I absolutely never do this. Maybe because I'm paranoid, and especially if I'm flying internationally, I never, ever, feel comfortable handing my mobile device to security or Immigration. I still get a paper one every time.
In any case, as the other person mentioned, my phone is locked the whole time. The only thing in the wallet they can access is my boarding pass.
Usually you don't hand your boarding pass to anybody. Instead you self-scan it once when you enter the passenger / transit area and a second time, when boarding the plane through a self-service gate.
This may vary by airport, but on most of my flights through a number of European airports access is pretty much automated.