Within the Schengen Zone it's only the check at the gate to check your ID matches your boarding pass (so no check at "immigration"), and to/from the UK the electronic gates have really sped things up (compared to the old days).
YMMV but I find I'm rarely waiting more than a few minutes at the immigration checks (to/from UK) nowadays.