The question is more - how hard is it to get airside without a pass?
I once went in through the main arrivals exit doors at Heathrow. I'd forgotten something so I went in, got it, came out again.
It was a stupid thing to do, but I wasn't arrested, stopped, or anythinged.
This was before 9/11 so I doubt you could do it today. But even so.