People who arrive join the front of the queue, and are either served next, or someone else joins in front of them, pushing them back. When they're position 2 in the queue, they will only be served if 2 people are served and 0 arrive (low probably). If another person arrives, they're now position 3 in the queue, so 3 need to be served before any arrivals (even lower probably).
At some point, they decide the probability of being served drops low enough that they give up and leave - this will usually happen quickly.
The only way to 'game' this system is to look up the street and join the queue when it looks like nobody else is coming. If many people try to do this, then those people will be waiting forever (both want to join last - stalemate), and anyone who doesn't employ the tactic gets served.