It’s OK-ish to just open the door, say something like “uh sorry our meeting just started in this room”, then leave but keep the door open.
I can't tell you how often my team knocked at door A to throw out another group that then all left by door B, leaving us waiting outside until we knocked again 5min later.
I always did the knock-on-door + tell them my meeting time has started and never had an issue, be assertive and people will move quickly. Escalating impoliteness doesn't help anyone except your own sense of self-justice.
Eventually broke the habit, but for the time I was doing it it did break up the previous meeting pretty quickly.