The linked sources don't say the four "came to me before you", as above, but rather "came before you". So it's a joke about the fickleness of dream interpretations. That is, a dream of driving a four horse chariot is taken by one interpreter as bestowing the speed of horses on the dreamer, while the next interpreter says oh no, it's a dream about having four in front of you!
I think four other competitors came also, so the fortuneteller realized that either a) he was going to lose because of the competition or b) he already told someone they were going to win so couldn't tell multiple
They had the same dream.
Edit: I think your interpretation below is the correct one!
After looking up what a quadriga is, I think the joke is that he's behind four horses. So the dream can be interpreted in two opposite ways. So presumably the other four dreamt of being a horse.
There's the reinforcement of him being behind them on making it to the fortune teller as well.