Yes - there's a relay server, but you don't have to trust the relay. (As of changes made in March - before that Croc sent the key in the clear to the relay.)
The first three characters of the passphrase are used to establish a shared channel between the sender and receiver, and the rest of the key is used to do a PAKE, which is a secure method for key exchange. The default passphrase is only three words long, and according to another comment the wordlist is only 1626 words, so you should assume the first word of the keyphrase is entirely blown and useless for securing you from the relay. So that makes the space effectively 1626^2 = 2.6M passwords. To MITM you the relay would have to guess which of those passwords is correct.
Magic Wormhole works a bit differently. IIRC it dynamically requests a channel from the server which is prepended to the passphrase as a number. This seems a bit more resilient to me, as the birthday problem suggests it's quite likely two people will end up with the same channel just by bad luck with Croc.
Edit: indeed, a test forcing similar passwords beginning with the same three characters completely screws Croc up. I can do "croc send -c 'xyz blahblahblah' filename" on multiple computers without error, but croc receives go to the first channel even if they have the wrong password.
Edit: There are only ~900 different three letter word beginnings in the wordlist, so you'd only need to claim that many channels on the server to make it unusable for anyone who lets Croc pick passwords, and occasional collisions between users are certain. Lol that should probably be fixed.