The middle-man in this case is a trusted one, the owner of the centralized infrastructure, not like in MITM.

Isnt possible that one peer encrypt, pass it to the central server who have the other key, the central server than encrypts again and share it with the real end making it believe the key he is using actually is the same one generated in the first part of the process?

Its like the OR from tor but with 3 parties instead.

How the receiving party can be sure the key was not switched by the all-mighty middle man who can control everything?