In that case, the authenticated device could have the ability to show the public key before accepting a request, and match it with the public key generated on the new device. But yes, at some point you'll have to put trust in the server not doing malicious things like that, which is one reason it's open source