You could have the server know the addresses to forward the message and then forget them. However, then the server knows this information at some point and it could be sniffed/recorded along the way.
Does anyone know any more details about the specifics of the protocol? How would you minimize metadata leakage if you were implementing such a protocol? I am not sure it is possible to guarantee the recipient list won't be leaked.