Arathorn has argued before that this is "only an issue for malicious homeservers" but that is literally the entire threat model for E2EE -- if you didn't care about malicious servers you could just use TLS and avoid all of the "unable to decrypt message" issues.
Let's not get into the fact that even though they copied the Signal protocol (which is a good thing!), the amount of metadata stored by users' homeservers probably invalidates any of the deniability properties that motivated the development of OTR and Signal in the first place. Yes, the ability to keep chat logs forever is a key feature of Matrix, but you really have to ask whether trying to adapt the Signal protocol to their needs really made sense.