There is a spec change proposal for fixing this: https://github.com/matrix-org/matrix-spec-proposals/pull/253...