I would take a look at Matrix[1], which is basically what you're describing (it's a federated replacement from group chats that has Signal-like E2EE and has an open protocol) and it's already implemented.

[1]: https://matrix.org/