Yet I do not understand why a rewrite of messaging as [matrix] was necessary, when XMPP was already there and matrix did not even have an edit-message feature on release, perhaps not even now.
And, more importantly, why would you want to do that? What would be the benefit if that was done?
Glad you asked! There's two main components to Matrix: the client-to-server (c2s) and server-to-server (s2s) APIs.
The c2s API on matrix is of little to no value and could have been implemented as an HTTP API to XMPP servers: the way this is done in the XMPP ecosystem is with BOSH so we wrap the XMPP protocol inside an HTTP-based bidirectional stream, but there's no reason you can't make a "native" HTTP API for your XMPP server.
The s2s protocol is the interesting part of matrix because it has a genuinely-useful property of treating a room as a single state handled by a consensus of varied servers. The usual XMPP protocol MUC is ill-suited for that, but the "new" (~2015) MIX specification is based on a pubsub model very similar to Matrix and building a consensus algorithm on top of that would certainly not be harder than to develop an entirely new protocol from scratch.
Now as for why you would want to do that: to avoid fragmenting the ecosystem. XMPP was developed to address technical limitations of IRC, which is not an extensible protocol (does not use namespaced trees for messages) ; implementing a new feature in IRC is always at the risk of breaking clients and the IRCv3 working group spends a lot of time trying to work around that. XMPP was from the start conceived to be extensible and so there was little reason to reinvent a new protocol and software ecosystem just to add a single feature (decentralized rooms).
After all, matrix and XMPP have the exact same selling points (over a decade apart): federated and interoperable communications with bridges to 3rd party protocols. Now we have two protocols to address the same use-case, each with their up and downsides but certainly both further away from their goals than they would be if they mutualized efforts instead of reinventing the wheel. I have criticisms of XMPP just like i have criticisms of HTTP/HTML. But i personally do not understand the appeal to reinvent an entirely new markup language and browser/server ecosystem just because i'd like support for a new HTML element.
XMPP clients running on a tiny VPS just feel very fast in comparison to Matrix clients running through a much larger VPS. Rooms still take ages to join whereas Snikket joins almost instantly. I've spent truly astounding amounts of time debugging Matrix's Postgres database, to video calls to Discord bridges. Snikket just keeps on ticking no matter what I throw at it.
It's starting to feel like the spec itself might be to blame but I welcome the input of experts, such as yourself.
YMMV™
We have yet to manage not getting any OMEMO issues in a 2 users, 5 clients situation (2 Conversations on android, 1 Gajim on win, 2 Pidgin on linux)
Quality of Implementation matters. If Pidgin "doesn't exactly have the reputation" and yet it's still notionally part of your ecosystem, then your whole ecosystem doesn't exactly deserve the reputation you thought it did.
Here's my unsolicited recommendation: Block Pidgin. Not just Pidgin, any time the XMPP community finds itself having to explain that oops, that client doesn't work so good, please use a different one, just block the client, permanently instead, from all popular and well known servers, tell anybody who will listen that's not a real XMPP client, not any more.
Any time the community just can't bring itself to block a client, cross off the features that don't work across the ecosystem due to that client. Those aren't really XMPP features any more since they exist only for some users and the community is apparently OK with that.
And each year, stand back and look at the resulting two steps forward three steps back juddering "Progress" and reconsider - was this the way to get where you wanted to go? Or are you all wasting your time on an exercise that would be easier done a different way.
When a web browser fails to display a certain page, would you recommend the site operator to block it? Maybe we could allowlist specific clients and send a warning message to others just in case they're not aware they're using something niche.
i wrote that omemo plugin and tbh I am pretty burned out. so many moving parts, it's hell to debug and i'm not even sure where to go next with it. (plus people keep talking badly about pidgin anyway and i am not sure if it's even worth continuing.) i am very thankful for all the contributions so far, especially the help with packaging it. sorry if someone feels let down.
i feel like this is the right moment to ask: if anyone reading has an idea how to improve the state of things, i'd be happy about some suggestions. the project page is https://github.com/gkdr/lurch
That's what's great about it and part of the reasons i think blocking Pidgin would be very wrong.
> i wrote that omemo plugin and tbh I am pretty burned out.
Thanks for that work! Maybe you could start by writing about your burnout on the README to encourage other people to contribute/maintain, or at least be less demanding of you?
> it's hell to debug and i'm not even sure where to go next with it.
Do you think this is specific to your plugin? To OMEMO? Or to pidgin plugins in general? Personally, i find it very challenging to write correct code in C, and i can only imagine the pains pidgin developers have to come through to make so many networks/protocols inteoperate. If you have some reading about your experiences, what lessons we can learn, and maybe some project (whether internal to pidgin or not) to implement them, i'd be interested to read.
> sorry if someone feels let down.
I don't think anyone feels really let down. Some people are not aware of just how experimental certain things are and so they get disappointed but life goes on. You however should take care of your mental health and not let maintenance burden and heartless cyber-ranters like me affect your mood and taste for computing <3
you're welcome! :D that sounds like a good idea, at least so people know i am not planning to abandon it even if i'm slow. i don't think people are demanding, i just feel bad that despite my best efforts it just doesn't work for some.
>Do you think this is specific to your plugin? To OMEMO? Or to pidgin plugins in general? Personally, i find it very challenging to write correct code in C, and i can only imagine the pains pidgin developers have to come through to make so many networks/protocols inteoperate.
in fact it's mostly related to your last point. there's just so many moving parts to this decentralized system: the os, installed libaries, exact version of the plugin, exact version of pidgin (or rather the underlying libpurple), the version of the used xmpp server, the version of the module implementing the XEP on the server, the version of the client on the other end, and how exactly that client implemented OMEMO and other XEPs in combination (e.g. the order of elements or their attributes in the received or sent xml).
right now there is a persistent bug that some omemo xml element cannot be parsed, and based on the input taken from the debug log i cannot reproduce it in unit tests, and i don't even know where to begin, and why it doesn't seem to affect other clients. also, persistent discovery issues that from my end don't look like a problem, and yet this plugin is the only one having these problems. i believe other clients must spam those discovery messages just in case, i didn't verify that yet though :)
>If you have some reading about your experiences, what lessons we can learn, and maybe some project (whether internal to pidgin or not) to implement them, i'd be interested to read.
i don't, but maybe i should write it down! venting in this reply already helped a lot, thank you :D
>I don't think anyone feels really let down. Some people are not aware of just how experimental certain things are and so they get disappointed but life goes on. You however should take care of your mental health and not let maintenance burden and heartless cyber-ranters like me affect your mood and taste for computing <3 aww, thank you! well, it's not like i don't get it, what i do can barely be called maintenance and libpurple's XMPP implementation isn't exactly modular, so you need to install 5+ plugins to get a "modern" messaging experience alone. i don't think i let some ranting get to me, don't worry about that! what i do feel bad for, aside from the people who are disappointed that it doesn't work, are the people who have actually contributed already, or were trying to contribute but i didn't look at the PRs in time and now the codebase just moved along too far.
i got some helpful lessons out of it either way, so i definitely don't regret it. a big part of it is what i already tried to say in my last post: i'm mostly not sure where to go with that project.
My problem is that I'm using several clients (mostly profanity on several machines, occasionally dino) for the same account and it seems that my contacts keep forgetting some of my OMEMO keys, so I often receive messages that some of my clients can't decrypt and I have to start multiple machines just to find one that can decrypt the message.
So far I haven't figured out how to debug this.
This is why some security-oriented distros such as Tails are looking for Pidgin replacements (Pidgin is arguably the widest attack surface for Tails RCE), and why in most public channels i hang around in people recommend to stay away from Pidgin if you're looking for an XMPP client that supports "modern" extensions.
Personally, i have a lot of admiration for Pidgin as a concept but i can only imagine how hard it is to interoperate with so many networks in a memory-unsafe language. I wish the situation keeps on improving thanks to devoted people like you and i'm interested if you have some more detailed reads (blogposts, issues) about the future of Pidgin or other such multi-network clients.
Thanks for your hard work and take care!
>"edhelas commented Nov 26, 2021
As OMEMO is now part of Movim master and that the main working features are working I'm considering this ticket as solved. There is still a lot of small things to fix but the main is done.
It took 6 years but Movim is finally having end to end encryptio edhelas commented Nov 26, 2021
As OMEMO is now part of Movim master and that the main working features are working I'm considering this ticket as solved. There is still a lot of small things to fix but the main is done.
It took 6 years but Movim is finally having end to end encryption."
As to whether one approach is better than the other is left as an exercise to the reader :) But having each approach belong to separate projects allows them to carry on as they see fit, and means they don't have to participate in a zero-sum game.
At this time there are 9 room versions. Once the room version had to be incremented because of a bug in the consensus algorithm, which required a new algorithm, i.e. room version.
That's it. Bugs happen when you create new tech...
> Except of course for censorship-resilient public rooms where it shines, which goes back to my point that "decentralized rooms" as an XMPP extension would have made sense.
There are many more reasons for wanting decentralized rooms, e.g. autonomy, reliability, availability, anti-takeover - and it works for private rooms as well. Prospectively, it enables things like P2P and multi homing
I would never want to go back to the XMPP like it is today. And XMPP today is not Matrix, even if you could turn it to Matrix, but what is the point of doing that? Matrix is already there!
If you want Matrix to be an XMPP extension, I suggest creating a new XEP pointing to spec.matrix.org and call it XMPP v2. No idea what you would gain from it though.
How so?
We have OMEMO now, key exchange is paranoia-level secure.
Currently, verification seems to work on a session basis and not just per user, which makes it very tedious to have everything verified e.g. a group chat. If it is not almost impossible given peoples sessions change with time.
Add a new session in the browser -> meet every contact to verify their sessions with your new browser session? Doesn't seem very practical, and reminds me of the old times in [matrix].
This opens room for MITM attacks
Also you don't have to meet with everyone to verify new keys, you can just use a known good key for that. Essentially, if you want 100% security by comparing keys in a meetup, you can do that once and then use that known good channel to verify every new key.
In reality, cryptonerds will be happy to verify every single new key, everyone else will never bother to even open the key signature view and I will get a confused call should it be opened by accident.
> This opens room for MITM attacks
Yes, in your constructed strawman world xmpp is prone to MITM. In reality it's equally or less prone than its competitors.
The server could keep an append-only log of published keys, and then you could require m-of-n signatures of other published devices/keys to register a new one, in which case no further verification would be verifiable if you already trusted your peer's other keys. In addition, you could introduce peer keys in private rooms automatically and/or employ some form zero-knowledge proofs in public venues.
XEP-0450 Automated Trust Management is a first exploration in this direction. There a lot to explore and I only wished skilled cryptographers spent more time researching these issues on federated networks instead of advancing yet-another-centralized-messenger-of-the-month.
> Also you don't have to meet with everyone to verify new keys, you can just use a known good key for that.
Also known as a keyserver. But you could also employ cryptographic challenges like OTR does, like a shared password to establish the session.
that's not true for most clients, though (AFAICT, only riot works with itself).
I might even say that the situation is better/more sustainable in XMPP where more diverse implementations of the encryption protocol are available, with all mainstream clients supporting it: https://omemo.top/
Riot doesn't exist anymore. It's been Element for a while now