Decentralize Messaging
rodarmor.com
rodarmor.com
delivery receipts => XEP-0184: Message Delivery Receipts
optional read receipts => same as above
user presence information => XMPP RFC
binary serialization for efficiency and extensibility => XEP-0231: Bits of Binary
end-to-end encryption => XEP-0373: OpenPGP for XMPP, XEP-0384: OMEMO Encryption, XEP-0378: OTR Discovery
WebRTC signalling for negotiating VOIP and video chat => XEP-0343: Signaling WebRTC datachannels in Jingle
signed introduction tokens to reduce spam => ?
a standard extension mechanism => https://xmpp.org/extensions/
Doing decentralized social media, on XMPP, is possible. Take XEP-0060: Publish-Subscribe, add Atom 1.0 (yes it's the power of XML, you can put a standard in another) and boom you have a social network with feeds, comments, subscriptions and everything, fully ready.
I'm doing that for years with Movim https://movim.eu/. The several major XMPP servers are handling that perfectly as well. How many Matrix servers are out there ? One, that is still in beta (Synapse).
XMPP is standard (IETF wise), is already massively deployed, is used by universities, governments, companies…, is exensible, is stable and is maintained by a big and motivated community.
Don't reinvent the wheel once more, just implement the standards.
> How many Matrix servers are out there
On a side note, that's not fair. Every project has to start somewhere and it might be difficult to gain traction. It doesn't mean that it has no chance of evolving into the greatest thing possible in its market, dwarfing once popular competition, simply because it didn't come first.
XEP 0357, implemented in servers ( https://modules.prosody.im/mod_cloud_notify.html for prosody, for example), and implemented by clients ( https://github.com/siacs/Conversations/issues/1171, for conversations for example).
Conversations also uses XEP 0198 and XEP 0352 for battery optimization: https://conversations.im/#optimizations
To compete with things like Slack, the website should be even simpler: A single choice of client / server that are known to work together perfectly, a big download button to an all-in-one installer that requires little configuration out of the box, and nice big screenshots of all the included features.
Same for IRC.
While Synapse is the most mature server implementation there are others under active development listed here - https://matrix.org/docs/projects/try-matrix-now.
Check out https://github.com/matrix-construct/construct and join us at #admins:matrix.org
(1) The address book.
(2) The easiest possible signup and setup.
More generally for #1, whatever method(s) you use to find people's contact information and connect with them.
Without that, you could have created the perfect chat experience in every way, but people still won't use it. Because their friends aren't on it and/or they can't find their friends.
A ton of chat apps fail because engineers sit around talking about how to architect things, when the truth is that once you've gotten past the almost-insurmountable hurdle of getting two users onto the system and starting a conversation, the actual mechanism for actually sending messages back and forth is not much more than an implementation detail.
Not that those things aren't important, but they are like 10% of the formula for success, and the other 90% is getting to the point where they can start a chat session.
People's primary need is to communicate. With the people they want or need to communicate with. And via the contact information they have available. (Exchanging contact info for a new service offline is a huge hurdle.) A better experience is better, but at the end of the day, these other concerns will nearly always win out. So you don't get very far trying to use a superior experience as leverage to get people using it. You need to attack that problem more directly.
I have helped thousands of people exchange contact info after meeting in various niche chat rooms first, but you can flip that and have your irl friends meet you in decn-chat dot org click the 808game room, we can meet in there and pm/dm a code word or something and make each other contacts in various ways.
Your point is something to keep in mind though, ultimately I think we also need apps that can display a qr code like a one time pad to give someone contact details or any chat rooms to meetup in, Other methods of exchanging info about contact and chat room / chat app options should be added to whatever is going to be really good.
2 - I really agree with this, I spent some money and time on rocketchat development to give an optional 'open lobby' to anyone who enters, just pick a name and you are in.. then if you want extra abilities you can add an email or verify or whatever.
easy to get in is important indeed.
I used to call it ‘email for mass communication’ but it confused people, since it’s not based on email, so I stopped calling it that. That’s the goal though.
I also gave a talk on it at the Internet Archive last week, if you want a quick intro in video from. https://archive.org/details/12120iadweb (My part starts at 1:13:30). If you have any questions, happy to answer.
Now, I recognize that regular e-mail uses proof of work as a spam prevention measure these days, so I don't want to be too harsh on a decentralized alternative.
That said, I've watched the energy expenditures of Bitcoin etc. with alarm, and I've become very concerned about the sustainability of anything using proof of work if it should become popular. Fundamentally proof of work makes people waste energy in order to accomplish tasks such as sending messages, at a time where we need to use as little energy as possible to reduce emissions.
Have you considered sustainability at all with Aether? Have you considered any alternatives to proof of work for dealing with spam?
By having them separate, I can make sure that the the P2P version isn’t influenced by money-making concerns, since we have a product explicitly made for that.
It does not need to be proof-of-work payment, or even a cryptocurrency payment. Gateways can have peering agreement with each other paid in dollar.
“Secure Scuttlebutt (SSB) is a peer-to peer communication protocol, mesh network, and self-hosted social media ecosystem. Each user hosts their own content and the content of the peers they follow, which provides fault tolerance and eventual consistency.[5] Messages are digitally signed and added to an append-only list of messages published by an author.[6] SSB is primarily used for implementing distributed social networks, and utilizes cryptography to assure that content remains unforged as it is propagated through the network.”
Read more: https://en.wikipedia.org/wiki/Secure_Scuttlebutt
Join us! There is a fully functioning client, and bustling Solarpunk community: http://scuttlebutt.nz
Also, append only => no deletes, correct?
This is something that will probably take some puzzling to get to a sustainable mechanism. The strategy I've heard talked about most often is to append metadata that marks it as 'deleted', so when things populate or 'gossip' further, it won't show up in the user interface. Instead it's labeled as 'archived' or 'deleted', or has a link to it's replacement.
Also, how much you download is related to how many people you follow, and text doesn't take up a ton of space.
As for text not taking up a ton of space. Sure. Except it's text with it's metadata, and possibly text generated by a few hundred people.
A friend and I managed to exchange ~24k messages on Slack over the last 3 years (~4k each / year); using it like some kind of "private twitter" where we would just post anything that might be of interest to the other.
What would it be like if I follow 300 or 1000 people doing the same, over Secure Scuttlebutt?
- At 1 byte per character (from what I've gathered, Unicode characters actually take up 1 to 4 bytes) - 140 characters / message (because Twitter proved it's usually enough) - messages' metadata (50 bytes?) - 4.000 messages / user / year - following 300 people
We'd get... 228MB / year?
That's actually way less than I expected!
I wasn't even worrying about the legality of it. Simply that you end up lugging those messages around on your devices.
And better be careful about what you post, because there's no taking it back.
And the protocol spec page is fantastic: https://ssbc.github.io/scuttlebutt-protocol-guide/
... that said, it has some enormous problems around multi-device, rather costly replication (start at the beginning, catch up to now, otherwise you can't verify anything), and it's fairly normal to miss part of a conversation and not have any real way to get it filled in. There are a number of branching concepts with semi-compatible protocols that strike me as a lot more promising, but odds seem pretty good that the community will shift if necessary (which is a good thing).
Worth a shot for anyone to explore if they're curious though, there's even a largely-functional mobile app: https://www.manyver.se/
[1] - https://delta.chat/en/
[1] https://valyr.io
Counter-intuitively, the current goal is to centralize the messaging interface, providing a common platform for users who are willing to pay for a consolidated system that saves time and confusion, and to reduce the waste of niche use cases (e.g. having the entire [name here] messaging app on your phone because one weird family member or friend insists on using it)
Bringing email, text, and (aspirationally) all other messaging systems into one.
In doing this, we cannot possibly hope to uniquely integrate each of the countless hundreds of services with their weird quirks and formats, so I aim to establish a standardized language for async messages equipped with the basic common features and extensions for prevalent (but not universal) features like read receipts, "liking/reacting" to messages, etc
It would be really cool to open up that standard to the world if we do a good job of designing it, or collaborate with anyone else who wants to participate in this goal. Our core service will be a niche, paid, closed system, but the underlying language I would like to be open source and interoperable with whatever else turns out to develop (Twitter's new endeavor and Matrix are the two efforts I am most aware of)
After all, Valyrio means Language
I've also been working on an adjacent concept with [https://jwmza.com/polymath], which is a new comprehensive markup language meant to define long-term compatible, human readable documents and websites/document structures which intermix existing standards like (La)TeX, HTML, CSS, Markdown etc.
Perhaps the underlying language and concepts here will mix with the effort to decentralize messaging as well, as messaging is fundamentally the asynchronous bidirectional transfer of documents of mostly text and sometimes other media or information.
Federating an RDF graph is powerful.
This is patently false. Look no further than the other comments in this thread. Nobody can agree on what they want out of the messaging experience. Some want a rich experience like telegram and others want a minimal one like IRC. Should this protocol's features maintain parity with the proprietary and centralized services? Better develop them rapidly to keep up. Should they stagnate to maintain interoperability across an ecosystem? Good luck being competitive. We haven't even gotten to the features themselves and I promise you there will be irreconcilable differences. Does the protocol maintain message history or is it ephemeral? What is the privacy model? Can we delete messages on other servers due to GDPR? When you spend time at the business end of the messaging space, these dichotomies flow endlessly, without consensus, and they cover the entire horizon of the space.
If you think you know the answers to these questions, you don't, because you're solving the wrong problem. If you want to decentralize messaging you have to solve the problem of how to agree to disagree while maintaining interoperability for a shared experience. Let me just make that last point clear: you cannot design a messaging protocol where one party sees emojis and the other doesn't. That doesn't work because you can't lose social information.
Matrix has failed to solve this problem. We need a new approach. What will emerge to finally conquer this space?
Two services don't need to have identical feature sets to interoperate, because you don't need perfect, full interoperability, if you can send most of your messages across platforms, that's still pretty good.
> Let me just make that last point clear: you cannot design a messaging protocol where one party sees emojis and the other doesn't.
Which service doesn't show emojis? Literally every mainstream mobile operating system and desktop browser supports emojis as regular text.
The tendency is for people to invoke features situationally. If you are communicating with someone who is using rich-replies there is a tendency to also invoke that feature yourself. If people are using emojicons there is a tendency for others to play along rather than use plaintext characters. Feature invocation is situational and relative.
> Two services don't need to have identical feature sets to interoperate, because you don't need perfect, full interoperability, if you can send most of your messages across platforms, that's still pretty good.
This has proven to result in an unacceptable user experience throughout the history of messaging. When a system loses social information from the environment it is no longer an effective communication tool. Worse, it is dangerous. If you send me a message and I respond to your message with some thumbs-up emoji, and your platform doesn't support it (e.g. a text-based IRC client) that information gets lost. You believe I have disregarded your message. You may assume I have been rude when the opposite is true. This is absolutely unacceptable UX.
This is an assumption that bad implementation is the only possible implementation, and it is wrong.
Obviously omitting content from missing features is bad UX - but it's absolutely not necessary to interoperate. It's easy (and should be expected) to avoid this by simply telling the user something from a missing feature has been sent - they can then inquire and get back the lost information / clear up the confusion.
Furthermore if interoperability is implemented on both ends, the sender's client should know to alert the sender before they attempt to use features missing on the receiver's end.
None of this is a difficult issue.
COI is chat over IMAP[0], I think it's one of the best solutions I've seen in the sapace.
He said the move is to specifically reduce fragmentation in public conversation.
I'm in the decentralized (dWeb) community (we have 10M+ users monthly, in-production, running thru GUN) and admittedly, I think we all get distracted with the war cry of privacy, p2p-for-sake-of-p2p, etc. but is not many other people's end goals.
1) not knowing what is supported on the server, or the other users server.
2) not handling mobile clients well (heavy battery use, weird deliverability issues.)
2 is BS with capital letters. See Converations and it's forks.
Did you ever wonder why there are no (to date) XMPP clients on iOS that are not complete and utter shit?
As a person who has first-hand knowledge of the matter, managing a team member tasked with the development of an iOS XMPP app (which is supposed to be NOT complete and utter shit), I tell you that creation of such app is impossible without reinvention of half of XMPP stack (basically every part that touches client-server interaction). Thankfully, s2s works rather fine in XMPP.
A phone user expects that messages would be displayed to him in notifications when received. One does not expect to be cut off communications because he had pressed a 'home' button, but it's what happens, and a user does not have any control to prevent it from happening.
To counter this and make XMPP work well on mobile and in browsers, we had to develop our own server and a bunch of new protocol extensions.
Real question: for email, I use K9, which supports IMAP Push, which, as far as I know, needs a live connection as well. Do you know if they are struggling as well, or is the xmpp push and imap push significantly different in some way?
Service -> Push service -> APNS / FCM -> Applications
With XMPP, servers act as 'service' from the scheme above, then relay a push initiation to 'Push service', maintained by a client app developers, Push service relays information to a respective service by Apple / Google, who deliver push to an Application, which connects to Service (or does not, pushes can contain all necessary information for displaying in notification, but that's a story for another time).
With email, I'm not really familiar with how IMAP Push works, but the scheme must be the same: an email server must relay info to a push service maintained by app developers, in your case, K9 guys.
From what I was able to gather after reading wikipedia for 5 min, an IMAP server upon receiving an email sends info to some 'mail user agent'. From wikipedia it's not really clear, but it appears that is is supposed to work on end devices. In this case, it won't really work as 'push'.
Google (or any other email provider, like Protonmail, who have their own app) get around this because they control both email server and push server, so it's trivial for them to initiate a push when an email is received.
XMPP server -> Push service -> APNS / FCM -> XMPP client.
Push service here is a service maintained by an XMPP client developers, cause you need a client app certificate to be able to send messages to it via APNS/FCM, and no sane devs will be handing out their certificates to third parties.
It should be noted, that the recent changes to how Apple push notifications work in iOS 13 (MORE restrictions on VoIP and 'silent' background notifications) require some updates to it, but so far, XEP-0357 does not look like something terrible.
With message archive management and message carbons there are no weird deliverability issues. I have not noticed battery issues with conversations running on several devices for the past couple years.
I actually thought about using matrix, but it apparently "modern" means "resource hungry". My xmpp instance server runs on a small server with 2GiB Ram, with more than 1.5GiB being unused.
Than allowing for messaging or "social network"-like features?
From the user in practice, Matrix and XMPP (IM) are the same. These protocols might have made different technical choices but the user still only wants to send message and do various other things that both these protocol support. So not replicating all that data is actually a feature IMO.
Yes, I'm full of anger. We had xmpp, pidgin, miranda, trillian. Now we have gazillion mobile only (no, a web gw through a server to your phone is NOT a desktop client) messaging, so let's throw matrix in to solve it, instead of building an xmpp client with up to date xep support. Xkcd 927 on steriods.
To date, no one ever developed XMPP chat applications as a product, which can be easily deployed and will work consistently on every platform. Client and server developers were always disjointed, working separately from each other. This lead to great inconsistencies in implementations of even such basic functions like adding a contact. Also, it often happens that when a client developers need some feature that honestly should be done by a server, a developer still does this on a client, because it's all he has, with subpar results.
Absent leadership from XSF also plays a role. This club now mostly cares about bureaucracy and following a set of self-imposed rules instead of developing a set of working standards that would allow XMPP apps to compete with the best messaging apps out there. That's why it is unlikely for any great product to appear under such guidance.
We're trying a different approach, maybe we'll even succeed. If so, you'll hear about it on HN.
https://play.google.com/store/apps/details?id=im.quicksy.cli...
> a product, which can be easily deployed and will work consistently on every platform.
Actually, I don't know why the ChatSecure guys don't get their software to work reliably. It got a lot better over the past two years, but sometimes I still wonder why a ChatSecure user doesn't read my message. In addition, it is not as easy to use as Quicksy/Whatsapp.
One of my friends uses Monal, but in essence, it is the same story as ChatSecure: It kinda works most of the time and got better over the past two years, but I am still not confident it works reliably.
So for iOS, I have no solution and neither do I know of any _high quality_ XMPP client that supports video calls. But in fact, those issues don't stop me from using it :-)
Email info@xabber.com, we'll send you a link to test version of Xabber for iOS. We hope to release it this month. It even has voip calls, for real!
When they have dropped Gtalk and went with Hangouts (likely, because of a WhatsApp envy), crippling XMPP compatibility one step at a time, they have claimed the reason for it is that it is 'impossible' to achieve great chat performance with XMPP. Of course, that was a lie. It is quite possible, and we're doing just that.
How could anything an XMPP server do affect how a client adds a contact?
That is no excuse for doing embrace, extend and extinguish as your Xabber seems to be attempting with the XMPP standard. I can't help but note that the Xabber project has outright rejected OMEMO capability.
If you are extending XMPP in incompatible ways (as Whatsapp did) it is only ethical to clearly state that to potential users.
2. We don't 'outright reject OMEMO capability'. I reject OMEMOdiots who moan excessively because their wishes aren't satisfied on the spot.
Why are not they satisfied? Because XMPP does not have a reliable control of message delivery, no way to remove messages from message archive, no good group chats, no client sync protocol, no way to revoke session tokens, no way to know which messages in archive were read, no way to add reactions to messages, no way to do threaded conversations, and while we're building all that we're constantly attacked by individuals who demand us drop everything we do and urgently add OMEMO so they can feel safe (and do it at our own expense, of course).
We'll provide some E2EE capability, but only after we are finished with everything else that we consider more important than this.
3. Extending XMPP in incompatible ways, I'll cover this a bit.
Do you even remotely imagine an amount of shit an XMPP client (say, a web client with no stored history) built on 'compatible' XEPs must do to display a telegram-like interface, with all the recent chats? How much time does it take? Best result we could achieve in our web version with 'compatible' server and 200 contacts is like 5 minutes. With our custom XEPs, it's currently down to 15-20 seconds, and our goal is to trim it to 3 seconds. Does it break anything with s2s? No. It only adds methods of how a client interacts with a server, that's it. Does it break legacy clients? No, they still can do it the old way, if someone is willing to wait 5 minutes.
Next, we have our group chats. They are powerful: archive search, moderation, anonymous mode, public mode, pinned messages, replies, mentions, extremely agile restriction/rights management. They are very easy to implement (on a client) and provide a nice fallback compatibility for any XMPP client even without support for them. You don't need support on participant server to use it (not as in MIX, to say). You can chat using multiple devices, with legacy clients alongside supporting clients, all working nicely with simple Carbons.
Documentation for all improvements is open, (it's not in english yet, but we're getting there), implementations are open-source and distributed under GNU licenses. No, I don't think your point about extending in incompatible ways is valid.
Probably, the more correct way to call it is that we are forking XMPP. Like, in git. We'll be sending XSF pull requests on every XEP we create - hopefully, one day XSF will come to their senses and see that their platform is fading into obscurity and irrelevance, and that to prevent it from happening they should do something differently.
It was not used by many people to talk outside of Google servers (federation ability went almost entirely unused), and it suffered from tons of inbound spam problems, so they shut it off.
To answer your question directly though, XMPP doesn’t seem to be that great of a protocol. I’ve heard repeatedly that implementing it is a big mess, and that XML was a poor choice. I’m not sure that this is the reason that users don’t use it, though. It could be that these days, the client-server model is mostly obsolete, as due to battery constraints, phone users mostly rely on Apple or Google push messages to notify.
At this point the closest thing to a standard for person to person messaging is SMS... which like the old guard of ICQ, uses a number rather than a screen name.
That said, I would love to see a good standard for person to person chat, especially one that doesn't rely on magic numbers.
The future of messaging, decentralized or not, is certainly something with a tractable protocol.
The biggest problem with social media is how it is ruining our mental health.
It’s sad that almost no one looking in to this.