Comment by maeln | Hacker News Reader

Comment by maeln | Hacker News Reader

Parent Full thread

> If your main complaint is that email is horrible to work with, the issue is most likely not email, it's probably your mail client.

This is definitely talking from someone who never had to write software that deal with emails. The reality with emails ? They are an horrible format. There is several RFC describing the "email" format, the usual reference is rfc2822, but technically, you can find other rfc that relate to it (ex, the MIME rfcs: RFC 20452, RFC 20463, RFC 20474, RFC 20485, RFC 2049, RFC 68388, RFC 4289, ...). There is a lot that left to be desired in those RFC in term of specifications. They tend to gloss over some part and also allow several format for retro-compatibility.

But then, even the part that are clear and well defined are not respected. I wrote software that dealt with emails for a company that receive several thousands emails on several mailbox everyday. The number of non-complying email that we received that broke our implementation was staggering. Nobody respect the date format, that is standard any more. Then we had some weird stuff, like fields that were mime-encoded even though the spec explicitly say it must not be, attachment metadata that were written completely wrong, headers that made no sense, ... And this was not from your small email provider, it was emails that were coming from a gmail mailbox, outlook, yahoo, ...

Oh and MIME is the best footgun in existence. It allows you to do anything and so of course, you receive anything, including mail-in-mail-in-attachement-in-mixed-content-in-multipart. And you better hope that the boundary are not bugged.

I wrote HTTP servers, IRC clients, BMP parser, and many more format. Nothing ever came close to the mess that is email. Anything is better than email, that is not the clients fault, it's the format fault. We just keep using it because of network effect. But you can bet that any other communication protocol / format sucks less than email.

Email, in some iteration or another (starting with RFC524), not only predates the internet, but predates TCP. The only older standardised communications systems still in widespread use are the telephone and the postal systems.

It's an unfortunate quirk of how the internet has developed that email has stuck around, like postal mail, as long as you put the correct address on the outside of the envelope, you can put almost anything inside.

RFC 822 makes it clear this is a feature:

> In this context, messages are viewed as having an envelope and contents. The envelope contains whatever information is needed to accomplish transmission and delivery. The contents compose the object to be delivered to the recipient. This standard applies only to the format and some of the semantics of message contents. It contains no specification of the information in the envelope.

I actually have written email software before (and dealt with the horrors of IMAP and MIME), but that's no tthe point of that sentence, it's about email management and not having your email client do stupid things. not about using the protocols

> it's about email management and not having your email client do stupid things. not about using the protocols

The format / protocol do impact and limit what the client can do. In a funny way, thanks to MIME, you can technically do whatever you want with emails, but that will be a poorly organized and structured whatever. In many way, emails are like HTML : A way to present information, not to structure and/or organize it. And therefore, there is only so much the client can figure out about the email. Thanks to Thread-id and the recipient list, you can group emails more easily and more relevantly, but that is about it.

For a suggested development workflow, it doesn't really matter that much since everybody is supposed to be using simple plaintext messages. Even patch content is just part of the body.

Otherwise, yes, email message format is a mess. I also had a pleasure of writing a MIME mail message parser in C. But it was quite fun to try to do it in streaming fashion with no dynamic memory allocation allowed.

Once, many years ago, I started writing a library for dealing with IMAP mailboxes.

My take way is I have no idea how on earth people have actual working IMAP implementations.

My first attempt about a decade ago resulted in me sending a patch to Ruby mainline :D

Email sucks less than any other communications protocol. The quote from the author is exactly correct. Users hate email because they use broken clients, like Gmail. Using a real client like mutt (https://lwn.net/Articles/837960/) is a pleasure; for one thing, it threads conversations correctly, unlike Gmail and almost everything else.

I used mutt, thunderbird, apple mail, outlook, gmail, sogo, roundcube and they pretty much all have the same issues. Yes, some make use of Thread-id a bit better than other (I like the way apple mail show thread and thunderbird is "okay", although sometimes it handle edge case in a weird way), but overall, they cannot fix fundamental flaws in the format in itself: Emails are hard to search, the threading model is simple but quickly turn into a graph with many leaf which become a mess to properly show and follow along, again, a lot of emails are outright broken and then it is up to the client to try to show what it can, the dates should not include timezone which is dumb so everybody do it anyway but follow their own standard, so you are not sure your client will correctly parse the date, files are encoded in base64 for hell-sake, ...

I'm currently touching the plan 9 email tools a lot, and recently rewrote the acme mail client. There are warts, of course, but the ones you've discussing are either related to the underlying communication and not the format, or are easy to deal with.

I'm not sure why you say emails are hard to search. Extract the text and put it into an engine like lucene. There are the usual small corpus problems, but these aren't dependent on format.

As threads are fundamentally a graph, so complaining that the structure matches the structure of the data is... Odd.. to me.

Including the time zone in the date is a small wart. Add an extra specifier to the format string and move on. Base64 is similar: a bit annoying, but decode and move on. And use RFC1652.

Indeed. Also, tools like grepmail make big mailboxes easy and fast to search through.

> Yes, some make use of Thread-id a bit better than other

Traditional email threading was based on the contents of the In-Reply-To and References headers. The algorithm is described in RFC 5256 for the IMAP SORT and THREAD extensions[1] and used in several mail user agents[2].

[1] https://tools.ietf.org/html/rfc5256#ref-THREADING

[2] https://www.jwz.org/doc/threading.html

>RFC 20452, RFC 20463, RFC 20474, RFC 20485

>RFC 68388

what are these numbers, I don't recall RFCs going into 5 digits yet

Yes my bad, it is from a bad copy-paste, you should ignore the last digit.

How do you copy paste something so wrong that it adds a random digit to each number?

They aren't random at all, they are practically in order. It was probably a footnote reference to the bottom of the source article, with a link to each RFC at the footnote.

The only way I think is that it was a numbered list and they pasted the RFC number before the list number so the list number was actually appended to the RFC number?

Reply on news.ycombinator.com