Decommission of Cyrus Email
cmu.edu
cmu.edu
We are committed to improving the project and keeping it open. As a member of the Cyrus IMAP board, I'm very proud of the 3.0 release that we recently made, and we're currently planning for the 3.1 release which will include further significant improvements.
I often worry about infrastructure projects like IMAP, SMTP, DNS, SSH, OpenSSL, etc. They often have very small teams, mostly volunteers, and the implications of a bug, particularly a security bug, can be catastrophic. And, if something we rely on were to be abandoned, we'd probably be screwed; even in cases where we have the technical ability in-house to maintain another major project, we don't have the time (or the budget to add more people to our small team).
Though admittedly I'm inclined to move the customer I have using it to Office365 hosted Exchange since they recently decided (against advice) that it was Very Important that their users start using Outlook which has meant a shift to local PST files.
The spam filtering isn't as good, but Google's false positive rate is atrocious
But I do pay for my Fastmail service. :) :)
It means we don't have to spend time working out how to trick you out of money, or convince somebody else to give us money in order to hire our staff. I did an all-staff presentation last week at our quarterly meeting - by far the majority of the money coming from our customers goes directly to paying our staff. Hiring good people costs money, and running a good system requires good people!
Honestly, blind users might have a better time because they don't have Google's UI nightmare in the way. I'm willing to bet their accessibility features are way more stable.
(hence the other comment further down this tree about having good protocols... I think that's the real way to solve people wanting unchanging UIs. Protocols change much more slowly than individual products)
How many users in total are you supporting?
What number of incidents do you see in a week/month?
How many extra help desk people have you had to employ/redeploy?
Whose budget do the extra support costs come from?
There _are_ open source alternatives, but they aren't as good, that's all there is to it. Setting up a mail server is hard, not because it needs to be but because the software requires eldritch incantations to make it work properly (there are too many things to keep track of in order to A) get email delivered at all and B) not become a spam relay)
It's possible to be something in between the "Just Works" walled garden that gives you no options or control and the archaic wizardry which requires a 2 foot grey beard to understand.
I remember when people thought spam was single handedly going to make email unusable.
Basically you don't control who gets to send you mails. You're entirely at Google's uncompromising and unreachable mercy.
I only ask because I don't recall any problems over the years with one to one emails either to my gmail or from me to other people's gmail accounts.
DKIM is signing your email with the public key in DNS.
SPF is whitelisting which mail servers send mail from your domain
DMARC is how receiving ends should report how your mail is delivered (or not).
Unless you're actively spamming (ahem, "marketing") to large volumes of users, you should have no trouble at all getting mail delivered to users.
If you are doing mass email marketing, stick with your core competence and let someone else handle the mail servers.
Here's some background https://wiki.list.org/DEV/DKIM
However, even that can only carry you so far. Gmail is screwed up in how they handle reputation, especially with regard to IPv6. The exact same message delivered to Gmail via IPv6 will get refused versus sending it via IPv4 will work fine.
This problem has existed for years. There are even standard hacks for postfix to cause all mail addressed to Gmail to be sent via IPv4 only, just to avoid this problem.
And any domain hosted with GAFYD is likewise screwed.
You can't know what your false positive stats are, because you don't have access to all the remote SMTP spools or logs on the 'net. All you can know for sure is the false positives that you find out about. But you literally have no way to tell how big the iceberg is under the water.
Aren't DMARC reports supposed to tell you that, especially ruf?
After fifteen years running my own mail systems, I have finally given up. Ever more outgoing mail was getting lost, despite everything being up to date and done by the book. The arch villains, over and over, were Gmail and the MS thing - Hotmail, outlook.com, Live, whatever they call it.
Same here. I've been hosting my own e-mail for two decades (as of this year) and I gave up earlier this year. My spouse's e-mail to friends who use GMail--something like 80% of them--would, every couple of months, disappear into the recipient's spam folder or sometimes just disappear entirely. My logs would show a successful delivery but the recipient couldn't find the message anywhere. Occasionally messages would start going into junk in the middle of a conversation.
Nothing I changed would permanently fix it. I'd send from a different IP in my block (something.something.194 instead of something.something.193) and mail would flow again...for a couple of months and then stop. My domain registration predates Google's existence by 18 months. I've been sending from the same IP subnet for six years. My colo provider is actually so small that I'm the only customer left on my "neighborhood" /24 (they've recently asked me if I want the entire block routed to me for the lulz). I have SPF, DKIM, and DMARC all set up and they've not changed for at least two years. The amount of mail outbound from my server is (was?) so little that I could tail the outbound mail log and keep up with each message being sent.
None of that mattered. I even signed up for Google's Postmaster Tools but, and this really cooked my goose, I send so little e-mail that I don't qualify to show up on their reports. I tried contacting Google but hahahahahahahahahahahaha--wheeze--hahahahahahahaha.
No other recipient had a problem with me. Messages were successfully delivered to Microsoft and Yahoo just fine. Only Google had a problem with my e-mail.
I finally caved. Google has SO MUCH of the e-mail market that unreachability to them is a catastrophe...and they know it. I don't have the leverage to make them play nice.
Thankfully, Fastmail does so I've changed to them. For the first time since February 1997 my MX doesn't point to my own server and that makes me a little sad.
There is a reason. It makes people switch from own email servers to gmail.
Do you have an idea what's triggering the false positives – keywords, IP reputation, etc?
The overall rate relative to my total e-mail volume is quite low, but it's enough that I have to check far more often than I'd like.
It wouldn't be a problem if I had a reliable way of whitelisting addresses, but none of the methods seems to be consistently reliable.
Note: We recommend that you do not mark your Spam/Junk Mail folder to
automatically learn "As spam". This can create a false positive feedback loop.
Imagine an email is incorrectly classified as spam, put in your Spam/Junk Mail
folder, and then learned as spam. That means future emails that aren't spam are
now more likely to be incorrectly marked as spam, sent to your Spam/Junk Mail
folder, and learned as spam. Only mark folders to learn "As spam" if they're
folders you manually move email to.
[0]: https://www.fastmail.com/help/receive/stopspam.html?u=b83140...
I've found the never mark as spam filter option to be reliable – amusingly I've also had the problem with Gmail flagging the Google Groups spam reports as spam, and this halted that.
So I have to constantly login to their site only to check the spam-folder (talk about convenience). I've setup rules to try to do my best to NEVER send anything to the spam folder but even with catch-all rules gmail still think they know better than me.
Their support for real (external) mail clients is also quite bad, I don't see why so many people in tech use gmail.
One other thing that I kinda sorta like in GMail is labels (as opposed to folders). As I understand, it's not a popular feature, because there's no reliable way to map it to IMAP? So anything that assumes that IMAP (rather than a custom client or web app) will be used as a mail reader, will probably only provide folders.
I have absolutely no problem with Fastmail's spam filtering. Maybe Gmail changed things since I switched, around 3-4 years ago, but with both, I used to get 0-1 spam emails per month, though I haven't gotten any on FM in the last few months.
Oh yeah, and MX records take some time to update within popular services, so some gmail users might be able to reach you while others will get a user not found error. If you think about it, it makes sense, but it drove me nuts the first time.
While it would be quite straightforward to clone an almost pixel-perfect version of Gmail, there is something in hacker culture that makes this kind of project unattractive. I don't really understand why.
Instead what we end up with when someone starts a new mail app is some half-cocked design that makes sense for that particular user, who is always an engineer, and the result is suboptimal for pretty much anyone else.
So the project fizzles and dies before it even gets started. This happens over and over again even for larger and more visible projects like Mailpile
I think you deleted the last "."
Gmail is _fast_. The browser doesn't reload when you navigate around and when you click on something it gets loaded very quickly. You can't just put a shiny UI in front of an IMAP server and get this sort of functionality, you have to write your own mail server with a rather intense database backing it. I have 70,000 emails spanning a decade (mostly useless) and searching is a breeze. Everything works fine and quick with a huge volume of data. When I send messages they're actually sent in a second or two, when I click next message the next message appears in a fraction of a second. The UI is the frosting on the cake. It's critical that it's very fast, but the underlying mechanics have to be very fast too, and that's not just a manner of copying pixels.
Have you tried FastMail? They use Cyrus...
> you have to write your own mail server
... but I concede they (FastMail) do sponsor most of Cyrus's development.
It is fast when a huge amount of JS gets loaded and latency times are low. But with high latency? A horrible experience and if it even works I'm grateful.
It is pitifully slow when on a limited connection. And the contortionist-like procedures needed to log in from an unfamiliar IP don't make it slow, just largely non responsive.
Zoho is sorta free and good for BYO domain-based basic web/email.
custom bundled ldap server with lots of java custom goop. Not fun to support at scale
That's what I thought, until I discovered Fastmail. I'm now a happy family account holder, enjoying great support from human beings, and happy that my children won't be data mined for advertising (at least, not in this particular case).
I'm not kidding about the support either. One of their developers was on Github helping out with an issue on an open source tool that, in the end, proved to be a problem with their implementation of a protocol. Fix was in Fastmail staging a week later, and in prod a few weeks after that.
>Unlike Google’s consumer offerings, which may show ads, we do not collect, scan or use your G Suite data for advertising purposes and do not display ads in G Suite, Education, or Government core services. We use your data to provide the G Suite services, and for system support, such as spam filtering, virus detection, spell-checking, capacity planning, traffic routing, and the ability to search for emails and files within an individual account.
Sure. I was talking about their "consumer" offerings though.
I agree it's unlikely, I'm just saying your quote leaves several possibilities open in unkind interpretations.
We might as well decomission decentralized e-mail and move everyone on to GMail and FB Messenger.
Sigh.
We (the Cyrus IMAP board - https://www.cyrusimap.org/overview/cyrus_bylaws.html) are looking for a new home for the project which is independent of any one single company. I'd hoped to have more news to post before this was made quite so public!
This sentence doesn't really make sense. When comparing two things an advantage of one thing doesn't "outweigh" a disadvantage of another. Those are both negatives for the second thing.
One of my employers uses Zimbra on their own servers. Does a Web based calendar and email and seems to work OK.
Reminds me of my university's slow-slow-slow Sun thin clients that they had all of the school in 2004. Logging in and getting into Netscape at peak times would take minutes. Luckily, I knew how to use Pine.
it makes you pine for oldschool outlook.
I left CMU & the project 14 years ago. It's nice to know that some code lives on.
Sad that gmail now owns ~50% of all email now though.
(I made that number up. Anyone have any idea how close it may be?)
While Roundcube is OK, and personally I like Zimbra as the best turn-key system (webmail+calendar, which we use for our company), their UIs are a bit outdated and feel like second-class solutions.
As the article stated they have a choice of Gmail or Exchange.