Hacker News Reader

>Everything that someone else could do on an email server that they run for you, they could just as well package into, say, an install CD image, which you then could use to run your own functionally equivalent, hopefully well-configured and well-updated, mail server.

Gmail won't give you their entire software stack (also good luck running it even if they did). Neither will any of the other big players.

Are you saying that gmail are the only ones who know how to run a mail server?

You were claiming "they could just as well package into, say, an install CD image" which is likely not going to happen, so he was just giving an example.

The perfect spam filter Gmail has its probably based on data and ML analysis, not a simple software you can put on a CD and install on some little server; so even if someone who works there wanted to help you they probably can't do it.

GMail's "perfect spam filter" is so bad that it is the reason my institution moved away from Google services. It's also the main reason so many people consider it 'impossible' to host your own email.

It's anything but perfect. It's a black box, with no user-serviceable parts inside. Completely useless for any organization with needs that deviate from Joe User.

How would you implement a spam filter?

You can sort of turn off spam filtering on incoming emails but I don't most people or organizations would want that.

> Completely useless for any organization with needs that deviate from Joe User.

Maybe but being slightly over aggressive means a world in terms of user happiness. Think about email before Gmail. How much junk did you see in your inbox? There is a lot LESS today. Joe User would be very happy if they dried to think about it for a second.

> How would you implement a spam filter?

What do you mean "implement"? You are aware that there exists free spam filtering software that you can install, like, say, spamassassin (which has been around longer than gmail)?

> Maybe but being slightly over aggressive means a world in terms of user happiness.

Dropping legitimate emails is good for user happiness? You have strange users. I imagine if the post office decided to throw away some letters "slighty over aggressive[ly]" ... there are people who would be happy if that happened?

> Think about email before Gmail. How much junk did you see in your inbox? There is a lot LESS today.

I see maybe one to three spam mails per day, with nearly zero false positives (maybe one per year). And that's without gmail. Obviously, I could reduce that to zero spam, at the cost of increased false positives, which is not a sensible option as far as I am concerned, deleting three mails per day isn't really all that much work.

> You were claiming "they could just as well package into, say, an install CD image" which is likely not going to happen, so he was just giving an example.

What's the point of pointing out the one example among millions of email server setups that is most likely to not be published? Especially so, given that that would be about the most useless setup to replicate, as noone needs the scalability of gmail for their own setup, and thus the complexity that would come with it.

> The perfect spam filter Gmail has its probably based on data and ML analysis, not a simple software you can put on a CD and install on some little server; so even if someone who works there wanted to help you they probably can't do it.

If you think that gmail's spam filter is perfect, I have a very simple even more perfect spam filter: Just throw away all emails. Gmail has massive false positive rates, that's not a perfect filter, that's just a filter that throws away a lot of emails.

If anything, the control that google has over which emails it arbitrarily labels as spam is completely unacceptable, especially so given that they don't even accept liability for incorrectly filtered emails.

We could probably build a distributed version of a spam filter by using a fancy online (single sample batch) training algorithm and combining models. I'm tired, so that exact solution may be infeasible, but I'll bet something similar is.

No, only that their security is better than anything you'll manage alone.

Hardly. Not providing a web interface or allowing any SMTP relay at all immediately comes to mind.

And not providing things like password recovery schemes that seem to be one of the most common way people have their accounts compromised.

1. Security is not some magical property of software created by large corporations. Neither are they perfectly secure, nor is it that hard to even write your own secure mail server, alone.

2. They are by themselves a major security problem because they necessarily have access to all your email. And what's worse, if everyone is using them, they have access to everyone's email. That's a huge amount of power waiting to be abused, if not directly by them, by someone bribing them, putting pressure on them, or simply by infiltrating them.

> Neither are they perfectly secure, nor is it that hard to even write your own secure mail server, alone.

The first part of that may be true, but the second part is definitely false. It is very hard to write your own secure anything-connected-to-a-network.

> The first part of that may be true, but the second part is definitely false. It is very hard to write your own secure anything-connected-to-a-network.

No, it's not. Where did you get that idea?

From years of using, writing, and reading about network software. I'm not sure if you're being obstinate or naive, or if you actually have some silver bullet for writing secure network software, but if it's the latter, I'm sure lots of people would love to pick your brain!

There is no silver bullet, but one very helpful principle: Avoid complexity.

But let me ask you: What are the big security problems that you see with a personal email server that you think are difficult to get right/to avoid?

All the same things that are difficult to get right in any other chunk of code that handles arbitrary input from across the internet. I have a couple tomes on this subject: "The Art of Software Security Assessment" is 1200 pages long, "The Web Application Hacker's Handbook" is another 900 pages. Nobody has solved all the issues discussed in those (and many other) books in such a way that makes this no longer "hard".

Note that I'm not at all claiming this stuff is impossible or a total lost cause, just that I think it easily passes the bar for "hard", and is certainly tricky enough to get right that I personally don't think it's worth the effort to run network-connected services unless they're for a business that I'm hoping will pay for my time and effort somehow.

> All the same things that are difficult to get right in any other chunk of code that handles arbitrary input from across the internet.

That's about as unspecific as you can be, isn't it? So, let me give the equally uninformative answer: All of those are either easy to prevent, or they lead to a DoS, which isn't a big concern for a personal email server.

> "The Art of Software Security Assessment" is 1200 pages long

Haven't read it, but by its TOC, it's mostly irrelevant to the given problem (except maybe that it teaches you to not write you personal mail server in C, who would have thought ...).

> "The Web Application Hacker's Handbook" is another 900 pages.

Didn't I say something about avoiding complexity? No, you should not build your mail server as a web application, in case that wasn't clear.

> Nobody has solved all the issues discussed in those (and many other) books in such a way that makes this no longer "hard".

Well, how about you give an example of one problem that you think is still hard enough to be a major obstacle to implementing you own secure personal mail server?

> [...] that I personally don't think it's worth the effort to run network-connected services unless they're for a business that I'm hoping will pay for my time and effort somehow.

How is it that you suddenly switched to "services"? I thought we were talking about "network software"? Am I supposed to believe that you are not running a web browser unless it's for a business that you're hoping will pay for your time and effort somehow? How about web applications? Or what is your justification for a distinction between "services" and "network software", as far as possible security impact is concerned?

As for your final paragraph about word choice: I'm not sure what you mean, I'm just talking about any software that is running somewhere and accessible across a network.

As for the rest of it... there was a time when I would have bothered trying to delineate some specifics for you, thinking I was providing some educational service or something, or that maybe I'd learn something from you in the discussion, but I'm getting the sense that if I did that, you would just be dismissive of all those things as "not a big concern", "because of complexity", "because of C", etc., and I would be wasting my time.

There are plenty of places you can go read about this stuff if you want to, but I don't think you do, I think you're just looking for an argument. Good luck to you!

> As for your final paragraph about word choice: I'm not sure what you mean, I'm just talking about any software that is running somewhere and accessible across a network.

Well, a web browser is accessible across a network. An email client is accessible across a network. What fundamentally distinguishes a mail server from either of those that justifies rejecting to run one of those yourself outright, while presumably being fine with the others?

You seem to have some implicit assumption that a piece of software that opens a listening TCP socket is somehow much more vulnerable than one that doesn't. I don't think that that is justified, and if anything, gives you a false sense of security.

> but I'm getting the sense that if I did that, you would just be dismissive of all those things as "not a big concern", "because of complexity", "because of C", etc., and I would be wasting my time.

I am not dismissing anything, I am just pointing out why those aren't problems. If you disagree, feel free to disagree. And maybe make an actual more specific argument than "there are a lot of things one can do wrong".

I mean, how is it relevant that there are tons of ways to screw up buffer handling in C when the obvious solution is to just not write your own personal mail server in C? Yes, it is hard to write secure software in C. Very hard. Yes, there are cases where there are good reasons to write network software in C despite that. Is your own personal mail server one of them? No.

Or how is it relevant that the web is a hugely complex mess of inherently insecure APIs when the task at hand is writing an email server? Just don't write your email server as a web application and you don't have to deal with any of the myriads of security problems that plague the web.

> There are plenty of places you can go read about this stuff if you want to, but I don't think you do, I think you're just looking for an argument. Good luck to you!

Thanks, but I think I have found enough vulnerabilities to not need any introductory books to understand how people constantly screw up security in their code.

Lookit, this thread started because you said "nor is it that hard to even write your own secure mail server, alone", which I disagreed with, saying that it is in fact "very hard". You seem to have a definition of the word "hard" that is just much closer to "infeasible" than my own, which is nearby "takes a good deal of expertise and / or effort". It really isn't very interesting which of us is more "right" about what is and isn't "hard".

I honestly hope you're right that someday I'll work on a project and find myself thinking "wow, it wasn't even that hard to get the security story right on this", and I guess that day has already come for you, but it certainly hasn't for me.

> You seem to have a definition of the word "hard" that is just much closer to "infeasible" than my own, which is nearby "takes a good deal of expertise and / or effort".

Well, maybe. But I don't really think it takes all that much expertise. Maybe it does, in the sense that there are so many tools out there that you in principle could use to try and build an email server, and without any expertise, it might be hard to tell which tools to use, and, more importantly, which ones to avoid.

So, without expertise, you might end up stuck in the local minimum that is C, say, which in turn then requires a lot of expertise and effort to build something secure, because there are just to many pitfalls to be constantly aware of if you want to pull it off. But if you manage to avoid that local minimum and use a memory-safe language with garbage collection, suddenly, avoiding buffer overflow exploits is trivial. You don't even need to understand what a buffer overflow is and why/how it can be exploited. They simply don't happen, without any detailed expertise needed.

The thing is that you very well might end up writing your email server in a memory-safe language by accident. Writing a secure email server in C by accident, though? Extremely unlikely.

Also, many possible security problems that plague software simply don't apply to email servers. The web is a mess and difficult to build something secure for. But email servers don't have to deal with the web, so you cannot even accidentally stumble into those problems when you are writing an email server.

> But I don't really think it takes all that much expertise.

I think you may be living in a bubble. Building any working software takes a lot of expertise. Maybe you just mean "expertise in addition to that necessary to build any working software". In which case, maybe I don't disagree with you so much. But again, we're just debating the definitions of words like "hard" and "expertise"!

There are more security issues to be concerned about than buffer overflow. Just as an example, your email server needs authentication and authorization, which are also tricky to get right. There are plenty of logic-level security issues like that for which just choosing a memory-safe language is not a solution.

Security is not "magical," no. In fact it's barely even a "property of software." Instead it's a property of a system. Some of the elements of this system are software. We can perhaps make some guarantees about the properties of this software, but these alone do not make the system secure (systems that are provably secure end-to-end are vanishingly rare, if they exist at all). Part of the system is configuration and data. What cryptographic keys and other systems do you trust, and how do you guarantee that trust? Part of the system is human process -- when a customer calls support, does someone give up private information or allow passwords to be reset? If the government knocks on the door of your cloud hosting provider, will they cooperate freely? When your linux distro announces a security issue to a trusted list ahead of a public announcement, do you have someone on that list ready to respond to the issue before your server is hit by a 0-day?

There's a lot that goes into making a secure system. And there is considerable economy of scale -- a large company can afford to have 24/7 monitoring and issue escalation processes. You cannot.

> We can perhaps make some guarantees about the properties of this software, but these alone do not make the system secure (systems that are provably secure end-to-end are vanishingly rare, if they exist at all).

Which applies equally to gmail.

> If the government knocks on the door of your cloud hosting provider, will they cooperate freely?

Which applies equally to gmail.

> When your linux distro announces a security issue to a trusted list ahead of a public announcement, do you have someone on that list ready to respond to the issue before your server is hit by a 0-day?

Which can be automated.

> There's a lot that goes into making a secure system. And there is considerable economy of scale -- a large company can afford to have 24/7 monitoring and issue escalation processes. You cannot.

And you don't need to. If you don't need to do this with your email client, you also don't need to do this with your email server.

Also, there is also a major economy of scale to compromising google instead of your personal email server.

No, of course, there is no "perfect security" (whatever that is). And that's true for both gmail and self-hosting. But it is perfectly possible to write an email server that's very unlikely to be exploited surprisingly into remote code execution as a single developer.

time to action is a byproduct of awareness. if i run my own server by myself and am asleep, my reaction time to a security incident is severely limited by my sleepfulness.

Except that you sleeping or not doesn't really change all that much whether you notice the security incident in the first place. It's not like you have some sixth sense for security incidents that just stops working while you are asleep, is it? And neither has Google. Or anyone. Now, if someone has been siphoning off your emails for months, does it matter whether you notice today or tomorrow?

If you are talking about easy to detect abuses, such as participation in a DDoS attack, say: (1) those aren't really that bad for you, and if you fix them within 24 hours, not even for the attacked, and (2) your upstream provider might just notice and shut off your machine until you are awake if it's really bad.

But with all that said: Neither of those is particularly likely to happen anyhow if you take care of installing updates in time. What gets owned are high-value targets, bad passwords, and machines with ancient software (as in: security update has been available for a few years), the latter two of which are trivial to avoid.

  Everything that someone else could do on an email 
  server that they run for you, they could just as well 
  package into, say, an install CD image, which you then 
  could use to run your own functionally equivalent, 
  hopefully well-configured and well-updated, mail server.

https://mailinabox.email/

The only remaining trouble is having to look up all those mail servers with improper DNS setup. No SPF, no DNSSEC, no keys.