Comment by phunel | Hacker News Reader

Comment by phunel | Hacker News Reader

I've been struggling with the tradeoffs in keeping Google as my email (among other things) provider. I own a few VPS's and have plenty of hardware that could serve as a private server. What has kept me from hosting my email up to this point is the fear of being inadvertently blacklisted by something like Spamhaus, downtime from Comcast (my ISP) making service intermittent, and generally having the email server management take over my life. In short, the same issues that affect many here by evidence of the recent chart of Y Combinator companies' hosting decisions (http://jpf.github.com/domain-profiler/ycombinator.html?2011).

I could host my own email on my VPS, but going through the trouble of that - it seems to make sense to just go ahead and host on a box I fully control.

To this point I admit I have had no issues with gmail or Google apps, both of which I use. But the landscape seems to be changing and I'm curious if anyone is considering migrating to their own hardware and what tools, etc. they are employing to make that process easily manageable as well as effective. I use email for sensitive business discussions, but the security required is tempered by the obvious fact that if the email doesn't get there, it's less than worthless.

I do have PGP setup, and I thought that this would be a good tradeoff while using a third party provider - but it is nearly impossible to keep partners and clients active in using it, if they use it at all.

The concept of the Freedombox (freedomboxfoundation.org) seems interesting, but academic at this point.

Anyone running their personal email servers care to comment on the overhead? Would a simple Postfix, Dovecot, Roundcube install with Spamassassin suffice? What are the pitfalls? Not worth the tradeoff of using Google Apps?

I'm thinking about keeping my websites on a VPS and using a MX record to point to a private, dedicated email server I keep in my house or other private property. Anyone running a similar setup?

So, if you want to relay mail out of your MX, it's going to be flaky from a DSL or cable line. The reason is because much of the email coming out of those IP ranges is virus-sent spam, many mail servers block those IP ranges. (ISPs are doing a pretty good job filtering port 25 these days. The only really annoying spam I get is from Sprint's mobile network. I would block those IP ranges, but I want to be able to relay mail from my Sprint mobile phone :)

I run a mail server on my Linode and have never had trouble sending mail. It's easy to see if you're on a blacklist and take corrective action, though I've never had to do this.

I recently switched from Postfix to Exim4 and like it a lot more. The spam checking is much better integrated and lets you reject messages at DATA time with full spam information, so real messages that are auto-rejected at least bounce with an informative message.

Finally, if you don't like delayed mail, get a backup MX. It's very easy to set up for your friends that run their own primary MX, and you can return the favor. (I buy a backup MX service, but run backup MX for people that have asked.)

Good idea on the backup MX.

I do have Exim4 setup on a Linode for outgoing status updates (mainly Fail2Ban), but have yet to use it in lieu of Gmail/Google Apps.

As mentioned previously I'm a bit nervous about leaving my mail on a third party box - though Linode has been nothing but a reliable service to date.

What's your reason for being worried? The government can get a search warrant for your house just as easily as they can get one for Linode's servers. The only difference is that when Lindoe's servers are seized, they eat the cost. (And you aren't awakened at 3am by dudes with guns. Not that this has happened to me :)

If you're worried for backup reasons, just sync with offlineimap. I do this to have a faster local cache (since I consider Linode more reliable than my desktop computer). (Even gmail is fine if you back it up.)

I'm far less worried about government intrusion than I am generally of corporate intrusion. The relationship, as it stands, consists of me willingly giving all my private communications to a third party I have no standing with. I'm questioning my own behavior in that equation much more than any scenario where a government entity serves and act on a warrant. I don't ever expect to be in that position, nor have I, but I have been in the position where my commercial and private correspondents has been violated by companies I am in competition with and employees who I no longer work with. Good luck with the lawsuit, the damage has been done in those situations.

Take a look at Google's privacy principles here:

http://www.google.com/intl/en/privacy/

I can't speak for Google, of course, but my own take is that privacy is taken very seriously internally. Perhaps more seriously than anything else, actually. Google is generally open with employees with respect to source code, financial data, and access controls, they are not that way with user data. I don't have access to it, and couldn't get access to it unless it's vital to the success of my project (and then, only for a limited time period).

I even have a sticker with the 5 privacy principles stuck to my monitor's base.

Don't confuse "internal secrecy" with "user privacy".

That user data is company property and its secrecy is what gives it value. Furthermore, if users caught wind of just how much of it exists that could jeopardize its source.

We tell users exactly how we use their data. We're even running an ad campaign on the NYC subway (and presumably elsewhere) about how we use user data in non-intuitive ways.

Here's the relevant section:

http://www.google.com/goodtoknow/data-on-google/

Thank you for responding. I notice that that page talks primarily about search terms (and in noncommittal terms about how much Google values privacy) and it doesn't seem to apply to the other parts of the expanding portal.

Just to pick one example, where does it say what you do with the data you collect on users when they click a link from YouTube. E.g., the page http://www.youtube.com/user/khanacademy has a link that displays on the page as

    Website: http://www.khanacademy.org

But when you click on it, it actually goes to

    http://www.youtube.com/redirect?q=http%3A%2F%2Fwww.khanacademy.org%2F&session_token=oJd2SDBY1T6kY0eXgYddT3firut8MTMyNzE5MDY5NkAxMzI3MTA0Mjk2

The information on users' link-clicking habits being skimmed from that little trick (and the many others like sourcing script from Google-controlled domains on most sites on the web), is it associated with the G+ identity? Or not? Where is this documented?

consider people who dont live in the us?

I know this is not an answer to your question, but the way I address these issues (especially since I have to do more system administration that I'd like to already) it to do business with laser focused companies. I.e. email from Fastmail (although they recently got bought by Opera, but so far besides some growing pains as they build out their base system this has not been a problem), rsync.net for backup (yeah, more expensive per GB than S3 but it does exactly what I want and that was "tornado tested" after the Joplin, Missouri May 22nd 2011 EF5 tornado) and Skype (except they were of course bought by Microsoft ... I and my friends are looking for alternatives since we know we'll need one sooner or later).

Companies that do only one thing well are often a better bet than companies that do many things at whatever level of quality (and while I don't advertise, I do perceive the quality of Google's search "product" dropping due to things like removing + from term modifiers).

I wouldn't worry about Google's software quality. We do a really good job on that front. Every machine that Google owns is capable of running Gmail, and failover is a planned use case. Data centers can fail, machine can fail, code can fail, and everything will still work for you. (The process is fully automated and regularly tested.) We have an entire job classification for people whose job it is to build automated testing infrastructure -- not write the tests, but write tools to make writing tests easier. We have people to write the tests, too, and of course every developer writes tests. Every code change, no matter how small, is reviewed by one other developer who knows the relevant code deeply. And all changes are tested internally before they ever go public.

So basically, it's pretty rare that you would lose your email. It's happened before, but everything was restored from tape. I run my own email server for fun, but I'd be lying if I said that I did it better than Google.

(Note: I was more referring to the fact that your various offerings are of varied level of quality, i.e. you don't have the laser like focus on remote storage like rsync.net, but your reply was entirely useful anyway.)

I don't worry about that level of quality and have e.g. used Google Docs for some very important things, I worry about the Product Manage level of quality. No in the trenches Google programmer up and decided to remove "+" as a search term (and were mulit-word double quoted phrases previously mandatory search terms?).

There's also the political problems that come with size and scope. Google is in the cross-hairs of many governments, and looking at history you can't say it's impossible that ugly things won't happen, e.g. a split that would put Gmail in a less capable smaller company.

Compounding that, to a degree uncommon in the industry, Google has made an all in bet on the Democratic Party and Blue State values. That strikes me as ... unwise in county that for many years has self-identified to Gallup as being 40% "Conservative", 40% "Independent" and 20% "Liberal".

To be fair, + wasn't really doing much anyway. I found out the other day that Google's segmentation algorithm is so good that you don't even need to put spaces in your search; searching for "givemealistofsearchengines" does!

Well, I was using it!

To be fair, I'm not a normal case, I started creating and querying full text retrieval systems for customers and myself in 1991 when I started my half-decade of document imaging work.

But I did use +, e.g. I'd do a query and get too many or the wrong results because one term wasn't being weighed well and then I'd add a + in front of it. Changing that to wrapping it in double quotes would be only an annoyance if the change wasn't entirely gratuitous (as I understand it) to the Google search function.

And, yeah, the segmentation algorithm is very good, although I haven't torture tested it ^_^.

(Side notes: LinkedIn is the only "social network" I'm likely to use in the foreseeable future, Facebook is anathema (they seem to have inverted your "don't be evil" motto) and Google+ is way too dangerous to use because being kicked off it kills the rest of your Google accounts.)

being kicked off it kills the rest of your Google accounts

Where did you saw that? According to Horowitz (one of their VPs), it doesn't:

    When an account is suspended for violating the Google+ common name 
    standards, access to Gmail or other products that don’t require a Google+
    profile are not removed,

This would seem to be a change in policy, and "violating the Google+ common name standards" (which I would not have intended to do or likely have accidentally done) is certainly not the only way to get kicked off of it.

The more Google intertwingles their services the more cascading policy "failures" can put you in a world of hurt. Hence my one company, one service policy. In the case of Google (ignoring membership in some Google Groups, but to a non-Google address) it's Google Docs. (I'm not counting the non-sticky ones like search and Maps.)

I run my own mail server for my personal mail at home off my DSL line. Most blacklists will block residential IP ranges, but it is easy to get around that. You can simply set up a smart relay to another SMTP server, probably your ISP's. When your mail server sends mail to the outside, world, it doesn't send directly to the recipient's mail server, it sends it to your ISP, and they relay it out.

Thank you. This seems like a fair tradeoff. In my case I imagine I'd just use my VPS as the relay, rather than my ISP. Just so I understand clearly, incoming mail will not be an issue and can be sent directly to your personal box, but relay the outgoing via your established ISP/VPS/etc?

Exactly. Mail to you goes to where ever your DNS MX record is pointing to (obviously in my case that is my home IP) and local delivery is configured how ever you choose to do local delivery. Sending is simply configured how ever smart relaying is configured on your MTA of choice. All mail sent from your MTA is sent directly to your smart relay and goes out to the world from there.

There is a trusted type of relationship between your MTA and the smart relay so the smart relay is not set up as an open relay. If your smart relay is an open relay, it may end up on blacklists as well, for different reasons than originating from a residential IP block.

I just checked to make sure, and creating new email addresses using google apps does not require creating a g+ account.

I've been happy with using google apps for my email server and have had no trouble staying in the free tier. It's clearly not for everyone, but it sounds like it will involve far fewer headaches for you than managing your own server.

True, but Google Apps tends to follow Gmail's lead, albeit at a delayed pace.

I agree that Google Apps is the best of the bunch, and much easier to manage than your own server - but the privacy concerns have just nagged at me for years. I keep getting back to the core issue: is giving this (or any) company the sum of all my written communications in exchange for ease of use worth it? It's something I'm still struggling with.

you don't feel covered by Google's terms of service?

Reply on news.ycombinator.com