Why do self-respecting hackers use Gmail & Co? | Hacker News Reader

Why do self-respecting hackers use Gmail & Co?

laforge.gnumonks.org

225 pointsgst262 comments

1. Running a mail server is an unbelievable pain in the rear from an IT perspective. Note that by "mail server" I mean a good setup with spam filtration, webmail, SMTPS, IMAPS, etc.

2. Really good rich webmail. I personally use Mac Mail.app most of the time, but having that rich webmail is nice.

3. Filters mail at the server side.

4. Very good spam filtering... I post my gmail addresses on web pages with no obfuscation and get maybe 1-2 spams per month.

Cumulative: it's one more thing I don't have to jerk around with. It just works.

The one that got me was spam filtering. Google does a pretty good job at that, and I was really drowning in it.

I think they deserve a lot of credit for creating a product that both my mother (definitely not a 'power user') and I (previous email client: Emacs' Gnus) can both be reasonably productive with.

It's really easy to install SpamAssassin and point your MTA at it. Even the default configuration is highly effective.

I did that, back in the day, but it just wasn't enough.

One could always mooch off of Gmail's spam filtering... Use SpamAssassin in conjunction with a gmail account that is only used as an attempt to attract as much spam as possible. Then just download the spam folder over IMAP and use it to teach SpamAssassin...

Gmail's spam filters are so good because they have a network effect going though, I doubt one spam folder would achieve much, you need a much bigger sample of all the spam than one account could ever attract

Pretty sure you are making some unwarranted assumptions here – other commenters mention that SpamAssassin can download updates to itself regularly, so you aren’t starting from scratch with a completely untrained Bayesian algorithm.

Respectfully, no.

The primary issue in spam filtering is not that we don't have training corpora, it's that spammers are very efficient at finding holes in your system.

One example is when the Hotmail team first enabled keyword filtering. When the spammers found out experimentally, they began injecting HTML comment tags into high-weighted words like "free", breaking the model. When the Hotmail team took steps to combat this problem, the amount of spam that employed this technique dropped from 5% to close to 0%[1] in a matter of days.

Spam detection is complicated and hard.

[1] Hulten, G., Penta, A., Seshadrinathan, G., and Mishra, M. Trends in spam products and methods.

I'm not saying it's perfect. Obviously having a network effect allows you to do things like flag messages that are exact duplicates over multiple sent-from and sent-to addresses.

I've seen this happen where 1 or 2 messages hit my inbox, but the next 20 or so are in my spam folder.

"The primary issue in spam filtering is not that we don't have training corpora, it's that spammers are very efficient at finding holes in your system."

Finding holes on sites like gmail and hotmail is actually much easier than on private sites, because the spammers no doubt have accounts on gmail and hotmail and can test what gets through and what fails and tweak their algorithms until their spam gets through.

Spammers don't have that luxury on private servers, so they have to spam blindly. So in this respect private servers have an advantage over gmail and hotmail.

On the other hand (as has been pointed out many times in this thread), gmail and hotmail have the advantage of advanced spam detection algorithms and virtually instantaneous feedback from millions of users.

> So in this respect private servers have an advantage over gmail and hotmail.

Besides, spammers care more about being able to circumvent Gmail's spam filtering than they care about circumventing your own system.

That's a perfect example of how spam detection is harder for a centralized server operator like Hotmail or Gmail than for somebody who's running their own server: if I were to add a delete-HTML-comment-tag preprocessing phase to my own copy of SpamAssassin, spammers would have a difficult time detecting that, and so would not be able to adapt. (Because none of the dozen or so people who can get mail on the server are spammers.)

SpamAssassin takes care of this for you by automatically downloading new filtering rules. You host the spam filtering software, but the actual rules are out-sourced by default. It even includes configuration for multiple DNSBLs, DNSWLs and RHSBLs by default. You can add your own local rules if absolutely necessary too.

I use SpamAssassin server side, and Thunderbird's own filter polishes off the rest. I see perhaps 1 spam mail per week (from the literally tens of thousands that hit my server).

It's still a service which you have to maintain and there's constant config & upgrading required since some spammers are smart enough to test their messages with it first.

I've run SA professionally for modest (low hundreds) of users. It works reasonably well but it's a job and I'm not paid to do it personally.

SpamAssasin will auto-update its rules if properly configured. I have touched my SA install in years.

nupark2 is correct. There's less maintenance involved in running SpamAssassin than there is in running Firefox.

Would you pay for this? For the last 3 years I've been creating a platform for this kind of servers for a small spanish ISP. It has grown to 15 Openvz hosts. A rails app connects to the host to create the virtual machine. Centos is installed and configured with puppet. A rails app is installed in every server and used to configure the accounts, domains, etc.

Postfix, Cyrus, IMAPS, SMTPS, POP3S, HTTPS, Sieve for filters on the server, roundcube as webmail (on a new product I would pay for @mail), decent spam filter (right now we run our own system based on commtouch and spamassasin), etc. You can export the whole cyrus mail storage folder or export mail to other server with imapsync, same thing with the database that contains the account, domains and aliases, so you are not locked. Billing can be done by hard drive space or accounts. They are also good as outgoing mail servers because you are on your own ip.

I run everything by myself, right now we have 126 servers with about 12300 accounts and thanks to puppet it's really easy to admin. I've always thought that I should try to do it by myself.

Why pay when you can get it from Gmail? :)

Aren't you a little bit uncomfortable giving Google all that data? Do you keep a back-up??

Mail.app is keeping a local backup (using imap)

Not even a little bit.

Total number of times Google has lost my data: 0

Total number of times I have accidentally lost my own data: Way more than 0

Total number of times Google has in any way caused me harm by having "access" to my e-mails: 0

Total number of times I've been glad Google used intelligence gathered from my e-mails: Many (pre-populating Google+ circles, training a great spam filter, Priority Inbox, great search, decently appropriate and well-targeted ads)

What happens when Google suspends your account and you loose all your email, docs, etc? Your online identity (you no longer have your main email address) gone?

Own your MX record and use forwarding and/or Google Apps for Your Domain?

That's what I do, and it gives you a portable email address, but doesn't address the other concerns. Ultimately, email is a communication medium, not a filesystem. Using IMAP folders for long-term storage is tempting, but it's off-label.

You only have ~100 email accounts per server?

yes, it's pretty stupid but it's the product that the commercial team sells (your own small managed private mail server). Some of the servers have more than 2000 accounts but others have 6 or 10... A few years ago, until 2006, the servers where small appliances installed on the client's office. Now they work the same way but mostly virtual and since most of the clients are small bussiness.. some have really few accounts.

On our shared mail system we have about 40k accounts in 2 cyrus servers (with 2 more for HA)

The primary reason I won't outsource this service is trust. I don't trust anyone with that data.

Access to my email allows for resetting all my passwords, seeing all my contacts, and tracking literally everything about my day-to-day business and personal life.

I hope all your email is encrypted, because otherwise every hop the email traveling to/from your server takes has access to all the data you hold so dear.

Actually most SMTP now talk SSL to each others, so while the hops (generally, just 2: sender and receiver) could also be compromised, the attack surface is smaller than one would think.

"most SMTP now talk SSL to each others"

Not to dispute your claim, but how do you know this? Is there some sort of global, internet-wide survey of SMTP servers somewhere that supports what you say?

It's easy to tell which mail servers are using SSL by looking at your mail server logs. He probably only cares that most of the mail servers talking to him use SSL (which is also true for every mail server I have administered).

This is a much smaller attack surface than a centralized data store holding static e-mail archives and accounts for a large number of users, open to at-will offline browsing.

DNSSEC and SSL further reduce this attack surface.

I disagree on most points...

1. It takes a few hours to set up, sure, but it isn't difficult – setting up a mail server on a *nix box isn't exactly uncharted territory, every possible aspect of it is documented to death on the web. And once it's up and running it takes practically no effort to maintain. Yes, even with spam filtering that keeps my inbox spam free, despite my address having been used in several public newsgroups and forums.

2. It's a matter of preference. I too use Mail.app primarily, but I like being able to SSH to my server and read my mail in Mutt. As nice as Gmail's UI is in many regards, it's useless for threaded discussions on mailing lists.

3. Err, ever heard of procmail or maildrop, among (many) others? This is one of the major advantages of running your own mail server, actually; Gmail gives you very limited filtering options, especially when it comes to mailing lists.

4. Gmail has great spam filtering, but I find my Postgrey + SpamAssassin setup to work just as well.

And in my experience, OP has it right.

1. Setting up the mail server is not the issue. It's the ongoing maintenance to ensure close to 100% uptime so you don't lose any mail. Having worked as a sysadmin maintaining linux boxes for several companies, mail server maintenance accounted for the vast majority of the hours we spent working on servers. This doesn't even account for the hardware upkeep to keep RAID sets in good shape, tape backups, etc.

2. There is simply no open source project that offers a UI anywhere close to as good as gmail. Yes, someone could take on that project, but it will be years before they achieve a similarly polished result.

3. Sure you could roll your own filtering that would be more effective than gmail's, for you. Again, this is further work that you have to do whereas gmail "just works".

4. SpamAssassin is one of the great open source projects in my opinion. Combined with ClamAV you can get a really robust solution that works fairly well. However (afaik) it lacks the learning heuristics that gmail's huge body of data can provide to keep even the newest template spam out of your inbox.

This is not to say that I don't agree with some of your points, but in my opinion the OP has a valid perspective, which I suspect many IT people who value their own time will share.

I do value my own time, and I'll say it again – post setup, I really spend no appreciable time keeping my mail server running. My VPS provider handles all hardware issues (with zero downtime in the last two years), backups consist of a nightly cron job and require no manual intervention, and the last time I touched my Postfix configuration was over a year ago.

"Rolling my own filtering" consists of placing a single mailbox_command line in the Postfix configuration file, it hardly qualifies as a time sink. Writing mail filters for procmail takes no more time than doing so in Gmail, the difference is that the former lets you write much more expressive and fine-grained rules.

And yes, SpamAssassin does have a built-in learning Bayesian classifier. Which I previously used to learn from anything that I manually classified as spam, but these days I don't even get any spam to invoke it on. See sa-learn(1) for details.

You are missing a very critical point: either you need to rent a virtual or real server and spend time configuring and maintaining it - or you could try and dare to run your mail server from at home on some linux box which will open a whole can of worms...

So gmail gives you an easy and free alternative and you can just sign up and access mail via web, pop3 and especially IMAP, which made it superior to pretty much every single other webmail when they launched that feature - and I believe that is still true today, most free webmail providers don't give you free IMAP. You are "paying" by giving away your private information but evidently, people prefer it to setting up their own mail servers.

I'm not sure where you're spending time on maintenance. I set up my current mail server 3.5 years ago (postfix, spamassassin, dovecot, sieve, roundcube) and barely ever touch it -- the only maintenance required is the very rare security update.

I use tarsnap for backups, so no tapes required.

I don't love running my own mail server, but as a hacker, I don't see that any of the alternatives are better.

Regarding point #1, I have maintained Linux mail servers both for large shops as a professional sysadmin as well as for myself and small organizations, and each has been a very different experience.

In the large environment, mail server maintenance does take a disproportionate amount of time. But that's because the environment is so complicated: there are several layers of servers, mail is stored on NFS so there are weird locking issues, LDAP is used for user accounts, etc. There are some very bad failure semantics and if we're not careful, we start bouncing mail.

My small setups use exactly the same software (Debian, postfix, dovecot, spamassassin) but the environments are much simpler: one server, no NFS, no LDAP. I use a VPS so I don't have to worry about hardware. Backups are an off-site rsync in cron. Failures rarely happen but when they do the semantics are much nicer: temporary failures so the mail is resent later. My VPS host failed for the first time in 5 years recently, so I couldn't get mail for a few hours, but I didn't lose any mail and didn't have to spend any time on recovery.

I also run other stuff on my VPS, so the monthly cost and recurring maintenance overhead (security updates/backups) is amortized. Adding mail as another service doesn't change these costs.

I'd love to ssh to my server and use some modern MUA like Sup, but there are lot of places where you cannot ssh through local firewalls.

When you work parttime as a consultant or when you just meet with some potential customers/investors/cofounders/whomever you often need to use THEIR infrastructure. So what then? Do you imagine that you'd be unable to get an email from the people you're talking on the spot just because you need to tunnel somehow to your server? Bit weird..

One option is to always use own mobile connection, but still there are places where there's no carrier (like some server rooms), or you have to use customer's intranet at the same time you're relying on what's in your emails..

A handy trick I have always used is to get a second IP address on my servers and set up ssh on port 443. From memory I have only had one case where a firewall blocked it.

I haven't used Mail.app since an incident about 2 years ago where it deleted a bunch of my messages. I was moving them from one folder to another when the connection to the server dropped.

I don't know why this should happen with IMAP (it is possible to move messages from one folder to another atomically) but it did.

So why is there such a high degree of Gmail usage among those groups?

Because we're lazy.

you give away control over your personal data

Yes, and if my data disappeared tomorrow, I'd be pretty pissed off. But since Gmail has a sort of critical mass, it would be likely that other people would lose data too. Lots of pissed off users would tarnish Google's reputation and it's in their interest to avoid that.

you put your personal data within the U.S. jurisdiction

A lot more than my email is within that jurisdiction and is much more important -- like my money, family, and possessions.

Besides, I'm a hacker. If I want to send something sensitive, I'll be smarter than sending it over SMTP and logging it via Gmail.

you give Google not only the social web information who mails whom, but also the full content of that communication

Yep, they data-mine my email anonymously, but they try to not be evil about it. There are much more nefarious groups tracking my behavior, too. Besides, the group effort cuts down spam.

Yes, and if my data disappeared tomorrow, I'd be pretty pissed off. But since Gmail has a sort of critical mass, it would be likely that other people would lose data too.

You're picturing some massive server failure, but not the more likely case of your Google account being disabled for any random reason, which has happened to more than a few people. As Gmail gains more users, you become that much less important to them.

Something like that happened to me - my account (everything, not only Gmail) was suddenly inaccessible. However, a quick email to support and within 12 hours they sent me a link where I entered three emails I wrote to recently... that's it, password reset, account reactivated... I think they're getting better at handling these situations.

Back up your email! Just because your data is in the cloud doesn't mean you shouldn't make a backup.

I just run getmail which retrieves mail via Gmail's IMAP interface every now and then. If Gmail went away tomorrow, I wouldn't be _that_ sad.

Agreed. Another compelling reason is data deletion -- accidental or malicious. While catastrophic loss due to hardware seems to have declined over the past few years, malicious deletion has gone up markedly.

I've pimped them out before, but this is why I set up backupify (handle more than just gmail, since your Google password is for all your Google accounts) and periodically export my data. I just wish they did more, like backup my Dropbox and Atlassian accounts.

Don't use IMAP for backups, use POP3 instead. If your e-mails are deleted in the cloud, your local IMAP e-mails will disappear as well.

Actually do. Since he's using getmail, its making a copy to his local system. It just happens to be able to do that over IMAP. You are better off doing this than doing the same with POP because Gmail's POP deletes mails as you read them, which is non-standard. You could lose emails if there are network, disk, or other errors.

Gmail POP does not deleted mail as you read it. I have a POP client on my Mac that output pulls Gmail as a backup of my webmail. Nothing gets deleted.

Ok you're right about it not deleting by default. But there is an option to delete mail after its been accessed with POP. And in any case Gmail hides the message after you've read it, but you could still have an error writing the message to disk and thus miss backing that message up.

gmail's POP server doesn't delete mail. What it does do is disappear them from the POP listing (ignoring any client "leave messages on server" config), although even this is resettable through the web interface.

If your e-mails are deleted in the cloud, your local IMAP e-mails will disappear as well

Vice-versa too. I learnt this the hard way :(

IMAP is just a protocol. You're talking about syncing over IMAP. You can just fetch over IMAP too.

Also, POP3 is not folder-aware. You can only access your inbox over POP3. If you want to backup anything (e.g. drafts, sent mail, etc), then you need more than POP3.

If there isn't much that you are worried about losing, you can just save a local copy from your browser. I only do that with a few per month, but it works fine with no extra fiddling.

I still believe the chance of Google losing or preventing access to my data is much lower than the chance of me accidentally screwing up my own server and finding out my backups don't work. It's a huge amount of work to get anywhere close to the reliability of GMail. You have to maintain the server, secure it, test backups, verify backups and deal with every security or maintenance issue that comes up.

It's a trade-off, and one I've come down on the side of GMail for.

As long as it's in Google's interest to provide such an outstanding service to their users, yes, that particular billion-dollar Top Coder champion-filled company will do a better job of managing a mail server than you and I combined.

> You're picturing some massive server failure, but not the more likely case of your Google account being disabled for any random reason, which has happened to more than a few people. As Gmail gains more users, you become that much less important to them.

"more than a few" is probably still in the ten-thousandths of percentage points if that. Some can't, but I can live with those odds.

And as Gmail gets more users, I'm guessing the odds get even better, particularly with so many of these new users being tied to Android. Google has a lot tied up with making sure things don't go south.

And then there's services like backupify.

I wouldn't even call it lazy, I call it efficient. Google makes email management easy so I can focus on something more interesting and hopefully more beneficial to my peers.

Yep, they data-mine my email anonymously, but they try to not be evil about it.

Eventually, as it happens in all public companies, the "do no evil" people are going to be out, and the "maximize the buck" people are going to be in.

You think a "maximize the buck" move by Google would be to start rooting around in your personal drawers?

Well, they already root around... That's why they can serve contextual ads.

Right now it's "harmless"... But people get a lot more personal in email or google searches than they do on, say, facebook.

There's some gold to be mined in there.

All it takes is one idiot in middle management that thinks "who's going to notice if we just ...". Like the recent Mocality case.

Which was rectified within a week, no?

I don't doubt you; this may eventually happen, but for the meantime, I'm trusting them (over my ISP, anyway), with my email and backing up/saving the stuff that warrants it.

> There are much more nefarious groups tracking my behavior, too.

Right, I'd personally be more concerned about random mailing list administrators "looking through their logs" and drawing conclusions about me than by Google scanning my email for the purpose of targeting better ads to me... hmm...

You pretty much said what I was going to say, only nicer.

Thanks :)

1. Rich webmail is an absolute must for me. I use too many different computers from too many different places for anything else to be remotely practical at this point.

2. No open source webmail servers that I've seen come close to Gmail's functionality, and I don't have time to write one that is.

3. Even if there was/I did, I couldn't get the spam filtering to anything like Google's level even in theory since I don't have nearly as much data to work on.

> 2. No open source webmail servers that I've seen come close to Gmail's functionality, and I don't have time to write one that is.

Obligatory comment pointing out an opportunity for disruption, etc., why aren't we all millionaires., etc. etc.

I'm not sure that an open source webmail is an easy way to become a millionaire. =P

Because not even "extremely security and privacy aware" hackers care enough to use it. That said, Zimbra is pretty bloody decent.

The business model isn't there. I have yet to see a large group of consumers willing to pay for secure email and the only other alternative, advertising, leads to the exact same problem that you face with GMail.

I would pay and so would many others.

That seems to imply you don't already. There are lots of paid email services that don't monetize user's emails. And many that are hosted outside the United States.

For the extremely vigilant, there is always CounterMail

https://countermail.com/

The threat of email interruption and snooping can't be completely avoided. Undersea cables get cut, governments change, servers crash, data centers get raided, and companies disappear. At some point the data gets decrypted and everything is retrievable unless you are any extremely hard core PGP user. Even HushMail has to bend to the feds when it's all said and done. Even savvy people realize paranoia only gets you so far with email.

+1 for willing to pay for secure email.

Ok that's 2 customers who are also HNers. I think you'd need to do better than that. We are hardly a representative of the general consumer.

There is another problem, US govt' need to get access all these messages. For example http://en.wikipedia.org/wiki/Hushmail, people seemed to have signed up, but Hushmail was forced to provide plaintext messages to US govt upon request. So they sort of compromised their main selling point.

That is probably the largest problem, you'd be stuck between a rock and a hard place. You either please your security conscious customers or please Uncle Sam. You can't please both.

"There is another problem, US govt' need to get access all these messages."

You're assuming the service has to be located within the US. Why?

It doesn't have to, but unless it is located in Iran, NK or other US-unfriendly place, US govt can always pressure the local govt to pressure the local business to turn things over. I wouldn't, for example, count on countries like Switzerland, US is already getting to its banks to turn over US accounts, and is supposed to be one of the most independent and un-influenced countries.

With the current legislation trend, eventually un-cooperative or "terrorist friendly" sites would just be filtered out and blocked, so you might have a hard time accessing your email. Some messages might never make it to you.

  > US govt can always pressure the local govt

That depends. In some places, the privacy laws are better than in the US. You would just have to choose wisely.

> In some places, the privacy laws are better than in the US.

That's nice, you still believe that the laws on file actually govern how the country is run.

Bug if you follow e.g. how the spanish SOPA-like was passed, you realize it's all a facade.

I was actually talking about writing a front-end that you could host on your own server, not a gmail clone. (I wish I could update my original comment.)

So... why don't you buy something like Rackspace email, which costs $2/mo?

Because the minimum cost is $10/mo for Rackspace email.

I personally have switched between payed google apps for domains and fastmail (those are both $50/yr and under).

Re: rackspace, you can always sign up for a trial account for something else, and once you're in, get the single email subscription for $2.

This is why I would never start a business in this space. You're saying $10/month is too much to protect the privacy of your email.

I think for most people, $10 IS too much. Although we all talk tough about privacy, most people (even here) don't have much to hide from anyone, and when the difference between definitely secure, and probably secure (in terms of privacy and ownership of data) is $10, it suddenly feels a lot more expensive.

Which takes me back to my original point. :-)

I am just about to migrate from Gmail to NeoMailbox. I'm choosing the Swiss hosting option. I'd prefer something in Iceland but the closest service I could find was OrangeWebsite and for them mail was pretty much an afterthought.

https://neomailbox.com/services/secure-email http://www.orangewebsite.com/web_hosting.php

lavabit.com

Hushmail would disagree - https://www.hushmail.com/

Nice to know, thanks for the tip

Did you try it and can compare it to gmail?

Especially on:

- linking other accounts (POP and IMAPS)

- spam detection

I actually host my own email. I developed an open source application for automatically encrypting all email with my public key as soon as it arrives, and I can't do this if somebody else hosts my email. You can read why and how here: https://grepular.com/Automatically_Encrypting_all_Incoming_E...

Brilliant! Thank you for sharing the post and code. I'd be interested in hearing thoughts on how to make it searchable. If you don't index, searching will take quite some time. I guess the best thing to do would be to index each e-mail (keywords?) before it is encrypted and then encrypt the index itself as well. There is a problem with the keyword approach though - if the index encrypts the words but the "link" between message id and encrypted keywords is not encrypted, then an attacker who is in posession of one or several other message bodies in plain text can see correlations between the content of known and unknown message bodies.

  >  advertising, leads to the exact same problem
  > that you face with GMail.

Not necessarily. For some people that largest factor is that Gmail brings all of your email under US jurisdiction. It may be enough to just do advertising-funded Gmail clone that is solely based out of another jurisdiction.

You may even be able to convince non-US businesses that this is a better alternative.

I would gladly switch to a different webmail provider (and pay a decent amount for this service) if they could replicate all of the benefits of gmail.

As a curiosity, what is "a decent amount"? $5/mo? $10/mo? More?

I would probably pay $200-$300/year, so maybe $15-25/month if this hypothetical service could completely match gmail in features.

I'd be really curious to see what Google makes off the advertisements served up in Gmail, compared to this price.

By the way, I meant someone creating an application you could install locally, instead of just being a gmail clone hosting everyone's e-mail at once.

There is Zimbra. http://www.zimbra.com/products/zimbra-open-source.html

Zimbra has a very heavy feel (it is more Outlook than usable client) and the mobile support is not a part of the open source edition.

We use Zimbra at work. I'd have a hard time recommending it unless you really, really want 80% of Exchange and 40% of the usability.

(Outlook Web Access is still the only webmail client I've found that is in GMail's ballpark. Since running an Exchange server is beyond my skillset, my interest, and my budget, and trusting a managed Exchange host feels odd--at that point I might as well just stick with GMail. So I do.

Google isn't going to find anything particularly interesting about me by my mail, anyway.)

Because if you got nothing to hide, you got nothing to fear! Yay!

A mischaracterization (and, given your tone, probably intentionally so).

A useful mail system has value to me. It outweighs the (in my mind vastly overhyped) dangers of Google doing something nefarious with my data. If you disagree, you're welcome to run your own mail server. Nobody's stopping you.

"[Gmail/Facebook/USGov/etc] isn't going to find anything particularly interesting about me by my mail, anyway."

There are two problems here:

1. We look at our data from our point of view. Other people and orgs have their own points of view, and it's their actions driven by their points of view that we need to worry and think about. For example, you think your mail is uninteresting, yet Google enhances the value of their ads by correlating your interests with the interests in your communication network.

Government surveillance can also learn a lot just by recording and analyzing who you talk to. When a law enforcement agency cannot get a full wiretap warrant, they will sometimes settle for a pen register, which just records the telephone numbers of your calls. This allows them to find out who you know, and who you talk to the most. This lets them form more detailed opinions about you and your known associates, which may or may not be accurate. You know, like Google does for ads.

https://en.wikipedia.org/wiki/Pen_register

Oh, you're just not that interesting to the government?

2. Innocent behavior being observed by the government is enough to disrupt your life in minor and major ways. Consider the British couple that was recently denied entry to the United States, merely because their tweets contained boisterous comments about destroying the US (i.e. partying really hard), and digging up Marilyn Monroe's grave. Imagine if they had used the British slang term "crack."

https://en.wikipedia.org/wiki/Champers#C

It takes nothing more than mere notice to disrupt your life. If you innocently have communication with someone who in turn has communication with a "person of interest," you could find your car carrying a GPS. Just from noting communication networks, never mind the communication content.

I'm not trying to go all tinfoil hat, I'm just suggesting that your view of your data is completely different from corporate and government's view of your data. You could be served an ad (horror!), or you could be served a warrant (horror), based on accurate or mistaken interpretation of any facet of your communication.

Yeah, but it's also email. You could delete every email you've ever created and everyone who's ever saved their copy still has their copy.

Even if you were trying to be all sneaky about your email by having your own service set up somewhere, you're still making copies of everything you send by virtue of sending it. It's more vectors for the government/Evil Google/whoever to need to scan to get a picture of the data you're creating, but it's not impossible.

I know you're just making a point, but to the people who actually are all tinfoil hat, this whole conversation to shift to whether or not you should even use email as a method of communicating.

> 1. Rich webmail is an absolute must for me. I use too many different computers from too many different places for anything else to be remotely practical at this point.

That's what IMAP is for. If these are computers you don't directly control, then I'd say any downsides of using gmail pale in comparison to the security implications inherent in your usage model.

[edit] Can't say I understand the negative downvotes. Maybe I inadvertently denigrated people's preference for webmail over mutt/mail.app/etc?

Regardless, the fact is that if you're utilizing webmail for remote e-mail access on shared computers, then there are much more significant risk scenarios at play than those of gmail.

Secure webmail access over HTTPS is in theory just as strong as IMAP access can be made to be.

Of course, by using untrusted hardware you are exposing yourself to various circumventions of the security protocols, but that's just as true of IMAP as it is of HTTPS.

In fact, IMAP is probably slightly worse since it does actually download messages to the local machine by default, where they must be explicitly cleaned up.

I took "the pragmatic UNIX way" about 15 years ago with email. It works. I've not had to change it and don't feel compelled to bother changing it.

I host my own mail server. It's not hard regardless of the platform you use. I use debian+postfix+mutt as it pretty much works out of the box. I've changed perhaps two lines of postfix configuration (to set up maildir) and added a couple of lines to my muttrc to pick up maildir and view html mail using links.

I don't get SPAM at all. I don't stick my email address anywhere on the Internet where it will get snagged into a spam database. I never have. I've not had a single SPAM message in 15 years with the same email address. I do not use any spam filtering software.

I use aliases for mailing lists which are created and destroyed on demand using a couple of 2 line scripts ("append, newalises" -and- "sed, newalises").

I can get into it quite happily from anywhere using SSH on my mobile device using MIDPSSH or another machine with PuTTY, iSSH, or good old terminal SSH.

I don't keep emails ever. I action them, then throw them away. Those who keep everything are like the crazy old people who live in rooms stacked to the ceiling with newspapers. I have nothing to backup or care for in that department. If I lose my mailbox, I have lost nothing.

I do not manage my tasks with email. I use a text file in my home directory called notes.txt.

My contacts list is a text file called contacts.txt. Works on anything. Can be grepped.

My calendar is a text file called cal.txt. Works on anything. Can be grepped.

I probably spent about an hour in the last 10 years on email server configuration. That's considerably less time than some of my peers spend dredging through their 5 years of gmail junk.

Self respecting hackers don't use Gmail and Co.

I unlike you, took the pragmatic way (without quotes or Unix). Email is part of my life and I don't want it going down.

I don't spend time configuring servers, updating packages, guaranteeing uptime. My email is federico@mheroin.com, and I write it everywhere. I still don't get any spam. If I need an alias I just append characters to the email address and I don't have to fiddle with anything else.

I can get it quite happily through IMAP using Emacs or the client on my phone. I can search for past emails in seconds without worrying about the space they consume. I can share my contacts and calendar appointments with one M-x command.

I'm curious about your setup since Debian/Postfix/Mutt are not enough for POP3 or IMAP. How do you access your email from other devices (phone, table, etc)?

I hate Gmail, with passion, but Google Apps is the easiest way to keep all my email working without thinking about reliability/installation/etc.

Some more info...

Email augments my life but does not control it. That's where the distinction is. Much as writing letters is a tool, so is email. If it went away, the world would not end for me and I intend to keep it that way.

Availability? If mail doesn't get delivered immediately, thanks to the joys of SMTP, it will come later (when my ADSL line is back up). If it's urgent, someone can put mouth to phone rather than finger to keyboard.

Updating packages? Cron does this for me once a week. Maintenance? There is none. Everything is done properly once and automated.

I access everything via SSH if I have to which is rarely. I avoid email on the move where possible as there are more important things to do like looking in front of you and not worrying about stuff.

I make sure I don't need past emails. Not everything everyone has written is golden. In fact most of it is useless and just noise.

For reference's sake, I trialled Google Apps on a domain for 2 months as a possible replacement for my setup. It was a pain in the butt. There is actually no support worth anything if it does go wrong which it did terribly (gmail stopped working entirely with a server error). The entire domain approval process is problematic and time consuming and their contracts are disagreeable. Plus it didn't deliver any additional value.

Ultimately, if you look at it, there is actually more work to setting up a system to backup contacts and calendars and watching that than there is to operate my setup entirely.

I assume you are backing up your google apps accounts?

"Email augments my life but does not control it." I can't really say that the way self respecting hackers are, but self respecting people are.

I'm not a hacker, i don't plan to be one. I run my own server with a similar config as you do [i do run spamassasin]. I don't let telephone, mail, im, or letters run my life; as i don't let google, yahoo, facebook or whatever do the same. I'm not a hacker, but i share and respect the values Harald Welte wrote about, as a person.

Can you _send_ emails from your ADSL line? Mine are rejected since I have a semi-dynamic IP address.

Most ISPs will let you smarthost out via their upstream mail server.

Oh, right, I tried that too, they said they've discontinued that service years ago.

If you ISP provides SMTP for their e-mail (almost all do), then barring outbound From address filters, you can set up your SMTP server to treat the customer facing mail server as a smarthost.

The ISP doesn't need to do anything specific to support it.

I was absolutely sure they did filter, but I've just tested and apparently they don't. Well, that's unexpected. I might switch to self-hosting again then.

This is what I do. My ISP has a relay which I send all non local email from postfix to. I have a static IP address.

Unfortunately, your lifestyle can't work for all of us.

I currently freelancing my way through school. Not as a developer, but as a stagehand. The way I get work is through email. At any given moment, one of any number of production managers, lighting supervisors, technical directors, or production supervisors might be sending me an email. This email will say when and what types of laborers they are looking for. They may need labor tomorrow or 4 months from now. Usually, these positions are booked on a first come first serve basis.

I need my email. If my email went away or didn't go to my phone reliably and on time for an hour, I could miss an opportunity for hundreds of dollars. If it didn't go to my phone for a week, I wouldn't make rent that month.

Yea, it sucks that my life is driven by email, but this is so much better than the way things used to be for stagehands.

If you email anyone ever, or use any ecommerce site, your email hits the spam databases when they are compromised.

That's never happened to me. Perhaps I use more reputable commerce sites (so far anyway).

"My contacts list is a text file called contacts.txt. Works on anything. Can be grepped. My calendar is a text file called cal.txt. Works on anything. Can be grepped."

I really like the sound of that. Can you elaborate a bit on the format, or provide a couple of sample lines from each?

Yes - it's very simple:

contacts.txt - in order of surname, firstname

Surname, FirstName, Primary telephone number with code, Email address so:

   Joe, Bloggs, 02033345545, joe@bloggs.com

I only keep a single number for people, but you could extend it to two numbers if you want.

cal.txt - latest dates at top of file:

   2012/02/02 16:00+ What to do         # time specific event
   2012/03/03 --:--  Someone's birthday # all day event

When something is done, the line is deleted. If something is not done on the date, it is just left as "overdue" at the top.

So I guess you don't have Gmail's recipient autocompletion, or you do? This is the killer feature of Gmail for me, with its keyboard shorcuts and search.

I don't need to search. My inbox (my only folder) is 4 items at the moment. I rarely initiate an email conversation but whe I do it's simply select the addres from the text file and paste with the right mouse button.

Mutt has an autocomplete function but TBH I can't be bothered to set it up.

I do something very similar for my calendar, but it's kept in markdown and instead of deleting the done items I move them under a heading for the date completed(along with any notes about the item). While I'm not required to submit a timesheet for my employment, I find it nice to have a log of my actions.

The contacts file sounds interesting, I'll give that a try.

Thanks for the information. Do you enter birthdays every year? Or just use sed or something to update the year? If you delete everything after the date, I guess you never need to refer to old appointments?

If it's a birthday, I actually manually add the birthday's next occurence to the appropriate point in the list. It helps keep the dates in mind as well!

"Self respecting hackers don't use Gmail and Co."

Are you not self-respecting? Just a few days ago, you said:

"I've got offline gmail if i want."

http://news.ycombinator.com/item?id=3533625

If I want i.e. I can download offline gmail if I want to. If you see my follow up reply, I did trial Google Apps but it didn't cut it. Part of that trial was to use the MIDP version of Google Mail. I don't use it now but it's there if I wanted to.

It would be unfair of me to blindly criticise Google Mail without trialling it, which I did.

Hope this clears things up.

1. Because Google solved the spam problem better than anyone else has

2. Because maintaining your own email server (which many on HN are perfectly capable of) is a giant pain, especially when you have to deal with SpamAssassin, DKIM, and all of the other things that you need to do correctly to have your email work

3. Because I still have my data even if Google loses it — I have a backup from downloaded email through Sparrow as well as an entire backup directly from my Gmail through Backupify (https://www.backupify.com/)

4. Because I like to outsource the tools and services that I need to people that are experts at it. It's the same reason that I use Beanstalk for Subversion instead of hosting my own server, use FreshBooks for invoicing clients instead of doing it myself, and send transactional email through Postmark.

It's easier. Much easier. And they are all very good at what they do.

Bottom line: So that I don't have to worry about email. It just works, which allows me to do exactly the same thing.

1) Black listing - if someone in your IP neighborhood sends spam you can be silently blacklisted by Comcast, AOL, some random crazy zealots running a blacklist that other random people choose to use… et.al. and people stop getting your mail. Good luck getting unlisted. Wasted hours. Sometimes it's futile.

2) Spam filtering - If you are the sort of hacker that publishes their email address in order to interact with the community, and you become popular with spammers, then it is difficult to beat Google's spam filtering on the hundreds of spam you will receive each day.

Until a few years ago I used to run my own servers and corporate servers. Nicely trained bogospam filters (this is work and involves the brain killing activity of reading borderline emails to categorize) got most of it with little risk of resource consumption failures. SpamAssassin got some of the remainder, albeit with some risk of exploding, but spam still got through.

Those same addresses now let about 4 spam per year through the Google filters. You can't beat them. Your sample size is too small.

I hosted my own e-mail on a VPS for a few months before going back to Google Hosted e-mail, mainly for the two reasons you list here. I do want to go back to hosting it myself though.

Because GMail users have different values than the OP does, and it's morally presumptuous to assume everyone makes the same tradeoffs the OP does.

Why their choices don't make sense to him is because he hasn't taken the time to understand the value systems of GMail users.

This point can't be made often enough. I can't say that I think there are "too many" (or even "an increasing number") of posts on HN that seem to assume that everyone here has precisely the same set of (often extreme) values, but these kinds of presumptuous, somewhat self-righteous postings do pop up from time to time, and this is precisely the reaction I always have. The definition of "self-respecting hacker" does, after all, probably very between self-respecting hackers.

I agree. His post smacks of the same attitude that many people (negatively) attribute to engineers in general. The, "this is how I do it, so this is how everyone should be doing it," attitude.

This article is link-bait, but I'll bite.

Simply put, Gmail provides a compelling user experience:

* fast and accurate search. This is key, and I don't know of a single desktop email client that even comes close to providing a decent search feature. Decent here means "answer full-text queries accurately within a second or so". I didn't realize how useful this was until switching to Gmail; now I have it, I can't switch back. I don't even bother organizing my email more than a minimal amount any more, I just search for things when I need them.

* remote access is also key. I regularly access the internet from at least 4 different devices, on at least 3 different operating systems. Sometimes I do so while traveling. Gmail makes this convenient --- all I need is a web browser. IMAP need not apply --- I found IMAP appallingly slow last time I tried it.

I use Rackspace Email, over IMAP on the desktop, IMAP on my phone, and webmail when on someone else's computer. It's never slow in any way. When I sign up for things, I hear the ding of the confirmation e-mail within 2-3 seconds of submitting the form -- fast as push mail.

It's using IMAP IDLE (https://en.wikipedia.org/wiki/IMAP_IDLE), which is push email.

> This is key, and I don't know of a single desktop email client that even comes close to providing a decent search feature.

I used to use Opera M2 built-in client and think it's even faster than Gmail web UI and very accurate.

> This article is link-bait

You should probably check out who Harald Welte is and what he does before making such accusations. I don't agree with him here, but "link bait" this is not.

My experience is the polar opposite. I have to rely on a native client to search because GMail search is so bad. It hasn't been that fast for a while -- waiting 30+s for a search is pretty routine nowadays. But worse, it's never really handled stemming well -- it has to be an exact word match, even for singular vs plural. And the number of false positives I get usually doesn't even make it worthwhile. I had resorted to deleting mail rather than archiving it, but that only helped marginally.

Having said that, I rely on too many marketplace apps now to make switching a reasonable alternative. So, I make do with native clients.

30s searches....I've gotta ask, how many gigabytes of email do you have? I've got about 3 and mine come back in less than a second.

One account has 2.5 GB, another has 700 MB. I'm on a lot of mailing lists and get a lot of transactional email. I just timed switching a label and that took 4s on the smaller account. This is in Chrome with SPDY enabled, too, so I doubt it's client transport.

I should probably note that it tends to be variable with the time of day. That's not unexpected, but still kinda sucks.

I have almost 2.5 GB's and also have zero problems with GMail's default search. Comes back in less than a second.

Maybe it's number of messages vs size of store. Or perhaps because a large percentage are similar in structure (google groups, monit alerts, etc.), they cause indexing to perform poorly. Dunno. But I wish I had < 1s searches.

I get some of the article's points, but I don't buy his solutions.

Quite frankly, I trust Gmail to keep my mail safe and secure more than I trust either myself, my friends or some NGO or other non-profit. I'd be lying to myself to think that I could do a better job of keeping a mail server secure, and I trust other small organizations even less. While I understand why the privacy issues make people uncomfortable, from a reliability perspective, Gmail is far and away the best option. So I can easily see why people might be willing to pay the privacy price in order to get better service.

When people have to choose between usability, accessibility, and reliability VS security and privacy they almost always choose the former. My response to this would be "Why do self-respecting open source and security advocates refuse to do the hard work of making security and privacy easily available to everyone?"

<IMHO>Laziness, disinterest, and fear of killing off lucrative security consulting.</IMHO>

There's no privacy in either case. Every single email passes in clear text through some ISP who may very well be storing copies of them. If you want privacy, you use PGP.

That's what I'm getting at. Have you tried using GPG? Most "security" products (or projects) have terrible usability.

Disclaimer: I work on a "security" product.

Usability isn't a great concern to me ;) I used GPG with Alpine. I stopped when my key expired without every having sent a single encrypted email.

"Usability" is ill-defined. GPG seems to have pretty good "usability" from mutt (which auto signs outgoing email and optionally encrypts with a keypress). Maybe your MUA is broken or you forgot to RTFM.

Usability (as I've employed it) is when someone non-technical can derive benefit from something beyond their understanding. Rest assured I've read every line of the GPG man page and then some.

Because it's the best email system available for most people. If you value productivity and usability the most, Gmail is the way to go. It's that simple.

Gmail is the only email system that I can tolerate. It actually makes using email borderline fun, whereas even something like Exchange is a nightmare. It works great with various applications, smartphones and has by far the best Web interface around. It handles spam well and allows me to easy filter messages (and have those filters work across devices).

For most of us, email is a tool that helps us get work done and communicate with family and friends. Rolling our own solutions is not worth the extra time, headaches and lost usability.

Google doesn't care about our data individually. They make money in anonymized, aggregate data. That's why I don't care that they are making money off of my data, because it's not my data that they care about. It's our data that they really care about.

And while I'd prefer if my email didn't fall into the hands of the US government, I don't actually have anything that I care that they see in it. It's more principle than anything else. And, as a US citizen, if I rolled my own solution, I don't think it would be any safer in my hands than Google's when it comes to warrants.

I do have real fears. My real fears with email are in using a system that isn't usable, isn't reliable and has data integrity issues. At the end of the day, Google's servers and technical know-how surpasses mine, and I feel that my email is safer and less likely to be lost due to hardware failure in their hands than in mine.

It really depends on what you value. If my email information was really sensitive, I would probably care more. If I were a company that valued sensitivity a lot, I might not use Gmail. Certainly if the work you do or the industry you are in needs the utmost privacy, you should look into the most secure option as possible.

But as an individual, Gmail is as good as it gets for me.

One point about privacy: E-mail is unencrypted and is delivered over public networks.

Assume that any government, be it your own, US or other will read your email if they so please, and encrypt anything you don't want them to read.

This trumps all the other privacy arguments. If the government wants to read your email, it need not access it at the endpoints -- it already has access to it in transmission over the compromised backbone. You would need to encrypt your emails to avoid this.

Someone I know once told me crypto is funny because all you have to do is compromise the OS's socket implementation.

How so? The application does the encryption, not the OS.

Is that before or after you write a gui interface using visual basic to track an IP address? :/

To be fair, this is a guy who does hardened, embedded RTOSes. In the normal world implementation, of course crypto matters.

Sometimes I need to take off my "all software sucks and I hate everything" hat before posting :-/

Which is why it baffles me that so few people use S/MIME--other than the trending preference for webmail which isn't well suited for encryption. S/MIME is simple to setup on most email clients, and offers encryption of the body of the mail if the recipient is uses S/MIME as well. There are several of us at work that send encrypted emails all the time.

Until recently the GMail web interface was simply unmatched. When GMail first shipped, their conversation view was so far ahead of what anyone was doing on the web or in a fat client. I don't try out as many email platforms as I used to, but from what I have seen, it's still the best implementation of conversation view available.

Additionally, you hinted at the other main reason I use GMail in your first bullet point: "Control over your own data means you own it, you have it on your hard disk, it is not on somebody else's storage medium."

Sure, this means Google has access. But it also means I don't have to find a way to make that data accessible to me everywhere I want it to be. I don't have to pay for the storage. It's a solved problem... and available at a great price point ($FREE).

I trust google slightly further than I can throw them, so for now this is an okay deal.

You imply that Gmail's web interface has been matched... I have not yet seen that to be the case, any suggestions?

Because we care about efficiency, not least with regards to our time, and running one's own mail server is not an efficient use of such a scarce resource.

It is if you don't want a potential competitor, potential acquirer, or just a large amoral corporation, to have an informational advantage over you.

I would LOVE for someone at Google to violate their TOS and read my mail. That would be far more profitable than any business idea I will execute this year.

> That would be far more profitable than any business idea I will execute this year.

Why do you assume that you would ever learn about it?

A simple answer: because running one's own mail server has little/nothing to do with being a "self-respecting hacker".

For myself personally, it's because I value my time being spent elsewhere on things I consider more important.

I simply don't care if that approach earns me a label of "non-self-respecting hacker" (as the original author implies).

Because email sucks. Hosting email sucks. Email clients suck. (Bonus: email spam sucks). On Gmail it sucks a bit less.

I'm sorry for myself in the first place, but until I can code my dream of an email killer (or email client fwiw) that does not suck, avoid making my eyes bleed (and make it a success...) I can't avoid gmail.

Funny. I just asked for alternatives to Gmail and other hosted software on the Richard Stallman post http://news.ycombinator.com/item?id=3551345

Harald's suggestions boil down to

  - share the administrative and financial effort with friends
  - use hosting form NGOs or non-profits
  - use small companies and ISPs

The first suggestions might work for me, but what does everyone else use? If you host your own what software?

Lastly, what do you think of a home appliance (basically a server the size of a router with a web interface) that people could install in their home to host some of their important data. Obviously it wouldn't be reliable enough for Email, but might be good enough for docs, password hosting, bookmarks, contact list, etc...

A device that sits in your home and hosts your docs, passwords, bookmarks, contact list, and so on? It sounds like you've reinvented the PC.

Well I should have clarified. I want an always on low power appliance with an easy web interface to host important data for people that are not capable or too lazy to set up a server.

I purchased a SheevaPlug[1] a couple years ago. It's a plug sized ARM server consuming 5W max. It's been great for a low-power, always-on debian server. For the lazy or not capable user, there is the Pogo Plug[2], which was prototyped using the Sheeva. You plug in ethernet and a usb hard drive, and it gives you access from the PogoPlug site.

[1]: http://www.globalscaletechnologies.com/p-22-sheevaplug-dev-k... [2]: https://pogoplug.com/devices

Plug servers. Google it. Marvell ARM chips mostly. I have been thinking about trying to build a little turnkey linux OS to run on those things that can provide similar functionality to what you are talking about.

Alas the technology does not seem to be quite there yet and the work required to get basically a full personally hosted webapp suite is not trivial. That said, I think in the future we will see a lot more 'appliances' that run as VMs or on low cost low power all ways on hardware. Backed by a business model something like wordpress. Meaning that there is a dot com where you can get it remotely hosted for you, and there is a dot org where you can download and host the app yourself.

Plus stuff like the personal router project http://pr.lcs.mit.edu/ would make a pretty interesting paradigm change.

> If you host your own what software?

Dovecot and Postfix on a small FreeBSD VPS. Easy to set up, and it takes almost no effort to maintain once you've got it running. (The last time I modified my Postfix configuration was over a year ago, to relax my attachment size limit.) Between the FreeBSD handbook and the official Postfix documentation, all the info one could possibly need is provided.

A combination of Postgrey and SpamAssassin keeps my inbox spam free. You can also use mutt rather than Dovecot IMAP if you prefer to read your mail on the command line. Likewise, Debian will work just as well as FreeBSD in this role, if you're more comfortable on Linux. (Debconf even gives you a menu-driven Postfix configuration builder, it doesn't get any easier.)

Backups are handled by nightly rsync cron job on a local machine. I don't really have to think about them, aside from checking once in a while to make sure they're still running.

I have to laugh at all the self-proclaimed hackers in this thread claiming that setting up a personal email server is too difficult, takes too much time – or that they have "better things to do". No, I'm not one of those who would argue that a "real hacker" always has to do things the hardest way possible, quite the opposite. But at some point you have to ask yourself: if setting up a small mail server on a *nix system – a task extremely well documented and understood, a task that yields real technical and privacy benefits, a task that the operating system itself will hold your hand through if you're using Debian or Ubuntu – is too much of a challenge for you, then in what sense can you possibly call yourself a hacker?

Getting the email server set up is easy, almost trivially so.

It's dealing with all the other issues that's an immense pain. SpamAssassin is not always a magic bullet, deliverability to third-party mail servers can be a major problem even if you follow all the rules, and Gmail's UI has a number of advantages that many mail programs can't compete with.

If you compare the hours of time a month that takes with the up-front elimination of hassle that Google Apps provides, it's not hard to see why a hacker might prefer to just outsource it and focus on tasks more pleasing to them.

Exactly, setting up a mail server is not hard. What's hard is that you have to keep your server secure; have to make backups, and make sure that backups work; have to troubleshoot problems when mails are not delivered, god knows how much time does it take; have to train your spam filter to get to the level as half efficient as Gmail is.

Just to give you an idea of how popular Gmail is, about 67% of the 7000+ subscribers for Hacker Newsletter (http://www.hackernewsletter.com) use gmail and I'm sure a good bit more are Google app accounts.

To add to this, yahoo.com is second with 3%.

I use Gmail because I don't care about my emails. Google can know that I'm subscribed to the Haskell, FSF and school mailing lists and that I sometimes talk about school projects or work, all without affecting me much. And that is all I use email for.

Now, I could run an email server myself, but I do not have the time, experience or inclination to do this. I could also use a provider that does not use proprietary software, but that would be pointless: since they're running the server, I would not have significantly more freedom than I do with Gmail.

So really, the reason is simple: it doesn't matter much from a practical or ideological standpoint, and I'm incredibly lazy.

I used to always set up and run my own mail servers. I no longer do so, not because I'm lazy, but because I have better things to do with my time than be a sys-admin, and worry about downtime, logs, security, and backups.

Sure, Google has your mail. So use multiple accounts and keep your really private communications somewhere else, or use S/MIME or PGP Mail.

But who the hell cares about them having data for an account that is mostly subscribed to public mailing lists?

well, i don't. spamassassin works fine. mairix on ssd searches as fast as google. mutt is an excellent reader; it can be used remotely via ssh.

none of this is hard. it doesn't require huge amounts of maintenance. and i own my own data.

before this, when leaving gmail, i looked around for an alternative webmail provider and almost went with runbox http://www.runbox.com/ - they seemed to be the best of the bunch, avoiding many of the issues with gmail (being based in norway). but in the end it seemed easier to just do it myself.

The article presents a fair argument, but the title is obviously flame bait.

Granted, I myself thought of setting up my own mail server, but the disadvantages quickly outweighed the benefits. If your argument is sheer privacy, unless you are hosting your mail on a physical machine at your house connected to your home ISP (which is pretty dumb if you want ~100% uptime), you will most likely either use a VPS or managed system under the control of yet another company, which privacy-wise is the equivalent of using Gmail anyway.

I'm sure setting up your own hobby mail server would be pretty fun and provide great deals of experience, but if you are doing so simply for data privacy or better up time, your point is moot, and you will probably be better off just using a provider like Google.

None of his alternatives — team up with family and friends to maintain a mail server, join a nonprofit to use their email (WTF?), use email from a small ISP — seem remotely practical for most people, so no wonder they use Gmail. Even worse, aside from the first alternative, all of his objections seem to apply to the alternatives as well (only with the name of the new email host in place of "Google").

I have messed with running my own email server from time to time, but it generally takes about a day before I get an email bounced back thanks to the overzealous IP blacklists--for instance, as I recall, all Comcast dynamic IPs are blocked automatically by Spamhaus. Great, so I either need to pay extra for a static IP which might be non-blacklisted, or I can pay for hosting and just hope that's not blacklisted. Gmail is big enough and rich enough to stay off the blacklists, I'm not. So I use Google Apps hosting, even though I'd rather not.

Unfortunately I cannot live without the Gmail web client. There is no comparison (even with the, worse for me, new layout). It's fast everywhere, even via my bad phone internet connection, it indicates properly when it can and cannot send stuff etc. It's just better than anything I know and have.

I have tried to use other things, but for my, quite insanely big mailbox, nothing else works. Not only did I import into Gmail all my mail I received since the late 80s, I also get immense amounts of email through the system as I use mail for creating alerts and emergency lists on servers I monitor and such. Wether or not this is proper use of mail (I think it is; I am very productive with ONLY the mail client open; everything I need comes to me in there, neatly filtered, including news, server health, error logs, bug tracking etc; everything has it's own email address I can mail and receive mail from), it works very well, but it's not supported by normal clients; Mail.app just stands there getting mail with high CPU load and Thunderbird is unusable as in stuck all the time (I haven't tried that for a while, but it used to be anyway). Gmail webclient is just nice and fast since the last infrastructure update Google did (the engineers mailed me nicely that I wouldn't be able to use my mail for about 1-2 days while migrating and this proved to be true).

I know what I need instead of mail (but including mail functionality); I just don't think it has been built yet :) Or has it?

  You team up with some friends, people you know and trust, and you share the administrative and financial effort

I have been burned by people I trusted much more often than by google.

  You use a local, small Internet service company rather than one of the big entities.

How is that better? You're still bequeathing your data to a commercial third party.

I get the OP's point, and hope I never have to eat my own words, but the reasons to go through all the hassle of moving out of google et al aren't all that compelling imo.

Because a hackers time is valuable, and google has solved my email requirements. I dont want to set up and maintain a email server. It can be annoying as hell I've done it before. It's 'fiddly'. So many config files(If you set it up properly so it uses encryption) and a lot of email services will block you if your not on a whitelist or sometimes blacklist you for silly reasons. I used to run my businesses email on ec2, you will be going through blacklists and whitelists for a long long time.

1. Setting up a mail server and keeping it running isn't easy 2. Spam filtering 3. (already mentioned) uptime - you don't want to lose email 4. The security of your email doesn't depend only on you - it also depends on where people you are communicating with are hosting their email.

It's much easier to get search privacy since the only other entity involved is the search engine. Which is why I use https://duckduckgo.com/ for search.

Lot's of space Great spam filter that catches 100,04% of spam ( 0.04% non-spam) Easy and fast search options Integration with calander, docs View and save PDF, DOC, XLS, from within browser Great interface Good apps for Iphone, Android, Ipad Some nice integration options with Chrome browser Sign in at lost of places with your gmail account (stack overflow much?) I don't have to spell out the provider g.m.a.i.l. when I say my address to somebody.

Note that he's basing all this on looking at mailing lists. There are surely many people who, like me, who often use gmail for mailing lists, but have our own mail servers for our "normal" mail.

Because being a hacker is about being curious and building stuff.

It is not about keeping some strange 60ies style counter culture/down with the man going.

Two-factor authentication is a big feature for me. I don't have to worry about anyone logging into my account without my cell phone.

I'm sure there are many other reasons, but also, at the time it rolled out (and/or when they integrated Postini), Gmail was sort of a miracle cure for spam. Just sign up, and spam-be-gone. A lot of people wanted such a "fire and forget" solution.

Because gmail is a good product.

Please don't tell us what not to use. Point us to a better product. (spam, uptime, ease of use, free, fast, etc) Got one?

Yeah, I figured.

Honestly at the end of the day it comes down to spam control for me. I'm done trying to configure adaptive spam filters or finding the latest spam list to scan against. Its not worth my time to deal with that anymore.

I self hosted mail til 2008.. Sometime in 2008 we were travelling more than at home so I moved to a virtual private server. During those travels, the VPS had catastrophic failures so I bit the bullet and moved the domain to google apps and it worked quite well. The intention was to move back to my own hosting when settled again, the gmail keyboard shortcuts are so good (not quite mutt) which has kept me there.

I use offlineimap to keep a local backup of my email, and like anyone should, I think of emails as postcards. If its only to be read by the recipient use encryption!

Because it has an ease of access and the plethora of related network effects. Plus, I know more than a few ways to send any information that I would be that concerned about other than through gmail.

I don't quite understand the constant articles about someone who is thumbing their nose at the "other idiots" who don't care about privacy. Is it not common knowledge that even if you never set foot online your private information has long been dispersed and used for much more than it ever should? We all should be more concerned with the places that we enter information that does not disclose how it is used in a friendly online location (banks, products, doctors office, grocery store etc...).

I host my own mail server on a VPS. I made a clear cut from the big G about a year ago. I switched to duckduckgo.com as my main search engine, installed noscript, blocking all analytics requests, and set up my own mail server on Arch Linux using dovecot, postfix, and spam assassin. This setup has been wonderful and I feel proud to use it since I was once afraid of various issues which, in reality, have been minimal! I can access my mail from any computer or mobile device that allows SSH or has an IMAP mail client.

Once after a system upgrade mail was no longer being delivered to my inbox. I noticed after a day and thought it was strange. I forgot to reboot the mail server software after upgrading. Soon after my mail server rebooted I had many emails delivered from when my mail server was down. This excited and surprised me. I thought I would never get those emails but I assume the mail clients just held on to them and tried sending again intermittently.

Any self-respecting hacker should at least try to setup a mail server on a virtual machine. The hacker will find it interesting and useful. It really only takes a couple hours and it is fun! For those hackers who are brave enough, here is the guide that I used:

http://www.gelens.org/archlinux-mailserver/

Sending mailservers typically try to redeliver mails for several days if something goes wrong (and if your mailserver does not respond with a permanent error).

Because the last thing I want to do is worry about managing my mail server, I have better things to do that ensure that have spam and webmail configured as nicely as Gmail.

Every time I email someone else who uses Gmail I'm right back to the same predicament.

deliverability.

I can't post to W3C administered mailing lists from my real email account because my domain gets bounced.

I've lost out on business deals because mail from business account gets blocked at the firewall in some places.

You can avoid all that with gmail.

Surprised no one has mentioned FastMail [1] already. Excellent service, reasonable price.

It's good to be a paying customer, so if something goes wrong there is a sense of accountability for the service provider to remedy the situation.

With Gmail and other free service providers, remember that you are not the customer, you are the product -- for advertisers.

1) http://fastmail.fm/

This is a pointless comment, but with all the whining here I feel like making a counterpoint.

I run my own mail/irc/web/file/etc -server. Postfix on OpenBSD with maybe 10 config variable changes to add a DNS blacklist of my choice + secondary MX for a friend. FDM for sorting mail and piping stuff through bogofilter. Mutt and SSH to read mail from anywhere.

I post my email address everywhere and get about 10-15 spam mails per day that escape the blacklist. Of those, 99% are sorted out by bogofilter. I scan them for false positives like I scan other mailinglists for interesting topics. There are about 700 mails from various ml's every day. The maintenance time this setup "costs" is recovered in less than a week of not having to click through the slow gmail interface. There is nothing to do. Backups are done via rsync and cron, but they have to be done anyway.

Point is, running a private mail server is easy. But then I don't call myself hacker...

The maintenance time this setup "costs" is recovered in less than a week of not having to click through the slow gmail interface.

Nothing forces you to use the Gmail interface to use their mail servers, though. I certainly don't, unless I'm away from my machine.

If my accounts were shut down and all my email handed over to any 3 letter agency on earth, I'd be mildy annoyed. Dare I say, inconvenienced. For like a day. There just isn't anything in it worth worrying about.

I can be up and running on another provider in minutes while they wade through my Amazon.com receipts and bug reports for CRUD apps.

This is actually a pretty good indicator that integrating PGP in a usable "it just works" fashion might be a really good thing to do these days. If the question is why are you allowing google the ability to violate your privacy, the solution of running your own mail server is only valid to the extent that we acknowledge the current reality that the vast majority of people do not use encryption or signing for email correspondence.

How about a simple sign / encrypt / forward service for untrusted (ie, pretty much everything) hosted email accounts that takes all incoming unsigned/unencrypted mail and at least encrypts it before delivering it to your untrusted mail server? Of course then the issue is how do you prove that you're any more trustworthy than google or any other party, but it's an interesting problem to consider.

Email isn't secure and should not be treated as such. None of the author's suggestions change this. The only way to be sure private email communication is private is to use encryption and only communicate over email with others who will do so and will keep their system secure.

Sorry, when I just throw two links in, but the people who like GMail for the way you handle your emails with it there are at least two interesting projects.

http://sup.rubyforge.org/

http://notmuchmail.org/

I live in a world where I expect and need email access from anywhere, any computer, any mobile device. I also expect tight integration with calendar and contacts. Those requirements mean I need webmail and ActiveSync. IMAP Idle (push) has reasonable mobile support, but CalDAV and CardDAV have limited support -- so ActiveSync it is. ActiveSync is non-existent in open source software. The best webmail client I've used that can be used with any IMAP server is Roundcube, but it doesn't have calendar support. Zimbra is the only open source software I've seen with good mail+calendar+contact integration, and it has a good web client. Perfect, except the open source version doesn't support ActiveSync, and is missing the backup utilities. The commercial version is a minimum $399/year. Fine, I'll pay it.

Now I need to host it. I want my email to be highly available. So I want to cluster two Zimbra servers in one data center, and cluster two backup MXs in another data center to store and forward my mail if the Zimbra DC drops off the net. Zimbra supports HA clustering with Red Hat Cluster Suite or through the HA clustering of virtualization like VMware. RHCS requires a highly available SAN as backend. Damn, an HA SAN is going to be expensive. Maybe I'll do VMs. But virtualization products like VMware and RHEV will require an HA SAN or NAS. Fine, I'll build an HA NAS with open source.

I'll deploy two ZFS heads in an active-active cluster... wait, clustering will require purchasing licenses from a company like Nexenta. Or I can get active-passive from iXsystems. So I'll buy the cheapest Nexenta license which is $1725, plus the HA Cluster license for $4900. I'll connect those heads to a dual-controller SAS JBOD which only costs $3795.

I'm now going to fly all this gear to a non-US-friendly location to install it in a data center. Each piece of hardware as redundant power supplies. I will connect the power supplies to separate power distribution units, the PDUs are on separate circuits. Each system has dual Ethernet going to separate switches, which in-turn uplink to dual-clustered routers, which uplink to the multi-homed internet provided by the data center. Because I'm on the other side of the world I'll also have a console server, and the PDUs can remotely cycle power.

Awesome! I now have the email setup I want, and it only cost me $30,000 and a month of my time!

Or I can sign up for Google Apps, where they will host my domain's email, and provide all the features I want, and have a better interface, and not require maintenance, and have better uptime, all for free!

I've administered old school sendmail+procmail setups without webmail. I've administered Postfix+Dovecot+Roundcube+LDAP. I've administered Zimbra. I've administered Google Apps for Business. It isn't worth my time and money to do anything other than Google Apps.

Judging by the replies, it seems to be a combination of preferring webmail and having poor ISPs.

Personally, I have run my own email server for several years now and have found this to require almost no maintenance. On the server side I use Postfix and Dovecot on Debian, and on the client side I use Thunderbird. For security, I only allow local IMAP connections and tunnel IMAP and SMTP from my laptop over SSH tunnels set up when I log in, and the server only accepts public key authorisation for SSH.

IMAP means I get any email within seconds and Thunderbird automatically sifts away 95% of what little spam I receive. The server is under my desk and the only downtime I have is for kernel upgrades and problems on the ISP end, which are both rare.

I just stayed up all night setting up my own on Linode. I barely know anything about servers. I wanted to do it for months and have tried for weeks.

I use a lot of google stuff and wanted to move off. I trust Linode a lot more than I trust google.

Now roundcube is working and I am a happy camper.

Well, to answer the question head-on. I am a self-respecting hacker and I do run my own private mail server for my personal mail.

But I use Gmail for what I think of as "impersonal" mail. This includes mailing lists such as those operated by the OP, as well as general notifications from other sites and services. I gain free storage and bulk management tools for a body of data I perceive of low value and sensitivity.

Moreover, Gmail's spam filters are better than mine (spamassassin). That has particular utility for these low-value accounts that are more likely to have the associated email address disclosed beyond my control.

For things I care more for; personal communications and so forth, well, they still land on the private host.

It's easy, safe, secure and fast - but I also have an Yahoo Mail account as backup.

And you can't beat the price, unless everybody decides to go French and fine Google for providing a great service for free when there are other companies that do it for a fee :-)...

Because I've never once found a decent no-nonsense no-ugly-web-interface-preinstalled-that-I-don't-want guide, and unlike anything else I have installed on my server, I find email baffling.

I'd switch if someone gave me a decent enough guide I'd go for it.

Frees up more time for hacking.

Pretty much this. I spend more than enough time fiddling with things as it is. Moreover, the article latches on to security/privacy whereas my primary concern is ease of use.

<code> But almost all of those lists are about very technical projects, where the only subscriber base should be people from either the IT security community, or the Free Software community.</code>

Sample bias.

Why would I be concerned about anything other than convenience with regards to the email account used for public email lists? The messages are public, my address is already likely been harvested, so anything I do is essentially public.

If I want to keep stuff private, that is maybe the time to set up my own email system. And use encryption and all the good stuff.

For a while now, I have used GMail purely as a drop box from which I fetch my mail (and delete from my account).

I'm hoping once the Raspberry Pi is released, I can get a good personal mail server going again.

Here are two reasons: it is convenient and it is easy to always have an up to date copy of GMail and other Google services data on my laptop which I then backup locally.

Setting the privacy and security issues aside, I wonder why do so many self-respecting hackers use the Gmail web interface (instead of, say, fetching their mail via IMAP and use another client). Dedicated mail clients like mutt can be much more efficient (keyboard-driven) and are a lot more customisable (run macros, pipe mail through external commands, etc.). I wonder why so many hackers aren't put off by Google's "one size fits all" approach.

Gmail's keyboard shortcuts are really pretty nice. hjkl to move around, enter to view a message, y to archive, m to mute, r to reply. That's a pretty damn good start.

off-topic, it's a little weird how it's expected that "true hackers" must become ill at the very sight of a mouse.

Yes, I know that Gmail has some keyboard shortcuts. I'm also aware that you can even customize them to some extent, but the flexibility has nothing to do with mutt's.

To give a concrete example, I am moderating some institution's mailing lists through a pretty exotic system, and I rely heavily on my mutt configuration to accept/reject mail and prepare rejection mail. In one keystroke, for instance, I prepare a reply to a certain MIME part of an incoming email notification and pipe it through a script to add boilerplate text and edit the message headers automatically. I wonder why so many hackers prefer Gmail's convenience to this sort of flexibility.

As for the mouse, like the keyboard, it is a tool which is appropriate for certain tasks and less appropriate for others. I think the keyboard is more appropriate to deal with email.

Two notes: you appear to be unaware that gmail can be fully keyboard driven (press ?). Second, after years of using mutt I very, very, rarely miss the extra features: 99% of my use was doing things like viewing attachments or links, which is implicit to webmail

It's quite simple: "self-respecting hackers" care more about getting shit done than futzing around with their email service. Gmail (for the most part) Just Works.

It obviously boils down to "good enough" and "free enough" but I've planned on moving back to Postfix/Dovecot at home for the reasons outlined, just a time thing.

I've been thinking of doing this myself as I've heard great things about Dovecot. An ideal setup for me would be a VM hosted somewhere in the cloud (presumably my hosted VM can't be warrently searched in the same way as email), running a barebones Arch+Postfix/Dovecot. The only inhibitors are cost of hosting and time of setup, if anyone has information on either good hosting solutions or setup information I think it would be a useful addition to this thread.

Note: These instructions look fairly straightforward. http://www.howtoforge.com/arch-linux-mailserver-with-postfix...

Running out of the home may be seem a bit asinine but the home is still somewhat sacred with respect to search warrants in the US and moving email around is insanely fast if you work out of the home.

Of course you'll likely need a business class connection with static IP and configurable rDNS and no port 25 block so not everyone will be lucky enough to be able to do it. But something like a plug computer will keep the cost pretty reasonable if you don't currently have a home server and for me it's cheaper than a VPS + consumer cable connection.

i tried running my own mail server for a while. it's mostly tedious, but there was some settings and configs to poke at. the problem is that playing with configs and live mail servers don't go together so well, half the time my email server was down and mail wasn't getting through because i was playing with things that shouldn't be played with.

so now i use gmail. no matter how much i try to screw that up, my mail still comes through.

Maybe I wasn't a self-respecting hacker when I first signed up for Gmail.

I prefer to not risk missing emails (from switching addresses to non-Gmail) me over the "hacker way".

> Control over your own data means you own it, you have it on your hard disk,

As a self-respecting hacker, I was able to put in a 'getmail_fetch` call to my crontab.

I use my own server for personal mail, and gmail for mailing lists and a backup address. I'd show up as a non-self-respecting hacker to the OP.

Whatever people may say, I run my own mail server on my DSL line at home (static ip), on a regular cheap PC. It's also the gateway, web server and a few things like that - since 2000.

It's currently running Dovecot, Postfixc, Roundcube, MySQL (even thus I plan to switch to PostgreSQL since, 10 years, but I'm lazy :P), amavisd-new, spamassassin. It's setup with SPF records and DKIM. I used to have grey listing as well, but I turned that off a year ago.

I have several emails, and friends, family also, hosted there.

* Spam: The "spam advantage" from Google's gmail makes me smile. I get less spam on my accounts, even the ones for which I post my email everywhere, than on my gmail account (which I never use anywhere, or even use for mail - its just my Google account - so its been guessed or used through OpenID, or Google leaked it somehow). Even with grey listing off.

Some guys seems happy with their 2 spam a month on Gmail. Well I get zero spam a month. And I almost never have false positive either (happened maybe 5 times in 12 years? and those were very spam-looking emails from automated services which i actually wanted to read)

* Maintenance: That makes me smile as well. Sure, I update the server every month or so. It's a 10min task. Stuff don't break. And yes, I know my stuff also, which helps. Backups are done via duplicity to a friend's system, and to a separate drive (had to switch the drives once)

Mail stay alive 5 days if box goes down. 12 years, did get network outages sometimes, never lost a mail. Not one. Sometimes, Ive friends with similar servers and we MX each others for safety, but that was not necessary up to now, because its never been down longer than a day (again its always "network outage" or "power outage" kind of issues).

* Interface. Well, I can understand it, but I don't care. Why? Because I very rarely use the web interface. I use IMAP clients and ssh+IMAP (mutt). Sometimes, I do use the web interface, and RoundCube is actually very decently good, even if its not super complete. It's fast and enjoyable at least.

* Domains refusing to talk to me Well, this has happened sometimes in the past. I sent requests for white list, and it got granted 99% of the time (in fact, its been only denied by trendnet). When denied I set the transport to use my ISP's SMTP. I haven't had to change those settings or make request in the last past 5 years or so. Again, its a much smaller deal than what you're lead to think.

I'm a self-respecting hacker and I use Yahoo mail. I guess this would put me even lower in author eyes.

It's just convenient (where I can access the Web I can access my email, no spam). As long as I am aware of the that fact, I can encrypt my mail if necessary or use alternate means of communication (such as a different email service possibly my own, etc).

To add to this article: Why do people not use PGP or S/MIME?

I've submitted my resume to a couple of people who posted in the Who's Hiring thread and checked for each of them if they have a PGP key submitted to the usual key servers (e.g. pgp.mit.edu). None of them had done that. Beats me.

I don't have a key because no one has keys ;)

I had a similar experience recently. Hardly anyone seems to do that. I shouldn't be too surprised, I guess. I had never used gpg to communicate with humans up to that point myself. But I had imagined that all the cool kids did.

When I did use PGP and just signed my mail, people regularly asked about my garbled message (pgp/inline) or broken attachments (pgp/mime). So, fix everybodys email client first, please.

There is really no alternative. Set something up with the same features and I'll gladly pay to use it.

I think I might switch over my email accounts, but there is also an issue of how long will these services be around for.

I had an Altavista account until they closed their email service. So, I need a reliable provider that wont be swallowed up by a bigger company and change my email address.

Get your own domain?

Because it works, it's reliable, it's nice to use and it's another thing I don't need to worry about.

I'd like to be off of Gmail, but I don't know of a good alternative. I'd pay a couple bucks a months for a viable Gmail replacement if I really trusted its security and reliability, and if the UI, keyboard shortcuts, feature-set etc. were just as good.

In the UK you are legally responsible to save all mail forever (if you run your own mailserver), in case the police want to search it. For this reason alone I never use my own mailserver - my content is actually MORE deniable this way.

According to Wikipedia that is wrong: http://en.wikipedia.org/wiki/Telecommunications_data_retenti...

Roundcube is getting really good for running your own webmail http://roundcube.net/screens

What about Conversation View and Priority Inbox?

Roundcube can take plugins too - someone could write them.

Because hackers want to work on interesting problems not ones that are solved for free by someone else.

Because the word hacker isn't a badge of honor.

".. sites like sourceforge ..."

articles showing its age

One word: Spam

Why don't all people make their own food?

The same argument applies to almost any category of product, with varying ramifications unique to each.

If I made my own soda, I'd leave out high fructose corn syrup. Why do I let Coca Cola poison me? And so on and so forth.

Hansons and Jones Soda man, they are so much better.

> Why don't all people make their own food?

I do not think that this is a valid comparison.

It would be more like, why don't farmers produce their own food (and rely on packaged food instead).

Most farmers don't produce most of their own food.

The modern world is built on specialization: most farmers have specialized in some direction or another, and the ones who haven't usually make less money. At best, a farmer with livestock will occasionally take a pig or cow off to be butchered (since butchering is another specialty entirely, needing specialized equipment to do well), or a farmer growing corn will plant a few rows of sweet corn in a field near his house, or a farmer growing produce -- usually just a half dozen varieties -- will keep back a little for himself.

But the calories a farmer will produce for himself are generally a small part of his intake. In the end, it's a better use of his time to focus on producing as much of a handful of crops as possible and buy what he needs from a grocery like the rest of us.

Because we could fu*n care less about what web mail provider we use, and Gmail is comfortable enough.

Reply on news.ycombinator.com