Hacker News Reader

Comment by grinich | Hacker News Reader

It's not a ploy-- just a design choice. The cloud sync engine does a lot of heavy lifting so N1 is faster and efficient. It's also open source free software so you can run it yourself! https://github.com/nylas/sync-engine

(I work at Nylas.)

View on HN

What sort of "heavy lifting" does this do that Thunderbird or Outlook don't? Certainly they're both performant enough. And more importantly, is there any compelling reason to introduce a 3rd party server to do this instead of doing it locally for better user privacy?

I understand I can run it myself, however I have to question the design and motivations of the developers.

Probably the biggest thing is that it translates a traditional document-based data structure (mailboxes via IMAP) into a stateless, streaming transactional data structure. This allows N1 to be way more efficient and faster, and only load about 2% of the total data in your mailbox.

The sync engine is essentially an abstraction layer for IMAP/SMTP/MIME/etc. This architecture has become pretty common in the past few years: Mailbox, Outlook Mobile (Acompli), Spark, Cloud Magic, Gmailify, and many more run similar systems. Although as far as I know, we're the only one who has open sourced our backend.:https://github.com/nylas/sync-engine

We actually released the API layer before N1, and lots of developers use it for their own apps and pay us to run the infrastructure. If you're curious, the full docs here: nylas.com/docs/platform

You can pretty easily set up the sync engine and run it yourself, either locally or on a VPC, and connect N1 to that instance. We do this ourselves when developing, and it's the best way to have 100% control over the entire stack. More info here: https://github.com/nylas/N1/blob/master/CONTRIBUTING.md#runn...

There's a bunch of community developers than hang out in our chat room and can also give you some tips for getting started. You can join that here: http://slack-invite.nylas.com/

> Probably the biggest thing is that it translates a traditional document-based data structure (mailboxes via IMAP) into a stateless, streaming transactional data structure.

So why can't this be done on the client side? Are you telling me that to avoid storing more data on the clients, they should give up their privacy to whoever operates the server? Not like Outlook or Thunderbird take a substantial amount of time to do most operations. Is this a good tradeoff for them?

I understand, yes, technically adept users can run the server, but there's still many who won't, who will use your server or someone else's server and be required to make this tradeoff.

Of the products you mention - yes, yours is the only one I'd consider and for this very reason, I can run the middle server myself. I just have to think about those people who won't do it though.

Haven't such users already "given up" their privacy by having their email hosted by a third party?

For some, it's adding another party, for others it's introducing a 3rd party at all. Still a problem either way.

>I understand, yes, technically adept users can run the server, but there's still many who won't, who will use your server or someone else's server and be required to make this tradeoff.

So? How's that different (if anything it's much better) than using Gmail or Outlook on the cloud?

Besides, nobody is "required" to do anything. You can always run some Thunderbird or any other desktop client without a cloud component.

I'm also a big fan of IMAP and would love to see everyone keep to it. At the moment I haven't found an IMAP client better than trojita (http://trojita.flaska.net/) which actually uses IMAP the way it was designed to and not as a poor man's sync mechanism.

However, there are things that IMAP can't do:

* Search has to be mailbox by mailbox. So if you have a huge number of mailboxes it's going to take a long time. There is, of course, an extension for multi-mailbox search, but its very low adoption hasn't made the thing practical

* Read notification just can't be done with the traditional email system, short of replying with a specifically-formatted email and introducing some kind of RPC

* Send Later while offline can't be done with the traditional email system. There's just nothing you can do for that.

>I'm also a big fan of IMAP and would love to see everyone keep to it.

Probably because you've never read the spec or tried to implement it.

Or, more probably, because I've actually read the spec and understood what IMAP really tries to do, saw how widespread its usage was in so many languages, and imagined what could be possible with a protocol we already have. I have toyed with it in the past already, and intend to do in the future because just everybody has it, with or without their knowledge, and can use its power.

Yes, IMAP is messy and inconsistent and hard to parse, if you blindly read examples and try to build a naive parser out of that. When you're trying to do as much as IMAP tries to do starting from scratch, that's barely surprising. Still, IMAP already has many tools, libraries and software that can be re-used.

>Still, IMAP already has many tools, libraries and software that can be re-used.

That's the problem though, those tools and libraries only give you IMAP-like behavior, and modern mail clients want to go beyond that -- from tagging to snoozing and lots more besides.

And IMAP makes it a hell to have something more intelligent on top.

No, they give you IMAP capabilities; it's up to the client to build something more intelligent on top of that -- tagging and snoozing are totally compatible with IMAP as long as the client does something intelligent (it doesn't even have to be extremely non-standard; IMAP keywords and mailboxes can do the job perfectly here). Your mail client doesn't have to be a 1-1 mapping to IMAP commands, like pretty much all mail clients are.

This problem is the same as GPG. Why is GPG not more widespread ? It is widely accepted that the technical parts may be old but are still solid, the real problem is the UI. It is still immensely difficult to create your own keychain and share your keys with someone else. Only in the latest versions does gpg introduce TOFU. It has always been a UI problem, and will remain this way until someone wants to tackle it... or replaces gpg.

>I understand I can run it myself, however I have to question the design and motivations of the developers.

What exactly do you feel like you "have to question"? You understand that Gmail and Outlook (the online versions) are hosting hundreds of millions of email accounts and store everything on the cloud too, right?

Yes, but Gmail and Outlook are "original" providers. I feel very uneasy, and find it unreliable and untrustworthy to allow another company access to my emails and contacts.

An email client for me is one that runs on my desktop, and talks to my "original" provider, nothing else.

Reply on news.ycombinator.com