Hacker News Reader

Comment by mostafah | Hacker News Reader

Email protocols are very old and network APIs have improved a lot during the last two decades. So, when FastMail (I’m a happy customer) introduced JMAP I was excited about it. But even then I didn’t anticipate much success for it. Big email providers (Gmail, iCloud, Outlook, …) don’t care about these things. And it’s a shame that email providers and clients are still wrestling with ancient protocols to provide a decent experience for saving drafts, search, folders/labels, and other basic things.

View on HN

I hope it gains more traction. In the mean time, I've switched, to Fastmail, and I couldn't be happier. It really is much faster than Gmail, I was very surprised because I didn't think email could be so fast. I guess Gmail had become the new normal.

Except for search really. I love FastMail but they need to work on the search speed. Atleast it's faster than Outlook.com though, that search speed is beyond unacceptable.

If you open a support ticket I can look at your search issues, ask for Bron. It shouldn't be slow unless you're deliberately bypassing the Xapian indexes by using substr searches.

How long is search supposed to take? Entering something common (like "gmail") takes around 5 seconds for me with 32k results. My old google mail account takes 1.5s to display the first page.

Looking at the Cyrus code, the server doesn't even start sending search results to the client until it has completed running the search. And due to the response being JSON, and there being no pagination in the protocol, presumably the Fastmail client wont start displaying the results until it has retrieved them all either. So I'm not surprised it's slow.

Are you looking at JMAP getMessageList here? It paginates the response (offset/anchor and count), but yes - it has to build the complete list. Not that we're using that in FastMail production yet, we're using XCONVMULTISORT still. It's roughly the same algorithm though.

The biggest expense is all the index.c stuff to support IMAP protocol requirements. There's a lot of cost in there which will go away with JMAP, and which will go away even more when we replace the separate cyrus.index files with a structured database per user. That's still got a ton of work to do though... one of the goals for long term Cyrus improvements.

Does this allow for the client to say, "Give me all results matching x", and for the server to say, here's the first N results, but there are potentially more, and then the client can then ask for more, in multiple requests until it has them all, allowing them to progressively show results until they're all retrieved?

That's how I would expect a JSON based protocol to work for pretty much all types of requests that provide potentially large lists in a response. I would expect getMailboxes to work this way too to be honest.

Yeah, because we fill out the whole count rather than returning just the first page, a search that returns tons of messages will be slower than gmail. You can limit the set of returned messages with time ranges.

One of our slack messages is something that's become a bit of a cliche! "JMAP will fix it" - in this case, a lot of the cost is generating the message counts by iterating every match, and that's not going to be cheaper in JMAP. We don't have data structures that allow sorted partial search responses, so we can't generate the first page until we've resolved the search for every message :(

Reply on news.ycombinator.com