Hacker News Reader

Comment by philpem | Hacker News Reader

The difficulty in a lot of cases is finding someone who has a complete copy of the group. Yahoo Groups also had file, photo and database features, and archives of those are likely to be incomplete. You'd have to go through the member list (primarily early members) and find someone who still had a copy of all the messages.

The other problem is making it available - I ran a Yahoo group for many years, and have Mbox and Maildir format archives. I'm still looking for a decent web-based browser for these. HyperKitty (Mailman's archive browser) came close, but seems to require most of Mailman to be installed in order to work.

In my case, I managed to archive a bunch of groups related to amateur radio -- and I will be placing these on archive.org as soon as I have a spare moment to zip them up. A difficult-to-access archive is better than no archive at all, the important part is getting the data into a safe place.

View on HN

I'm actually applying for an SBIR grant right now to work on the NLP algorithms that power fwdeveryone.com, if you have any interest in writing a letter of support. Basically it would eventually enable someone to mass export something like an MBOX archive onto the web in a cleaned up format with accessible typography. You can play around with prettyfwd.com to get an idea of the current state of the tech, it works well for 95% of (non-commercial) email threads but still needs some more work to support the rest.

I have the same problem. I used some old script back in 2006 to download a couple of groups in ... I think it's Mbox format. It's just not clear what to do with it.

Hmm... a standalone viewer for these formats (that exposed a webserver that could be accessed in a browser) sounds like it would pretty trivial given a parser for the email format itself. Especially maildir!

How big are these archives? Do you have any samples? Does the viewer need any special features? (threading?)

> pretty trivial given a parser for the email format itself

The problem is that there isn't any standard that defines what can and can't go inside the body of an email message. So if you want to post each email message exactly in a thread exactly as is, i.e. each with completely different typography and with all the replies attached and not sanitized in any way, then that's relatively easy. But it's also completely unreadable for more than about 30 seconds, and doesn't allow for good search functionality. These problems aren't a deal breaker if you're only trying to make sense of your own inbox, but when you're looking for specific information across millions of people's inboxes then they're a complete nonstarter.

I remember how great Gmane used to be with several incredible web-based views of mailing list archives. Too bad it sounds like the source code was lost and never open sourced. Another on the list of services that died without passing on enough of the torch.

Reply on news.ycombinator.com