Hacker News Reader

I have to imagine that's not available to avoid training by commercial entities. https://www.transformativeworks.org/ai-and-data-scraping-on-...

Makes sense. Somewhat strange (but not too strange since it's not standardized) they don't have a Crawl-Delay except for Slurp. Why not on User-Agent: *

Confusing. Would even prefer a commented crawl delay that specifies in human language since then I could special-case it.

I feel like I am coming in to the middle of a discussion. Towards what end?

Just data hoarding tendencies, faster and better search, etc.

there's some dumps on archive.org

i worked on a project a long time ago and downloaded a bunch of it

Thank you. Do you still have your archive? I'd email you if you had address on profile.

I think I uploaded the only archive I had