Hacker News Reader

Comment by jph | Hacker News Reader

This is a great project and tutorial. IMHO you have a value prop that's much larger/better than what you're describing here.

To me it sounds like you're creating a data mining annotation tool that can work on any large corpus of free-form documents that have discoverable labels, such as medical records, legal cases, press releases, SEC filings, customer reviews, etc.

Can you speak to any of these? And do you have a pitch deck or similar ask for funding/help/advisors?

View on HN

Thanks so much for your comment! You're right that this annotation tool can be used on any form of free-form documents found online. I tackled Wikipedia first because it was an obvious first choice and they have an API to read the html. This could be opened to other sources of data, but I also do not want this to become a scraping tool, so we would need to weigh costs/benefits of adding new data sources. The additional cost of adding a new source is mostly about how difficult it is to read and parse the content. In the future, I could integrate with some paying sources (e.g. news publications), where people have to pay for the content they scrape & label.

I have a pitch deck and I'm looking for all the things you mentioned :-). I can send the pitch deck to anyone interested.

Similar question, the CSV contains a collection of Wiki URLs, why not pass URLs of any website that has "free-form documents"?

The only issue here is what I mentioned in the comment above: how easy is it to read and parse the content of said website and is it legal to read the content programmatically? Do you have any website(s) in mind?

Reply on news.ycombinator.com