Hacker News Reader

How did you decide to mine this data specifically if you have trouble finding a market for it? What was the insight?

We had a client that needed similar project, but for small amounts of domains, and we had the list of domains 'just lying around', so we thought 'wait, if our client knows where to use it then there's a market', and, well - wrote some more and combined the two.

Sorry, missed the point of the question. Finding specifically links - because of that client. As we were working on his project we slowly began to realise that cross-referencing sites, links to competitors and stuff like that might be useful for someone. Also, with links u can go deeper into the site, and scrape links from all pages.

And maaybe we got too in love with the concept so now we're firmly stuck in it, finding reasons for our behaviour that are not "woohoo it's so fun to play with it and we have justification to spend time here - it's gonna work for us!"

Some time ago, I did the same thing, and doing something without knowing the full path might be worthless, so it's better to know exactly what to do in the future, having learned from my own mistakes.

Reply on news.ycombinator.com