Hacker News Reader

Comment by fenollp | Hacker News Reader

This is a really naive search method for a person whose job is to be good at this.

It's like watching the Spotlight team going through volumes of data themselves…

Isn't Data Journalist a thing already?

What's your argument exactly? That they should have used narrower terms until they found an article that was already written?

They use a syntax-aware search (thus refining for ME, MY, …). I thought (hoped) that journalists among all people would be the best at semantic search.

Here the guy is just saying "whoa great dude and next I'll use regexps oh my god woaw". We have word2vec, doc2vec, many cores, … why aren't journalists using the tools they should be using?

I would read this post if you wrote it... because I don't know what word2vec & doc2vec are and why they are useful.

If you write the post maybe someone will be inspired to build a tool for journalists using these algos.

None of the journalists I have met or worked with in six years in publishing and investigative due diligence know about the tools you are proposing. I recognize regex, because of a problem I needed to solve (that later became two problems), but I would love to learn more about the others. If you would consider writing that up, I would happily edit it for you.

That's an interesting question - if you read the stories of how Edward Snowden's story came to light, you'll find that his main issue with Glenn Greenwald was to get the guy to use technology properly.

Hopefully out of the many tech heads who read HN, one will have the same thought as you and build the tool.

i disagree. i don't think the author's search method is intuitive at all. i think, even for a journalist, it's normal behavior to search by proper nouns and words of specificity, rather than pronouns and what we normally deem "filler" words such as "me" and "my".

Oh good. Another Data X that we can debate the meaning of for the next couple years.

Their job isn't to be good. Their job is to convince the masses that they are. In order to sell that idea, it is beneficial if they believe their own bullshit, and that selects people who are not very good.

Reply on news.ycombinator.com