nickdavidhaynes

http://automatedinsights.applytojob.com/apply/oMu2ojUP8M/Sof...

I'm the tech lead for the R&D team at Automated Insights, a company that turns raw data into human-sounding narratives. Our SaaS platform, Wordsmith, generated billions of articles last year for companies like the AP, Yahoo, and Activision.

The team's responsibility is mainly algorithm development (machine learning and natural language processing, as well as more traditional methods), with the goal of making Wordsmith more powerful and easier to use. We primarily develop in Python (spaCy, gensim, scikit-learn, Tensorflow) and NodeJS. We're looking for an engineer with a couple years experience and familiarity with machine learning. A research background and/or advanced degree is a plus.

Interview: Quick chat with HR, technical phone screen, at-home programming evaluation, on-site interview. No adversarial whiteboard sessions or trivia quizzes.

Here's the link to the study again: [1]. This is specifically in reference to the reporting on quarterly earnings reports that we automate for the Associated Press. It's an objective summary of the financial performance of these companies that appears in news outlets across the country (for example, [2,3,4,5,6]). The companies being reported on have no influence over the content of the articles.

From the summary:

>These articles synthesize information from firms’ press releases, analyst reports and stock performance, and are widely disseminated by major news outlets within hours of publication...This study found a positive effect between the public dissemination of objective information and market efficiency.

[1] https://insights.ap.org/industry-trends/study-news-automatio...

[2] http://www.thenewstribune.com/news/business/article158779784...

[3] http://wtop.com/business-finance/2017/05/yum-beats-street-1q...

[4] http://www.foxbusiness.com/markets/2017/04/27/dominos-sales-...

[5] http://www.businessinsider.com/ap-fedex-beats-street-1q-fore...

[6] https://www.usnews.com/news/business/articles/2017-04-26/her...

As far as visuals vs prose, I see it as "both-and" rather than "either-or". And in addition to our journalism and personalization work, we also integrate with interactive visualization tools like Tableau.

Many of your criticisms are totally valid. Lots of the phrasing is awkward - even the lede is really bad ("Tesla, Inc. (TSLA) has been having a set of eventful trading activity"...wat). And it feels really deceptive to put a human byline on an automated article.

We're pretty open about the fact that our solution to this problem is not "magical" at all [1, 2] - it's good, old-fashioned automation. This approach allows our customers to QA their content heavily before pushing it to production, which eliminates many of the problems with awkward/incorrect phrasing that people who rely more heavily on machine learning tend to run into. And the news articles we publish always have a note at the end saying that they were generated by Automated Insights, and don't include a human byline.

There is real value in this type of reporting - a recent study [3] found that the articles we produce for less well-known publicly-traded companies has increased the trading volume for those companies. The idea is that, yes, the content is fairly formulaic, but there's now reporting on companies that had very little coverage before we existed. There are similar arguments for mass personalization work we've done for companies like Activision Yahoo - having prose that describes raw data (even if it is formulaic to an extent) is often better than not having prose.

[1] https://automatedinsights.com/blog/the-state-of-artificial-i...

[2] https://automatedinsights.com/blog/creating-great-automated-...

[3] https://insights.ap.org/industry-trends/study-news-automatio...

http://automatedinsights.applytojob.com/apply/oMu2ojUP8M/Sof...

I'm the tech lead for the R&D team at Automated Insights, a company that turns raw data into human-sounding narratives. Our SaaS platform, Wordsmith, generated billions of articles last year for companies like the AP, Yahoo, and Activision.

My team's responsibility is mainly algorithm development (machine learning and natural language processing, as well as more traditional methods), with the goal of making Wordsmith more powerful and easier to use. We primarily develop in Python (spaCy, gensim, scikit-learn, Tensorflow) and NodeJS.

We're looking for an engineer with a couple years experience and familiarity with machine learning. A research background and/or advanced degree is a plus.

Interview: Quick chat with HR, technical phone screen, at-home programming evaluation, on-site interview. No adversarial whiteboard sessions or trivia quizzes.

>Now, instead of humans designing algorithms to be executed by a computer, the computer is designing the algorithms. (Albeit guided by human-devised algorithms)

This line is way off in both tone and substance. On tone, it really underplays the human effort involved in effective machine learning (as it is practiced in 2017) and anthropomorphizes "machines" to an unreasonable extent. In substance, I fail to see how a machine that "designs its own algorithms" according to an algorithm designed and implemented by a human is fundamentally different than an algorithm coded directly by a human. To use the author's example, machine learning allows humans to build complex software systems in less time just as a bicycle allows humans to cover more distance with less energy. It's a big improvement, but it's not, say, teleportation.

>it is only now that the machines are creating themselves, at least to a degree. (And, by extension, there is at least a plausible path to general intelligence)

I could not disagree more strongly with this addendum. Simply put, I fail to see any path from state-of-the-art ML/DL research today to AGI, and I would even go so far as to say that humans have made approximately zero progress on this task since it was first formulated in the 50s. I think we know about as much about "intelligence" (and consequently, what would constitute AGI) as star-gazers in ancient times know about the universe. That's not to say that it will take millennia to invent AGI, but the path to get there is probably quite orthogonal to modern ML research.

Coincidentally, I wrote a blog post [1] that went up just this morning that, in part, compares spaCy with the other giant in the Python NLP ecosystem, NLTK. TLDR - I think that, right now, the majority of users are better served by spaCy than NLTK.

[1] https://automatedinsights.com/blog/the-python-nlp-ccosystem-...

You've never met a number theorist.

I think the way you're using the word "theory" here dilutes its technical meaning. More accurate to call it speculation or philosophizing.

-Some candidates can write great code, but don't have the math background to understand what ML black boxes are doing.

-Then there are STEM PhDs that have never written non-research (i.e. maintainable) code or had to formulate a qualitative business problem into a quantitative problem they can solve.

Both types of candidates need to come in at a "junior" level and do some on-the-job learning in order to be fully successful data scientists. IMO it appears to be easier to teach STEM PhDs how to code than programmers how to do math, but that might be personal bias (since I came from the former group).

[1] http://www.tandfonline.com/doi/full/10.1080/00031305.2016.11...

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1112991/

A fundamental challenge I see here is how bottom-heavy data science feels now. There are tons of people out there trying to "get into data science" from other fields, but the number of people with substantive domain knowledge, strong programming skills, and the math background to be able to understand the ML black boxes is quite small relative to the number of people calling themselves data scientists. In other words, real insight definitely is (or should be) the goal, but real insight is really hard, and scikit-learn is so easy.

My hope is that this improves over the next 5-10 years - the more mature data science becomes as a discipline/career, the better the education will be and the more experienced people there will be. There is a risk in the mean time, though, that a flood of relatively inexperienced people causes a collapse in expectations for data science, making businesses less eager to hire them in the future.

Pretty much no way to use neural networks (except for playing, like above) without writing code.

I agree that streaming media is becoming a commodity. If Amazon made modern apps and shelled out some cash for better libraries, they could deal serious blows to Spotify and Netflix.

>The breakthrough in deep learning is to model the brain, not the world... Artificial, software-based calculators that approximate the function of neurons in a brain are connected together.

This comparison is criminally incorrect.

And don't even get me started on figures 2+3 - 2 adjacent figures, with different horizontal and vertical scales, that use highly technical terms without introduction?? It's really hard to see how a lay-reader could get anything useful out of those plots.

Despite the author's promise to "get behind the headlines" and "cut through the hype", this is just a regurgitation of "intro to machine learning" cliches.

Bottom line, when you're using an established Python scientific computing library to do numerical calculations, you're calling C (or maybe Fortran) routines.

Joel's book is a great resource for preparing for interviews or learning really basic stuff and less of an introduction to the tools.