Comment by jedwhite | Hacker News Reader

Comment by jedwhite | Hacker News Reader

Parent Full thread

Thank you! Under the hood, there are about 80 different predictions or scoring attributes being run. A lot of those are deep learning models.

A mood or a persona is really an algorithm representing a batch of those. A lot of them you can get pretty close to just by using the Power Filter directly. But there is some secret sauce.

But broadly speaking, you can think of a Hacker persona as being someone interested in Science and Tech and Education as key broad topic areas, wanting more serious content, and wanting content from credible science-based sources. They prioritise useful content, with more in-depth coverage.

An Activist might be someone with a center-left to left stance, interested in social fields, politics and psychology, but also influenced by trends and what is currently generating a lot of "heat".

If you have representations of what good quality content "within that community of personas" looks like, then you can train models and test scoring systems that try to capture them.

Content quality is a big topic. But our working thesis is that there are canonical examples with domains of expertise of what great content looks like - PG's essays, Pulitzer Prize Winning Journalism, award winning research papers. And there are similar examples of what very bad content looks like - hate speech, extremism etc.

So that's been the broad basis for the models that underly the predictions.

[edits for typos]

Just wondering if you could provide more details on your model like do you use a CNN or an LSTM? Is there a paper you have written about it? Also how do you get the data for training your models? Also do you do any retraining as new data comes in? Do you also have some groundtruth to measure the performance of your models?

Thanks, there is a little more information in my reply to abrichr above. But in short, there are a large number of predictions run on each piece of content, and they each use different approaches - including traditional statistical models and ML, off-the-shelf APIs, as well as custom deep learning models we've developed through experimentation. So there's an evolving mix of models using different approaches (including CNNs, LSTM-like approaches / word2vec and some topic modelling using a doc2vec-style approach). It's still very much experimental though, and some predictions are way better than others!

I'd love to do an arxiv.org paper or mlconf presentation down the track, as I think we have some interesting work here and I have an academic interest in the field, but it's way too early. In essence, we're working this out as we go to "make the thing" as practitioners first based on some unique insights, rather than it being a product derived from research first.

[Edits for typos/clarity]

I just noticed your post (https://news.ycombinator.com/item?id=18050422) about courses but cannot comment on it because it is too old. I am an employer with strong opinions. If you'd like to hear them please e-mail me here: https://www.idtechex.com/contact/team/dr_jon_harrop.asp

Thanks. Makes sense. Then is definitely commercial opportunity I’m sure :)

Reply on news.ycombinator.com