4 karma · joined March 21, 2017
Best of luck.
Thanks!
Sweetest users is a good question. I'll have to build out that view some time.
We'll relaunch in about two weeks with an improved model that we're currently training on a much larger labeled dataset and SOTA BERT model. :)
This kind of feedback gets me pumped about continuing to work on it.
The Muffin Tops part is making me lol.
For the curious you can see the dictionary here: https://github.com/sloria/TextBlob/blob/eb08c120d364e9086467...
The package used is a pretty popular one called TextBlob. It is nifty for working with unlabeled data like we have with the HackerNews dataset.
We really focused our definition of saltiness around being a combination of (subjective + negative) comments.
We reduced the impact of (objective + negative) as we feel that criticism, while at times painful, if presented objectively isn't necessarily salty.
We built this model fast (1 week) and have since iterated this week into developing a Fine Tuned BERT model that we are training over a much broader set of toxicity, demographic, and polarity features. The training set is much larger and higher quality so we are expecting a large jump in precision upon deployment.
I hope the app gave you some good chuckles as you went around though. It's hard to explain how excited I felt when I saw pg_is_a_butt at the top of my pandas data frame the first time I processed the data.
It's doing a little bit right. :)
Thank you for trying the app and hopefully V2 will leave you feeling like the system is more precise.
It does have a LONG way to go. These are valid criticisms. And all areas we are working to improve on with our next ML model. (Sad vs Negative, rating numbers as "salty",
This model is pretty simple. It's using TextBlob and looking for a combination of negative sentiment (not necessarily condescending) and subjectivity. Essentially hand built heuristics derived by weighting each word in the sentence. Not a great way to make predictions.
The model is FAR from great. But great from afar. For high level (overall user saltiness) it performs better.
The unlabeled dataset of this size presents some unique challenges but in our testing of our new model (based on SOTA BERT fine-tuining & a large labeled training dataset) the results look promising. I'm really looking forward to getting it deployed.
I am encouraged by the words of @pg who said "you can and should give users an insanely great experience with an early, incomplete, buggy product, if you make up the difference with attentiveness.
Can, perhaps, but should? Yes. Over-engaging with early users is not just a permissible technique for getting growth rolling. For most successful startups it's a necessary part of the feedback loop that makes the product good. Making a better mousetrap is not an atomic operation. Even if you start the way most successful startups have, by building something you yourself need, the first thing you build is never quite right. And except in domains with big penalties for making mistakes, it's often better not to aim for perfection initially."
https://github.com/pim-book/programmers-introduction-to-math...