273 karma · joined March 18, 2014
What I mean is that if my output 'sentiment scores' are skewed towards the negative and centered around a negative number (~70% of my documents are scored as 'negative sentiment' using the above scheme) then I know that my model is broken because I know that my document "test set" is necessarily neutral (or even positively skewed).
Mathematically, I need to ensure my 'cost function' is a reflection of reality which means my sentiment scores at the end of the day need to be symmetrical or slightly positive skew with a mean of approximately 0.
I can use regularization to trick the model into looking like this but I don't want to overfit and I can't think of a theoretical reason that I'm getting negative bias for this simple model except perhaps that my positive corpus is missing 'true positive sentiment' (ala a parameter versus an estimate in frequentist statistics) which could be the byproduct of the simplistic 'bag of words' assumption (words are assumed independent) aka breaking down my analysis on the basis of 1-grams (single words). As per my other comment, 2-grams could be the more natural way to express positive sentiment while 1-gram negative words are more readily available. Perhaps this is a sign of our pessimism as a species haha :)
A simple 1-gram model like in the question does not model many complexities of natural language e.g. negation ("not bad" != "bad") so you would expect your estimator to over-represent the dictionary with more words that are equal to their adverb-adjusted equivalent. e.g. "not bad" can be described as 'terrible' more readily than 'very good' can be described as excellent since people assign a hyperbolic weighting to their own happiness (utility theory 101)
The sentiment would only tend to a normal distribution if we had perfect estimators for document sentiment which requires advanced POS tagging and models more complex than a 1-gram bag of words aggregation :)
I am familiar with logistic regression (studying to be an actuary) but the problem is that my documents are unlabeled: I literally have 2000+ unlabeled documents and a list of positive and negative words.
I'm willing to label 10% (~200) documents but how should I 'score' them? On a scale of [-1,1]? Just {-1,0,1} for negative, neutral, positive? How do I create a training set?
Also can you point me in the direction of some of the implementation details e.g. how do you translate text into logistic regression model?
I would also like to implement POS tagging and 2-grams (e.g. "not bad" != "bad") - any advice on incorporating this into the system?
Thank you for your input!
A persistent lack of sleep can cause your body to shut-off non-essential processes e.g. hormone regulation. My testosterone (I'm a guy) was abnormally low (<5th percentile) and it was largely due to lack of sleep, poor eating and 0 exercise. Also I took a testosterone cream supplement for a couple months and literally within weeks of this + gym + healthy food, I put on 10% my weight of muscle and magically had twice the energy I had before. I literally feel like life was a dream before changing my daily routines to keep my testosterone up (cold shower, heavy weights 4x per week, eat protein rich big meals with veg, etc. ) and keep myself sane (20min mindfulness meditation first thing in the morning). That's the other thing: your body functions best while the sun was up (we have had electricity for ~0.001% of man's complete lifetime so our bodies didn't evolve to function at night and we've only had lights for like 100 years. Before that you made a fire and had to chill in one spot!
TLDR: no sleep + poor diet = messed up hormones = low energy ... get your blood tested at your GP!
Definitely going to look into n-grams for production implementation but right now trying to resolve the negative bias issue
I know simple counting not the greatest approach but I started out by trying to replicate a research paper put out by a stock broker (not the most advanced research haha!)
I would love some suggestions for a method that does work :)
Will implement and report back
I've looked into using Naive Bayes but my understanding is you need labeled training documents and then I face the problem of scoring documents which introduces subjectivity compared to just counting the 'sentiment words'.
I understand complexity is needed to deal with negation ('not bad' != 'bad') but I'd imagine that the sentiment scoring process would be the same regardless of algorithm which brings us back to the problem of how to correct bias in 'word list' asymmetries
Interesting reflection on society if there are more 1-gram ways of communicating negativity than positivity e.g. I'm more inclined to say 'terrible' for something very bad while it feels more natural to say 'very good' than 'excellent'. If that makes any sense :)
I suspect the bias currently giving me P(detected|negative)>P(detected|positive) is resulting from my simplification to looking only at 1-grams
I don't think asymmetrical test-sets would be a problem if I had training data for documents since you can reweight to compensate - it would seem my problem is that over-representing the universe of matches for negative points due to a bigger 'negative word list' is introcucing bias and I'm not sure how to solve that.
Please see my reply on reweighting in this thread (if you reweight positive words to normalize the over-represented negative word count then a neutral sentence will have a positive sentiment score)
Will implement and see if this fixes the bias introduced by asymmetrical corpus sizes.
Fundamentally, I'm not sure there is a 'solution' available or even necessary to the issue of a bigger negative word list than positive word list. So what if there are more ways of saying negative things, all that matters is how many times positive or negative things are said. My problem then, however, is even if your range of terms that can create a match is greater for negative words, you should still have representative positive/negative counts and that doesn't explain why I'm getting much more negative sentiment scores than would be expected given my test documents.
Totally agree - definitely need to do something about negation e.g. "Not bad" != "bad"
My understanding is that this is usually handled using a list of adverbs e.g. 'not' / 'very' ('very bad' > 'bad') etc.
Not sure of a better approach than word counting though?
I think you mean to upweight my positive list by 6 (since it is 1/6 of the size of the negative list) but the problem with this is the same as my reply to the other comment where you just shift the bias:
Consider the sentence: 'there are strong and weak divisions in company X's Europe operations'
The only word matches in your word lists are 'strong' on your positive list and 'weak' on your negative list.
If you weight these counts as you describe, your sentiment for this sentence will be -0.44 + 1 = 0.66 even though the sentence is clearly 'neutral' and should have a score of 0.
Consider the sentence: 'there are strong and weak divisions in company X's Europe operations'
The only word matches in your word lists are 'strong' on your positive list and 'weak' on your negative list.
If you weight these counts as you describe, your sentiment for this sentence will be -0.44 + 1 = 0.66 even though the sentence is clearly 'neutral' and should have a score of 0.
:)
http://www.forbes.com/sites/erikkain/2013/07/02/quake-iii-ar...
Could be urban legend but fun story nonetheless :)
I got a job in finance as a quant and spent the first year building automated processes to pull portfolio holdings data from email attachments and scrape data from a bunch of different sources.
(...which is a common counter-argument for the markets being efficient. The reasoning behind passive investment is that, net of fees, it's difficult to out-perform the market although of course, everyone can't outperform the market because everyone collectively IS the market)
Rudimentary comparison: there are 422 <a> tags on CafePress's home page and only 47 on TeeSpring's
Further, once you're on a TeeSpring product's page, it's the only product there (focused call to action to buy that one product being marketed) compared to CafePress where the product page has a bunch of other "similar" and "recommended" products diluting focus.
May be counter-intuitive but adding more options seems to be to CafePress' detriment.