HNHacker News
TopNewBestAskShowJobs

markovbling

273 karma · joined March 18, 2014

submissionscomments
markovbling··on Topic models: Past, present, and future
Very cool podcast - thanks for pointing it out!
markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
On second thought, not sure resampling |S| words from L is a good idea because you want 100% coverage of your sentiment universe and an asymmetrical corpus is not a priori incorrect so resampling does not solve anything except reweighting match lists which may not even lead to symmetrical sentiment since the distribution of words amongst the language (TF-IDF helps here) is not necessarily the same in your negative and positive lists
markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
To be honest I haven't thought about measuring the performance of different approaches but I have thought about a metric which will signal poor performance and right now I'm interested in eliminating poor performance in my simplistic methodology.

What I mean is that if my output 'sentiment scores' are skewed towards the negative and centered around a negative number (~70% of my documents are scored as 'negative sentiment' using the above scheme) then I know that my model is broken because I know that my document "test set" is necessarily neutral (or even positively skewed).

Mathematically, I need to ensure my 'cost function' is a reflection of reality which means my sentiment scores at the end of the day need to be symmetrical or slightly positive skew with a mean of approximately 0.

I can use regularization to trick the model into looking like this but I don't want to overfit and I can't think of a theoretical reason that I'm getting negative bias for this simple model except perhaps that my positive corpus is missing 'true positive sentiment' (ala a parameter versus an estimate in frequentist statistics) which could be the byproduct of the simplistic 'bag of words' assumption (words are assumed independent) aka breaking down my analysis on the basis of 1-grams (single words). As per my other comment, 2-grams could be the more natural way to express positive sentiment while 1-gram negative words are more readily available. Perhaps this is a sign of our pessimism as a species haha :)

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
It only tends to a Normal distribution if you estimate P(negative|matches in -ve list) & P(positive|matches in +ve list) with an unbiased, consistent estimator.

A simple 1-gram model like in the question does not model many complexities of natural language e.g. negation ("not bad" != "bad") so you would expect your estimator to over-represent the dictionary with more words that are equal to their adverb-adjusted equivalent. e.g. "not bad" can be described as 'terrible' more readily than 'very good' can be described as excellent since people assign a hyperbolic weighting to their own happiness (utility theory 101)

The sentiment would only tend to a normal distribution if we had perfect estimators for document sentiment which requires advanced POS tagging and models more complex than a 1-gram bag of words aggregation :)

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
I meant -0.167 + 1 = 0.83 > 0 therefore positive sentiment :)
markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Wow, thank you so much for pointing out BM25 - hadn't heard of it but looks very cool. Implementing it ASAP.
markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
This sounds like a great idea!

I am familiar with logistic regression (studying to be an actuary) but the problem is that my documents are unlabeled: I literally have 2000+ unlabeled documents and a list of positive and negative words.

I'm willing to label 10% (~200) documents but how should I 'score' them? On a scale of [-1,1]? Just {-1,0,1} for negative, neutral, positive? How do I create a training set?

Also can you point me in the direction of some of the implementation details e.g. how do you translate text into logistic regression model?

I would also like to implement POS tagging and 2-grams (e.g. "not bad" != "bad") - any advice on incorporating this into the system?

Thank you for your input!

markovbling··on Ask HN: Working through burnout?
Check out your hormone levels - I burned out HAAAARD trying to write actuarial exams while working 10 hours and applying for YC (didn't get in ;) - had to take 2 weeks off.

A persistent lack of sleep can cause your body to shut-off non-essential processes e.g. hormone regulation. My testosterone (I'm a guy) was abnormally low (<5th percentile) and it was largely due to lack of sleep, poor eating and 0 exercise. Also I took a testosterone cream supplement for a couple months and literally within weeks of this + gym + healthy food, I put on 10% my weight of muscle and magically had twice the energy I had before. I literally feel like life was a dream before changing my daily routines to keep my testosterone up (cold shower, heavy weights 4x per week, eat protein rich big meals with veg, etc. ) and keep myself sane (20min mindfulness meditation first thing in the morning). That's the other thing: your body functions best while the sun was up (we have had electricity for ~0.001% of man's complete lifetime so our bodies didn't evolve to function at night and we've only had lights for like 100 years. Before that you made a fire and had to chill in one spot!

TLDR: no sleep + poor diet = messed up hormones = low energy ... get your blood tested at your GP!

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
but wouldn't you miss sentiment terms in the text if you sample a subset of your negative dictionary?
markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Cosine-similarity is a measure of comparison between 2 vectors so what would you use for these 2 vectors in this case?

Definitely going to look into n-grams for production implementation but right now trying to resolve the negative bias issue

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Ahh resampling |S| words from L is a great idea! :)

I know simple counting not the greatest approach but I started out by trying to replicate a research paper put out by a stock broker (not the most advanced research haha!)

I would love some suggestions for a method that does work :)

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Reweighting sentiment by looking at the number of occurrences of positive and negative words in my assumed neutral corpus is a great idea :)

Will implement and report back

I've looked into using Naive Bayes but my understanding is you need labeled training documents and then I face the problem of scoring documents which introduces subjectivity compared to just counting the 'sentiment words'.

I understand complexity is needed to deal with negation ('not bad' != 'bad') but I'd imagine that the sentiment scoring process would be the same regardless of algorithm which brings us back to the problem of how to correct bias in 'word list' asymmetries

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Definitely think I should look at using bi-grams and tri-grams

Interesting reflection on society if there are more 1-gram ways of communicating negativity than positivity e.g. I'm more inclined to say 'terrible' for something very bad while it feels more natural to say 'very good' than 'excellent'. If that makes any sense :)

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
This is an interesting approach to normalisation - will give it a go :)
markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Totally agree

I suspect the bias currently giving me P(detected|negative)>P(detected|positive) is resulting from my simplification to looking only at 1-grams

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
This is part of my problem - I don't have a labeled dataset outside of my 'positive words' / 'negative words' lists.

I don't think asymmetrical test-sets would be a problem if I had training data for documents since you can reweight to compensate - it would seem my problem is that over-representing the universe of matches for negative points due to a bigger 'negative word list' is introcucing bias and I'm not sure how to solve that.

Please see my reply on reweighting in this thread (if you reweight positive words to normalize the over-represented negative word count then a neutral sentence will have a positive sentiment score)

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Great idea! I've looked at term-weighting approaches such as TF-IDF but I don't have a training set of positive / negative sentences so would have to term-weight just the occurances of each of positive/sentiment list and compute a net sentiment on this basis.

Will implement and see if this fixes the bias introduced by asymmetrical corpus sizes.

Fundamentally, I'm not sure there is a 'solution' available or even necessary to the issue of a bigger negative word list than positive word list. So what if there are more ways of saying negative things, all that matters is how many times positive or negative things are said. My problem then, however, is even if your range of terms that can create a match is greater for negative words, you should still have representative positive/negative counts and that doesn't explain why I'm getting much more negative sentiment scores than would be expected given my test documents.

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
Haha! :)

Totally agree - definitely need to do something about negation e.g. "Not bad" != "bad"

My understanding is that this is usually handled using a list of adverbs e.g. 'not' / 'very' ('very bad' > 'bad') etc.

Not sure of a better approach than word counting though?

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
I have a list of positive and negative words and a set of documents which I want to score so not sure if I have a 'training set'.

I think you mean to upweight my positive list by 6 (since it is 1/6 of the size of the negative list) but the problem with this is the same as my reply to the other comment where you just shift the bias:

Consider the sentence: 'there are strong and weak divisions in company X's Europe operations'

The only word matches in your word lists are 'strong' on your positive list and 'weak' on your negative list.

If you weight these counts as you describe, your sentiment for this sentence will be -0.44 + 1 = 0.66 even though the sentence is clearly 'neutral' and should have a score of 0.

markovbling··on Ask HN: Sentiment Analysis – how to handle biased word list lengths?
I tried weighting the terms by the relative sizes of the corpuses (as you suggest) but the problem is you just shift the bias instead of removing it.

Consider the sentence: 'there are strong and weak divisions in company X's Europe operations'

The only word matches in your word lists are 'strong' on your positive list and 'weak' on your negative list.

If you weight these counts as you describe, your sentiment for this sentence will be -0.44 + 1 = 0.66 even though the sentence is clearly 'neutral' and should have a score of 0.

:)

markovbling··on [dead]
how did you generate the google link?
markovbling··on Charles Barkley Doesn’t Love Analytics, but Analytics Sure Love Him
smooth!
markovbling··on The Importance of Anxiety
Caffeine + Theanine together have a synergistic effect - I do 150mg caffeine pill + 300mg theanine pill as soon as I wake up - can't recommend highly enough :)

http://jn.nutrition.org/content/138/8/1572S.full

markovbling··on The Importance of Anxiety
300mg of Theanine every morning 30min before breakfast very good for anxiety and strong productivity side effects when taken with caffeine ;)
markovbling··on Testosterone is the drug of the future
Ask your GP to do a blood test - reasonably cheap and can get conclusive answer.
markovbling··on Genetic Algorithm Walkers
Reminds me of the 'bot armistice' where a guy let a bot-vs-bot match run for 4 years and the steady state was 'the bots on both teams are simply standing still, not doing anything. The server is running and the game isn’t frozen. The bots are simply standing there, not killing one another.'

http://www.forbes.com/sites/erikkain/2013/07/02/quake-iii-ar...

Could be urban legend but fun story nonetheless :)

markovbling··on How to Get a Quant Job in Finance
Spot on.

I got a job in finance as a quant and spent the first year building automated processes to pull portfolio holdings data from email attachments and scrape data from a bunch of different sources.

markovbling··on Hard-won lessons about money and investing
that's the paradox of the efficient market hypothesis - if everyone is a passive investor then who keeps the market efficient?

(...which is a common counter-argument for the markets being efficient. The reasoning behind passive investment is that, net of fees, it's difficult to out-perform the market although of course, everyone can't outperform the market because everyone collectively IS the market)

markovbling··on Teespring Says It's Minting New Millionaires Selling Its T-Shirts, Raises $35M
It's simplicity - CafePress's website is complete chaos whereas TeeSpring is dead simple.

Rudimentary comparison: there are 422 <a> tags on CafePress's home page and only 47 on TeeSpring's

Further, once you're on a TeeSpring product's page, it's the only product there (focused call to action to buy that one product being marketed) compared to CafePress where the product page has a bunch of other "similar" and "recommended" products diluting focus.

May be counter-intuitive but adding more options seems to be to CafePress' detriment.

markovbling··on Introducing Visual Studio’s Emulator for Android
there's a Starcraft analogy in there somewhere
← PreviousPage 3 of 5Next →