Computer program can detect depression in blogs and online texts
medgadget.com
medgadget.com
So, it's never been tested? Say what?
No one actually looked at the author and determined if the author was depressed.
In otherwords, the program agrees with psychologists 78% of the time if a sample of text may indicate the author is depressed. We're not close at all to a magical diagnosis machine...
What hasn't been established is the correlation between the program and the mental state of the authors as measured using more conventional methods.
However, I don't think substituting one human error (that of the programmers) by another human error (that of the psychologists) proves anything. You could say that the second group is more qualified as domain experts but it still doesn't make up for complete lack of real life testing. If it did, you wouldn't have to conduct human trials for drugs - you'd just get a bunch of doctors to say that they think the drug works.
The fact that the program purports to detect depression, something that amounts to artificial intelligence, makes me especially skeptical. Not to mention the software author's example of using combinations of the word "black" with terms describing "sleep deprivation". By this definition, anyone who blogs about, say, New York Fashion Week, is depressed.
It's a non-trivial problem to get a representative sample of bloggers to submit to a valid screening for depression. On the other hand, collecting a data set of writings by depressed vs. non-depressed people is easier, such data being (I suspect) readily available.
If the panel of experts can correlate between depressed writing and depressed people, then there is validity to their correlations of the writing of unknown people, without having to screen the unknowns separately for depression.
I suspect even a panel of non-experts could screen writing into depressed and non-depressed. Humans are pretty good at detecting malaise in others.
Oh. Actually in take that back. We have no idea what the FN rate is. They haven't tested it at all.
There are a number NLP programs that can detect a writers gender from text at a much greater accuracy than raw chance, especially in a limited domain. The problem is that the world is a big, messy domain.
There's no reason in principal that other traits wouldn't be 'detectable' (although I doubt sexuality), but the problem is getting a good set of training/test data in context, and what your acceptable threshold for accuracy is.
I'd be willing to bet that people whose text messages contain lots of U's and L8R's probably have worse credit than those who text in proper English, but there are much better ways of doing a credit check. If that's only accurate to, say, 100 FICA score points it's probably not worth using it.
I wouldn't be too worried about big brother(s) using semantic analysis to read your mind, but don't discredit the fact that language patterns encode certain cultural information that is identifiable, or at least correlated with behavior.
Oh really?
Briton Convicted for ‘Menacing’ Tweet Against Robin Hood Airport http://thelede.blogs.nytimes.com/2010/05/10/briton-convicted...
Do you mean this?
http://www.boston.com/bostonglobe/ideas/articles/2009/09/20/...
http://www.boston.com/bostonglobe/ideas/brainiac/2009/10/pro...
(I learned about this on Hacker News back when it was first reported.)
The worst thing that can happen is the Context Trap: http://en.wikipedia.org/wiki/On_being_sane_in_insane_places
1) Humans want to look for X 2) Design software to look for X 3) Software finds X 4) Humans believe software but software is incorrect 5) X is screwed
This kind of analysis will never be perfect, but then, real humans aren't perfect at it either. I think we can predict that software will get better at this kind of 'guessing' over time. It's not half bad right now.