To assess the statistical significance of the SOFNN achieving
the above mentioned accuracy of 87.6% in predicting the up and down
movement of the DJIA we calculate the odds of this result occurring
by chance. The binomial distribution in- dicates that the probability
of achieving exactly 87.6% correct guesses over 15 trials (20 days
minus weekends) with a 50% chance of success on each single trial
equals 0.32%. Taken over the entire length of our data set (February
28 to December 20, excluding weekends) we find approximately 10.9
of such 20 day periods. The odds that the mentioned probability
would hold by chance for a random period of 20 days within that
period is then estimated to be 1−(1−0.0032)10.9 = 0.0343 or
3.4%. The SOFNN direction accuracy is thus most likely not the result
of chance nor our selecting a specifically favorable test period.
I'm not sure about the result not being chance. In particular, aren't they privileging the hypothesis by using the probability of 87.6% guesses being correct? They say the result only had a 3.4% chance of happening, yet we could assign a similar probability to a wide variety of other seemingly unlikely outcomes. I'd much rather have seen the predictor tuned on a subset of the data and then tested on the rest of the data - perhaps the authors will release their predictor and we can try it on 2009.Given that twitter and the stock market are both correlated to what is happening in the world (in fact, caused by what is happening in the world,) it is not surprising that they are correlated with each other. Somehow I don't think correlation in the other direction would get the upvotes though - stock market crash predicts sadness on twitter? You don't say.
In short, I don't follow all of their statistics, but I am highly dubious of their model having any predictive power.