Re: the insufficient data sets problem, I've been surprised how hard it is to find large quantities of historical financial data.
Decent sources of the data seem very expensive, and I'm surprised there isn't a startup out there that is tackling this well since I'd be willing to pay a decent amount of money for minute by minute or hour by hour tick data--but am finding most subscription prices crazy. Is this data really as hard to come by as it seems?