Nassim Taleb makes exactly that point in Fooled by Randomness, which is a terrific read incidentally.