A further point to consider if we're talking about real world applications is that it is not actually established that markets have finite variance - seriously.
In the 1960's, Benoit Mandelbrot began his research into chaos and fractal by looking at markets and finding that non-Gaussian, Levi-Stable distributions modeled changes in the market best[1]. And these L-stable distributions don't generally have a finite variance and sometimes don't have a finite mean [2].
And it is fairly easy to see how a market tends to not be Gaussian; change based on a Gaussian distribution tends to be random walk a la Brownian motion, where the final position of a variable is the sum of many small changes in the variable. Non-Gaussian, infinite-variance-distribution-based movement on the other hand has the property that the final result of a variable tends to be more the result of a finite number of large changes rather than a lot of small changes. And this is what the stock market often looks like. A few wild moves often impact things as much as the incremental changes. The apparent mean, variance and distribution of stocks on a day-to-day basis may not pan out in extreme situations and these can eat away the rest of your profits. If the stocks that seemed independently in normal conditions all go down in crash, your estimated-correlation-based-diversification hasn't protected you very well.
The Black Swan is a sadly too-simplified popular summary of these points.[3] It does point to the general idea. The higher-level take-away is that infinite variance distributions exist and indeed, you can not apriori assume a given distribution you are working with isn't one.
[1] http://books.google.com/books?id=6KGSYANlwHAC&lpg=PP1... [2] http://en.wikipedia.org/wiki/L%C3%A9vy_distribution [3] http://en.wikipedia.org/wiki/The_Black_Swan_(Taleb_book)