In general, people have a poor understanding of how to evaluate an investment manager. It's not enough to just look at absolute returns and compare them to the S&P, you need to correct for market exposure (the beta). Even then, it is not that straightforward: this is one of the best overviews I've seen (the author of the blog, Robert Frey, was a former managing director at Renaissance Technologies, the most successful hedge fund of all time)
http://keplerianfinance.com/2013/07/alpha-and-evaluating-inv...
To make the "correcting for exposure" aspect concrete, suppose you have the opportunity to invest in a poker player that generates a 10% return on capital per year. It wouldn't really make sense to compare this return to the S&P 500 returns, because the beta is very close to 0.