While I agree with your sentiment, the actual quote is subtly different, which changes the meaning:

"Think of how stupid the average person is, and realize half of them are stupider than that."