I made Font Awesome, and I concur! If you really care about load times and performance, definitely subset your icon fonts!