My concern has always been the one-day-in-a-thousand when the CDN will be down and so my sites won't work. When my sites go down it doesn't matter as the entire package stops working, but I'd hate myself ten times more if things stopped working due to an admittedly extremely rare CDN failure.
That said, I'm open to the idea that I'm being extremely foolish about this ;)