They exist on fewer servers, so they have to be pulled from deeper levels of cache, or maybe even no cache. Popular videos will be cached on edge servers all over the place, and will be quick and easy to deliver.
That's a good point that I didn't consider. Still, I don't think this was the right direction to go. If the videos were already rarely watched, going out of your way to make them less watched isn't going to save much bandwidth.