110 Million people watch the superbowl, a 30 second commercial costs about $3 million (time slot fee only). Are there more than 30 commercials during the superbowl? I couldn't find that number, but 30 seems like a lot, so let's go with that. The revenue per viewer is about $1.20. So even if the cost to watch the superbowl was $4, the broadcaster would still be making more than they currently are with the advertising model.
Where this breaks down is the amount of time people spend channel surfing. I think as people channel surf less, they'll watch less, which is why the model of paying directly for each piece of content is a challenge.
BTW 90% of my US sports are not done on ESPN. Of the 4 major sports they only do a few select games.