Agreed. I've done it myself, but my main problem with it is simply that it's just not fun to use. It's clunky, having a keyboard and mouse on the couch or having to bounce between the couch and my computer desk. Roku and Apple TV on the other hand are, generally, very nice and actually fun to browse with. They've honestly changed the way I think about TV.
Plus the prices (and form factors) are awesome. $69 (Roku) for a plug and play box that can sit on the shelf with my other components.
But I guess the point here is: If I can hook a computer up to a TV, why do networks try and differentiate between 'computer,' 'tv,' and even 'tablet?'