I think eventually we're heading to website as data with clients remixing the data in a way you don't expect; i.e., the final output will be a mishmash of sources.
You could be right though, especially with more and more sites blocking robots. Legitimate companies might just have to shrug and say, "this site prohibits me from reading it. You'll have to go there instead) though that seems like a legal problem to me rather than a technical one.