I would say (and I've been saying this all along) is that it's first and foremost a matter of economic impossibility.
Under the assumption that you can make the technology work well enough for a sufficiently large group of potential customers, the business case for streaming games simply doesn't make sense to me. You need to invest huge amounts of money to build, maintain, and update (keep current) highly-specialized server farms that are mostly useless for other purposes, in the hopes that people will pay money to play games on them. All while hardware prices keep going down, and game visuals (including resolution and GPU requirements) keep going up. If you stop upgrading your server farms, within a year or 2 what would previously be top-of-the-line hardware will be so cheap (new in the case of PC's or second-hand in the case of consoles), that unless you are dead set on being able to play the same games no matter your physical location, you can get a much better experience for less money by just buying the hardware.
It has always occurred to me that many people are still convinced cloud gaming is a good idea, but the repeated failures of services like OnLive seem to have proven this wrong for now...