Mercedes-Benz has a very nice product, but it's not hands-free. There's a huge difference.
So yeah, it's not some apocalyptic, universally-agreed failure that happens every day, but people have actually been killed[2] because the machine did something the driver wouldn't have done if they were in full control. (Maybe it's "compensated" for by deaths that were prevented by having cruise control -- I suppose we can't know for sure.)
[0] http://www.dailymail.co.uk/news/article-1236020/Horror-ride-...
[1] http://embeddedgurus.com/state-space/2014/02/are-we-shooting...