In my experience driving Teslas on autopilot (a few dozen hours), you need to keep watching the road. The whole time. Especially on on/off ramps and around traffic lights. I almost watched a Tesla I was operating drive into a meridian on a highway off-ramp because the concrete was approximately the same color as the road. I slowed down to see how long it would take to detect the coming collision ... and if I hadn't been watching carefully, it would have been too late.
Yes, drivers must still pay attention in self-driving cars. Probably a future exists where that is no longer necessary, but that's not today.