The only phases of flight where you really need humans at the controls are takeoff, landing, and uh-oh what the fuck is going on (this being a broad "catch all" category which encompasses everything from an exploded engine and hydraulic failure, see United Airlines / Sioux City, to a medical emergency).
I would never fly on a pilotless airliner, but I'm fine with the pilots being asleep at the wheel during the en route phase of flight. There really isn't much to do except look at the GPS moving map and say "yep, we're over Iowa...... yep, we're still over Iowa". Course/altitude adjustments could all be beamed in from ATC.
I'd prefer them to be rested up for landing, or in case something unexpected pops up.