It absolutely makes sense to put a human in a car you're testing and don't have full confidence in yet but are sending into real traffic. Not doing so would be ludicrously negligent. However, humans simply don't maintain focus for a long period of time without some sort of extreme, years-long training. The ability of monks to concentrate on a single thought for hours is correctly seen as very, very difficult. There's no way any regular person could deal well with a scenario like "sometime in the next 8 hours, you may or may not need to smash this brake pedal with a second's notice, but other than that you have no task." That's one of the things that most worries me about "almost there" systems like Tesla's.
That said, if you've decided to attempt this anyway, checking your phone is definitely not okay.