This makes it sound much more dramatic than it was (I watched the video). The car mistook a green arrow for a green light, and started to move forward, and Musk stopped it. It was pretty low key. Musk had his hands off the wheel during the drive because he was holding the phone streaming the video, but he was clearly alert and aware of what the car was doing — that's what the conversation with Elluswamy was mostly about.
A nag-free test mode makes a lot of sense for the engineers testing the cars. I don't see anything wrong with that.