So you are claiming that having to look away from the road for possibly whole seconds to deal with a phone is not really dangerous, but what is dangerous is your mind wandering? I'm sorry but experience and common sense says otherwise.
I would also argue that your second point is not as not clear cut either. Having some sort of distraction is definitely helpful most of the time when driving long distances. I personally find audio books very greatly help to fight off monotony, which in turn keeps my attention higher than if I was not being distracted.