So? That's an ad populum fallacy. Remember when plenty of places thought smoking cigarettes everywhere was normal?
In addition, laws often contain compromises for feasibility. I expect many jurisdictions permit hands-free conversations partly because it's difficult to prohibit them.
If you want to argue that phone-conversations while driving are safe compared to quiet driving, you should be pointing to experiments rather than popular belief.