I'd still like driverless cars, but privacy and freedom of movement rights protections need to be thought about.
The problems you mention are real, but a driverless car doesn't change much. Havoc due to hacking is already possible today, with OTA software updates for cars. Tracking is also possible with cell phones. Driverless car software could be like GPS updates today: Some corporation provides a yearly update and that's it.
But it's still legal to buy and drive a car from the 80s that can't be hacked.
Looking ahead, my concern is that we'll begin to see laws that restrict human controlled vehicles from autonomous lanes and eventually whole roads. This would take a lot of burden off the manufacturers. If cars can use a common protocol to communicate across brands, you don't even need traffic lights or turn signals; you don't even need to stop at intersections. Cars can just slow down or speed up by a tiny bit in coordination with other traffic. At that point it will be impossible for a human to drive at all.
It's impossible to separate increasing vehicle autonomy from increased government control. Once there's an ability to make any car pull over to the side remotely, knowing who is in it and where it's going, all freedom of movement and therefore all human autonomy exists completely at the whim of government. That level of control will be abused sooner rather than later, if not in America then certainly in authoritarian states.
Full self driving is an authoritarian trap.
To me this is an extremely dystopian outcome, but it's inexorably where we're going once we have full self-driving. And the Tesla fans, not to mention people who are like "oh humans shouldn't even be allowed to drive! Too dangerous!" will be living in this and looking back wistfully at the days when you could make eye contact with a driver and know whether it was safe to cross the street.
My guess is we'll establish a maximum speed allowed to drive for humans, say 30km/h up to 50km/h. Speeds above this are reserved for robotic drivers, outside inhabited zones. Some of these roads are vehicle only, like we have E- routes in the EU or I-routes in the US. Here, non human traffic is forbidden. Then there are the secondary routes, where high speed should be robot-only and robots have to take care for other traffic. Roads can have markings adapted for easy optical recognition, but driverless cars are required to drive safely without them.
Eye contact with a driver does not at all make it safe to cross a street, as current death toll proves.
You're right, the bicyclists will likely ruin the possibility of total automation for the self-driving fans. I hadn't thought about that. Two clashing visions of a testicle-less utopia. Both entirely devoted to their vision of different worlds in which no one dares take a big machine themselves and drive it down a road without permission.
If it comes to the point where you cannot drive your own car on the highway without computers in it, then you have no freedom of mobility. Your most basic right as a human is gone and your children will grow up in a world where they don't know they have a natural right to go places without asking permission. The only people who will go wherever they please without asking permission will be the people who sold you that technology and who captured the regulatory process to make it required by law.