It wouldn't be long before every car used in this way became like a small limousine. The efficiency of driverless cars would easily outweigh the cost of such a redesign.

Most of the reason for that (at least in my experience) isn't about some deep desire for privacy, but for the prosaic reason that it's a nuisance to arrange with someone else to meet at a certain time and place, to make sure that everyone in the car pool gets out of the house in the right 10 minute window, etc

Being able to just call a car to drive you to work, while an automatic system figures out the car sharing arrangements in real time, could remove a lot of the barriers.

Current cars are personalized which makes sense given who uses it and how much they cost. Sharing is annoying for logistical reasons and because you don't want people messing with your stuff.

Driverless cars have many possible solutions. A simple one is pricing - it costs $5 more if you refuse to share. (And another $5 more if you want massage seats.) It is possible to completely partition the seats from each other so each one is effectively its own cocoon where you can play your own music as loud as you want and are completely isolated from the other passengers.

Heck it is even possible like many current electrics to have a base chassis that includes the motors and batteries in the floor, and have pods that mount on top. You could have your own personalized pod you keep at home.