People want to do the same things at more or less the same time: Having time off while your social circle is at work is not as fun. Many jobs require customers, and others require entire groups of people to be there together. This means the minimum capacity needed to handle peaks is not going to be very far off from where we are right now.
Think of your typical morning traffic: One side of the highway is in gridlock, while the other is doing well. In the evenings, the directions reverse. A lot of people want to be at the office at 8:00, so you need capacity for that. And what do the cars do now that it's 8:05 and they are downtown? Probably nothing, as there is no demand for that much capacity for most of the day.
Same thing with a sporting event: People will get to the stadium to watch the 49ers, but Santa Clara doesn't have any use for 40K+ cars for 3 hours on sunday morning. The most you can do is have the cars do some otherwise useless miles to park in a cheaper area further away from the stadium.
I'd love to be wrong on this, but this phenomenon would make it surprising if we got a double digit decrease in traffic with shared self driving cars.