Honestly, in Manhattan they should eliminate automobile access to a great deal of the streets and avenues.
There still needs to be some sort of delivery truck access, irregardless. Parts of downtown Chicago provide an interesting example with multi-leveled streets, the big deliveries and trash collection can be all be done out of sight.
I suspect very soon we will see a city, or multiple cities, start blocking off certain streets and then neighborhoods to non-autonomous vehicles. Maybe it will be Manhattan, maybe it will be another city. This lowers the bar for how advanced the self driving tech needs to be by a lot. You can have 100% traffic enforcement. Pedestrian deaths should almost be completely eliminated short of some very unusual edge cases.
https://en.wikipedia.org/wiki/Pneumatic_tube_mail_in_New_Yor...