Your argument just tells me you need a ped bridge, not slower speed limits. Even as it is, bikes are traffic, and drivers can wait. The differential is not the root issue in your case.
Then they built a ped bridge, and threatened to kick off cyclists because cyclists are vehicles that travel 10-20 mph faster than pedestrians (and don't integrate well with pedestrian traffic generally).