So, if I put a pro rider on a bicycle you're cool with me having them scream around crowded areas?

Traffic regulations are fundamentally about tail events. Most of the time, vehicular traffic is boring.

The right metric is kinetic energy. But, it'd be unpopular for speed limits to depend upon vehicle mass. Or, in the case of bicycles vehicle and rider mass since the rider mass isn't negligible.