Road speeds need to be a tradeoff between safety and convenience. Unfortunately, in our mind safety and convenience are not just quantitative things, they carry moral connotations. Safety is good, convenience is at best neutral.
Until people develop a good framework to express this tradeoff, speed limits will be more or less arbitrary.