Let's say speeding causes a harm of 500 dollars, and there's a 10% chance of getting caught. Then you would put the fine at 5000 dollars. However, it might be the case that a 500 dollar fine (with an expected payment of 50 dollars) is already enough to stop everyone from doing it, since nobody likes speeding so much that they would pay 50 dollars for it on average. In that case, setting a 5000 dollar fine would just be overkill, and it could hardly be called efficient.
Moreover, your answer assumes that utility can always be measured solely in terms of money. If we measure the utility of society in terms of the number of people speeding (lower is better), then no amount of money can compensate for an increase in the number of people speeding. Then the answer would be to set the fine such that the number of people who choose to speed is the lowest, and does not involve the "expected harm" to society at all.
Even if you do assume that there is a conversion factor between "number of speeders" and "money", then it might still be sufficient to set the fine at a much lower number than you would get using your formula. You cannot just take into account the utility of the society, you also have to take into account the preferences of individuals (which guide their actions).
This can also be used to justify individual fines: if a 100 dollar fine is enough to stop you from speeding, and a 200 dollar fine is necessary to stop someone else from doing it, then why should I set the fine for both at 200 dollars? How can this be called efficient? You could argue it's more fair, but that's more a question of what's moral and has little to do with simple calculations, which you seem to be advocating for.