What exactly would be a good alternative? Jail time is also not a good penalty. See: marijuana.
The fines, by contrast, seem to be working pretty well: how many other companies get busted by the FCC for violating satellite launching regulations? The fact that it is so rare is a testament to how effective the deterrent has been in regulating companies in this space.