And one more reason: deterrence. Supposedly, would-be criminals are deterred by seeing others get jail time. Depending on the crime, that purpose is rarely accomplished as well.
Unfortunately, it's much easier to crank up the penalties and posture as "tough on crime" than it is to make sure police are actually closing cases.