It's not just violent crime (something which might be affected by harsh penalties for such) which is rare in Japan, but petty crime as well. As far as I can figure, it's because of strong societal bonds and low income disparities, and that there's less of the sort of sense of disenfranchisement/alienation which seems so common in e.g. the U.S.