The Japanese criminal justice system is brutal by world standards. This usually goes unmentioned when people bring up Japanese crime rates.
It's not just violent crime (something which might be affected by harsh penalties for such) which is rare in Japan, but petty crime as well. As far as I can figure, it's because of strong societal bonds and low income disparities, and that there's less of the sort of sense of disenfranchisement/alienation which seems so common in e.g. the U.S.
Ha! There are loads of accounts of the falsely accused getting locked up with zero due process. You can find yourself in an unheated cell and forced to kneel on the floor face against the wall for hours a day under threat of beatings before things are sorted out weeks later. Read about the death penalty in Japan for a taste of how they operate. You get killed in secret and your execution is announced months after the fact. None of this years of appeals stuff.
So while the Japanese justice system may have some bad habits, that is not an explanation for very low Japanese crime rates.