To me it's not tangential, but rather the core of the issue.
Let's take the druggie. I agree with you that 'removing consequences' of crime committed to acquire more drugs is a terrible idea. Justice to the people affected is important.
However, if this consequence is jail time, a criminal record, or any other form of punishment that doesn't benefit both the addict and those affected by him, we're only sweeping the problem under the rug, and we're not acknowledging the complexities of addiction and the dignity of the human behind the 'druggie'. Furthermore, it's likely that said addict will become a worse problem for society down the road. It's lose-lose.
There are plenty of solutions where the addict's crime have consequences other than simply punishing them that are more effective, less dehumanizing, and ultimately better for society as a whole.
On a more... ethical level, I feel that where we fundamentally go wrong in these situations is that we forget that being poor, being an addict, and even to a degree being a criminal is not just a simple matter of choice. I've personally known many addicts, criminals and poor people, and I can say with full conviction that the vast majority of them were good people who made mistakes that were not 100% their responsibility. Instead of bickering over how much responsible they are, accepting this reality and trying to approach it with compassion for all sides should already improve many things. Hell, if more people would just go a bit below 100% it'd be a good start.
I think I'm a pretty good person, but putting myself in the position of my addict, poor, or criminal friends, with the same environment, parents, friends, and choices available, all of them mostly out of their control, well, I'd likely make those same choices.
And applying everything I've learned about willpower, group dynamics (groupthink), peer pressure, and whatnot, makes me even more convinced that punishment is not only a solution that doesn't work, but morally wrong and detrimental to society as a whole.