" (which is the core conservative stance.)"
Except that it isn't. People conflate all sorts of ideas about particular issues with the larger overarching political ideology. It's entirely possible to be conservative, be pro-gay marriage, pro abortion, pro legalization of marijuana, etc. There are some specific issues which the conservative ideology does fall squarely on, such as a resounding no to universal healthcare, but these are few and far between.
The conservative ideology in the most simplistic terms is "a smaller central government with less power is a better government". Modern conservative and liberal ideologies are nothing more than an extension of federalism and anti-federalism from the founding of the nation. Conflating this ideology with some of the worst people who claim it is unfair and divisive without cause or care.
This, in fact, is the exact thing the person you're responding to is pointing out. You are claiming conservative = "a belief in punitive actions against marginalized members of society" (paraphrasing you slightly for clarity). This simply isn't true, by any measure. It's a nice rhetorical soundbite to raise the ire of your political fellows as a clarion call, but it's not factually accurate and espousing this viewpoint about conservatives normalizes and encourages the type of political divisiveness we are seeing today.
Fun story, if we go by the last 5 years (honestly if we go by the last 100 years, but that's a deeper discussion about ideology shifts related to specific key issues) the majority of political violence occurring in the US is being done in the name of liberal ideologies, with much irony I presume. This very position you are putting forth is actually rather anti-liberal, but it's not conservative either. As a shock to all, there are more than two sides to a cube.