If your proposed system ended up leading to large amounts of suffering, worse than the current system, would that convince you it was less moral?
This doesn't mean it's a "less moral" system. As a counter-example: Would putting surveillance cameras inside every house to prevent domestic violence be a "more moral" system just because it reduces more suffering? Where do we draw the line for which rights we're okay with violating in order to stop suffering? I just put it at a very non-arbitrary spot, which is the NAP.
Consequentialist libertarians seek to maximize individual Liberty. Deontological libertarians seek to minimize immoral actions.