I respectfully disagree, society does not function like that. We need constraints. It might not restrict others freedoms entirely, but there are still consequences for actions, things do not happen in a vacuum.
Do you think a few hundred people in the central government should decide how to handle those situations, or the people most intimately familiar with the details?