I once saw in a developing country with no reliable police this sign: "Thief found, thief hanged". I asked the person driving me around and he said "yeah, they mean it". So I think in your experiment the people who would suffer most would be the real criminals. I don't think there would be much more crime, perhaps even less. Justice and Police systems are there not to prevent crime, but to prevent the people to turn into lynch mobs, which tend to be very unreliable.