Sounds nice in theory, but the whole idea behind the right to face your accuser is that people depend on their senses to judge if someone's lying or not by how he/she reacts to the confrontation.
Is it to make you feel better or to improve society?
Personally I'd go for improving society because as fun as lynchings are we probably shouldn't kill minorities we don't like.