Secret ballots were devised in response to people historically doing things like this. Then they stopped happening because the countermeasures worked. If you take away the countermeasures, what then? Have criminals ceased to exist? Are the incentives somehow different now?