Voting wallets? Assign crypto keys? You mean just like issuing a ballot? How would someone do a recount? An end-to-end audit? Keep track of who was assigned which key (thereby destroying the secret ballot)? Who voted and who didn't? How do you handle provisional ballots? Etc.
PM me once your proposal satisfies the requirements (works as a replacement) for current election administration systems.
Okay, maybe I'm being too dismissive, which I normally hate. But blockchains, crypto, signatures are not magical. Design the whole system, make it auditable while accounting for information leakage, simulate a few real world elections. Do the leg work, unlike Chaum, Rivet, others. THEN we can talk.