Passports would need to become like hardware wallets with screens such as Trezor. Who's firmware and design are open source.
The screen means I can see what I'm signing and later be shown what I signed.
Now, I don't want people to know who I vote for. So we make it e-passport gate so that no one can enter with me.
Now I want to be able to verify my vote at a later date but not let others know it was me. Well this too is solved by multiple cryptocurrencies like monero and z-cash.
I can go to my local town hall enter a room alone using a e-passport style gate and check my vote at the machine. My device can verify my vote.
Someone with the vote can only see that 1 of N voters did it. Where N is so large they can't identify me.
No duplicate vote is possible.
Literally no one would agree to this however.