To be done sufficiently, you need to not only prove that your system is fair, reliable, scalable, tamperproof, anonymous etc., but also that you system actually runs. That combination is nigh impossible. That is the fundamental reason, bootstrapping that trust. Imagine you have a 100% open source system, cryptographically verified etc. How do I know that is actually what is running in the voting box?