In general, we'd have to close the loop over hardware the way we did with software.
In software, you can write open-source code that depends only on other open-source code, and you write and distribute it with open-source tools. Trusting-trust-style attacks aside[0], everything on the software layer can be inspectable by anyone, and in principle you could read through all of the code and understand what's doing[1].
With hardware though, there pretty much isn't any usable open-source hardware stack that's a) fully open-source at every level, and b) verifiable. The latter is particularly damning, because chips are made in commercial factories you can't inspect, and the actual process is secret. (1/2)
Until the day you can literally print[2] your own circuits from sand, using an open-source designed and manufactured electronics printer, you can't trust hardware without trusting people who manufactured it.
Ultimately, I doubt we'll ever be able to have 100% trust in any human-created system. The one thing we can do, though, is to structure our systems to maximize cost of a successful attack, trying to push it so high that the results are not worth the effort.
--
[0] - http://vxer.org/lib/pdf/Reflections%20on%20Trusting%20Trust....
[1] - it's obviously not true in practice, due to complexity, bugs, and finite lifespan of humans.
[2] - or "nanolathe", as we called it in the late 90s; cheers Total Annihilation fans!