Keep in mind that the word "dumb" gets thrown around in all sorts of situations and has no objective, easily-verifiable test for "dumbness". It's entirely plausible that merely "making a decision I disagree with" could be construed as "dumb". Imagine if, say, Biden and Fauci had decided to disenfranchise anyone who refused the COVID vaccine as "dumb", or, worse, Trump and RFK decided that taking those vaccines made you "dumb". See how easily this can be abused?
Even with that aside, the natural consequence of disenfranchising "dumb" people is that you want to fuck them over somehow. No thanks.
No matter what kind of voter you particularly would like to target, the people administering the poll tests will (ab)use them to disenfranchise whomever they please.
Even if you believe the people in power who administer your elections would do this the "right" way, if you give them power, then the next people who come along -- who you may not agree with -- will also have that power.
A literacy/civics/logic/knowledge/etc. requirement would at best discriminate against people who are uninformed, unintelligent, uneducated, unable or unwilling to take a written test, etc. But they are people whom the laws affect and who should be able to vote. I don't think that they are a comparable category to the exceptions mentioned above, but that's debatable.
Most importantly, the test would be optimized to exclude whoever the test designers and proctors consider, knowingly or not, unworthy.
I understand the desire to exclude the dummies. I know thoughtful and educated people who espouse the same wish, but it's just too dangerous.