Since we've built up large storage of nuclear weapons, the clock is ticking. Even if the chance of one crazy guy coming to power in one of the governments that can nuke the planet is 0.01% per year, after a few thousand years it will approach dangerous levels...
And I'm not just talking about nukes, but about other potentially mankind ending threats like superviruses, hauling large asteroids onto earth, etc. The more technology we discover, the more methods to end each other's existence. Also, as technological progress continues, so does the spread of these technologies over the world. Smaller and smaller groups get access to them and with the larger number of groups, the likelihood that one of them wants to end all life on earth increases.
Preventing us from inventing these things would require halting and reversing progress in most scientific domains, a pretty sad idea. You can't have nuclear reactors without the danger that someone uses the knowledge of the people who built it to build nukes (I know the two are not the same but so often you hear countries claiming their nuclear program is civilian while in reality it isn't). You can't build rockets without the danger that they might be converted into ICBMs. You can't do advanced biology without understanding infection well enough to know which changes might make an infectious agent more easy to spread, more deadly, etc. You can't do asteroid mining without giving the mining company the power to misdirect the asteroid from an assigned target area to inhabited areas.
So one should rather manage the existence of these technologies instead. No organization ran by humans can reduce the chance of one of these fermi filters from going off well enough to stop them being a threat over extremely long time scales. Only computers can achieve that. I imagine building a computer that is tasked to prevent humans from intentionally destroying large subpopulations of their own. One would have to give it absolute power so that it can ensure this, but one should instruct it to otherwise let humans figure out their governance on their own.
Structures where humans are involved are simply too fragile over the long term.