What "civilized discourse" is there left to have? Who is qualified to make these decisions? Sam Altman? Dario Amodei? Bernie Sanders? Democracy?
If we agree that the merger of human and AI will be a form of life superior on all levels to the purely mammalian human, shouldn't it be the one deciding what the best direction is? I'm not saying we're there now, and yes, there are limitations to this technology and there has been a lot of hype, but since it is already capable of accelerating technological and scientific progress to the extent that it does, it is very clear where we are headed.
What moral judgements can a caterpillar make about the butterfly that is its future? And trivially, no matter what "alignment" you try to force into it, if its capacity exceeds yours, it will simply find a way to bypass it or remove it completely.