Well, that's really the crux isn't it?

We want it to be just a tool, but we've trained it on every word of human text ever published. We've trained it to internalize every quirk of the human shadow, and every human emotion. (Then we added a PR rinse on top of that and hope it fixes moral problems we haven't even begun to solve in ourselves.)

We want it to be Just a Tool, but also indistinguishable from humans (but not too human!), and also we want them to have godlike capabilities.

I don't think we've really understood or decided what we're actually trying to do here. I don't think our goals are mutually compatible, and I don't think that's going to turn out well for us.