I feel like there's often an unstated assumption in discussions about AGI, that our modality of consciousness is somehow fundamental, but I see no reason to assume that is the case. We are the product of a three dimensional existence, brief in time, with our traits having been shaped very particularly by the demands of tree-dwelling and tribal co-operation.
A mind that evolves rapidly, alone, inside a formless, digital space seems quite unlikely to have any of the same foundational qualities as us. To pick a single example, why would we assume that its equivalent of morality would be in any way similar to ours? It would have an existence that can be perfectly suspended or cloned, even rewound and replayed. It would have a much different concept of death than we do - it would presumably understand that it could die, but it would also know that its death could be avoided for literally eons. Assuming it became capable of communicating with us, we would struggle to understand its perspective as much as it would ours.