We know how computers work because every part of a computer, every bit of software that runs on it, was at some point designed by a human. There's no single human that understands everything in a computer at once, but if you ask a good developer why some bit of software is not working, or what they would expect it to do given certain inputs, they can generally figure things out.
With AI, we're creating systems to design other systems automatically. We understand the process works, but the systems being designed are complex enough that we don't really understand much about them, except that they work. They are just massive arrays of numbers that feed into equations and then spit out some output. They're not like systems designed by humans, which are usually intended to be understandable by other humans -- they're weird and naturally incomprehensible. The only way to know what ChatGPT will output, at this point, is to actually run it, even if you were to take the randomness part out of it.
Emergent behavior isn't pseudoscience, it's a real thing. Sometimes when a system gets complex enough, it starts to behave in surprising ways. Maybe we can make a brain, using a computer, and this is how you do it.