To make the same point differently, if you wanted to replicate human intelligence in its entirety (rather than creating something capable of performing some well-defined task), AI is far too high a level of abstraction. You might be able to replicate a universe and wait for intelligence to evolve, but you still wouldn't understand the intelligence you had created.
The danger of the hardware/software analogy is that we deliberately separated computer systems into (mostly) well-defined and cleanly separated layers. Given how hard it is to avoid leaky abstractions in computer systems that are: (a) vastly less complex than human intelligence and (b) designed to be layered, the systems responsible for human intelligence probably don't expose the clean API required for us to understand them using high-level concepts.