We can tell when our simulations behaviour doesn't match the goal. But when it does, then what?
You seem to imply there is some (actual, literal) magic we are missing.
When we build bad simulations, then we can distinguishable them fine. Then we understand more, and do better, and then they become indistinguishable.
The simulations aren't going to get any better than what 'we' understand.
That's not entirely true. We simulate vision systems using RNNs, and we don't entirely understand how they learn.
( https://www.youtube.com/watch?v=r2OBznK6HkI )
Yes, it evolves. Well, in theory : running time is nowhere near what would be necessary at the moment.
Note that non-evolving versions would be extremely valuable as well. Evolution happens, but individuals don't. So if you want, say, medical advances, evolution is probably one of the first things you'd throw out.
But if you are generally looking for artificial intelligence that evolves, there's tons and tons of examples. The problem here is that people aren't usually all that comfortable with real running evolution during "production".
People want standard, reliable solutions that perform every time. The way this is done is to "freeze" the algorithm's evolution before actually using it (or even, as Geoffrey Hinton pointed out, first train an algorithm on the actual problem, then train a second algorithm to simply exactly imitate the first's behavior, which has much better results. Or even train a second one using a non-evolving algorithm like decision trees, which I'm told is often done in finance).