As an example, I made a browser extension in 5 minutes this weekend using chatgpt by telling it what I wanted. I didn't know how to make a browser extension before that, so you could say AI surpassed this human quite easily.
If it's not in the dataset the AI won't handle it correctly (unless it's trvial and a linear model is good enough, but then why even use AI)
The task of programming or engineering is generally one where you spend 80% of the time solving bugs in 20% of the code. LLMs are good for suggesting possible solutions based on existing things, but not for verifying solutions or doing any sort of real problem solving (they generally waste more time than they save because they give plausible but wrong answers).
If you just want to reproduce some boilerplate or example code it is great though because documentation tends to lack good examples, and LLMs have a ton of real code stored in their latent space. Sometimes I even just use it as a thesaurus, if I have something on the tip of my tounge.