Sure, occasionally they hypothesize a really incorrect answer, but they also bring out a lot of subtleties that even field experts sometimes don't know.
It is a huge help while learning.
Now I use ChatGPT for scripting help especially LaTeX. It took me 3 weeks to produce a "Hello World" PDF 5 years ago and it took ChatGPT to provide me the completed tex code within 15 minutes few weeks ago. Ever since, I been exploring a lot of LaTeX syntax and see how much I can do with it with ChatGPT help. Now, I am learning about using `hyperref` package for interactive fields in PDF. I have produced few Word document for forms with tables in the past (tables in Word is a complicated b*tch) and used external PDF editor to add the interactive field for form filling. Now I am working on converting those Word documents to LaTeX.
Also, ChatGPT is a big help with AutoHotKey script & UserScript for TamperMonkey. I told ChatGPT of my intention and what I am trying to do. It produced the script exactly what I expected to work. ChatGPT is a amazing tools to use for a lot of thing.
I've tried that extensively, with no luck.
I have some experience with AHK1 and wanted ChatGPT to basically convert scripts to AHK2.
It's pretty much a loop: first response gives a syntax error. When I reply with the error message, it apologizes, explains where the error is, and gives another solution. That solution has different errors. When pasting the error message, it apologizes again, and gives a third version with syntax errors. And then it starts with the first version again, and I can repeat the loop.
If it start to get off the rail, I copy the code and close the chat. Then I create a new chat and tell it to use this code as a strict reference. And I tell it what I am trying to do, then it starts to improve the code further.
I found that it is best to give it my snippet of code and it will be able to use it as a template and modify it from there.
For your case, if you want to convert it from ahk1 to ahk2, give it a small section of the script. And it should be able to convert it from there. If that didn't work, then you could start from fresh and tell it of what you want in ahk2 script.
I am surprised how bad it is with ahk scripts and wonder why. What is indeed "original" is that it gives code with unexisting functions, which it never does for the other languages (in my experience).
Perhaps the language is not that famous compared to js or C but i doubt. There are tons of forums discussing issues... Strange.