You could simply try it. Find an API that is not publicly documented and thus not part of any training set. Take any AI model, give it the documentation and tell it to create some code that uses that API.
They linked some internal llm with all the confluence documentation sets
My product owner tried to find information about our product, as a regular user would, so feel the experience from a user's perspective
It was a failure: only random, generic answers were received
Then, he retried, specifically citing our main confluence pages: it was a success
Conclusion: for the users that already know were is the documentation, the tool is helpful. Reading the page would be also helpful.
Conclusion (reworded): if you know where is the treasure, then the magic gold-detector will work;