As existing models become out of date, somebody has to pay the massive cost for training updated models to replace them, including, of course, the free models.
I need to fill in timesheets, which uses a proprietary API. Claude guessed and iterated until it found out how to use that API, and filling in those sheets is now much less bothersome. I only need to enter a password every three days instead of four times per day.
Top models can search the web and read the API docs.
Opus 5.5, for example, can write zig code very well, even though there is very little zig code repos out there.
Training models (of same size) gets more efficient over time from combination of software, hardware, and infrastructure improvements, so retraining that open model to update it, will be cheaper and cheaper.
Related, for the CHM's IBM 1620 project, we received 4 entire cabinets filled with punched cards from Indiana University -- a prof there took every useful IBM 1620 program, and immediately made a copy and put those cards into what ended up being 4 card cabinets -- those deep things, with decks on both sides of each drawer. About 10' x 5' tall x 6' deep. As part of the project, we had a card reader that was capable of reading 'column binary' -- that is, it could read the bits on the cards, not just the encodings. I wrote a program to read ALL of those card decks in column binary and then took those around 200 files and put them into a single .ZIP file.
That .ZIP file holding ALL of the known commercial / scientific computing SW for the IBM 1620 fit onto exactly .... ONE standard 1.44 MB Floppy Diskette.
You don't rely on the model knowing about such things up front. You feed it the latest API documentation and then work off that.
You could simply try it. Find an API that is not publicly documented and thus not part of any training set. Take any AI model, give it the documentation and tell it to create some code that uses that API.
They linked some internal llm with all the confluence documentation sets
My product owner tried to find information about our product, as a regular user would, so feel the experience from a user's perspective
It was a failure: only random, generic answers were received
Then, he retried, specifically citing our main confluence pages: it was a success
Conclusion: for the users that already know were is the documentation, the tool is helpful. Reading the page would be also helpful.
Conclusion (reworded): if you know where is the treasure, then the magic gold-detector will work;
Docs, OpenAPI schemas, web fetch, and scripts to understand the request/response are used by models today. It works for any API over time.
RAG or other tool calling, like you suggest, the points in OP are addressed.