Having a second adversarial network could absolutely be part of the solution, I wasn't arguing against that. My problem is with in context learning, ie the idea that including background information in the prompt can solve hallucination.
Like the time I decided to sit on the ceiling to spite a friend who was being pushy.
Made perfect sense, and I was about to. Until a different part kicked in and assured the whole of myself that it wasn’t possible.
Neural networks are going to spawn occasional nonsense. Aiming for perfection is noble, but nature shows a different path that already works.
But fundamentally I think we’re agreeing- just the twist is I’m saying LLMs should be an aspect of a whole rather than seen as a solitary solution to be perfected.