But I often get answers faster this way, compared to doing a plain search, so they've been useful for me.
Useful, but not useful enough to do my job for me, as most of the hard work is fitting into an existing large codebase with its own style, architecture, private libraries etc., or where I'm making a UI fit a design doc that the AI can't really see despite having some textual representation of the design as well as the UI code.
Fortunately for me, 128k token context windows aren't enough to fit a 120 kloc project, and the haystack demos don't fully reflect the needs.
But 5 years ago, the best we had was GPT-2 / AI Dungeon, so I'm just going to keep saving and plan around not being able to get a next job — even if I'm being fooled by a Clever Hans, I'd still get early retirement.
It looks to work fine on a one-off single script thing. Which is marginally useful sometimes. I did create my company single page html site with it because I didn’t really care about it and just needed it for a bank to believe I’m selling services.