1. I asked ChatGPT to explain to me what NEM12 and NEM13 files are in the Australian energy market, what's the difference, how they're used and how they relate to energy meter types. It confidently gave me an answer that sounded very reasonable but upon further research was wrong in subtle but fundamental ways that made the whole exercise pointless.
2. I had an idea that you could hook up ChatGPT to the APIs exposed by most racing sims and have it be a dynamic "race engineer". I fed it with detailed information about the state of the race and asked it "in the style of a Formula 1 race engineer, craft a radio message for your driver given the following information. Only include the most important pieces of information. If you have strategic recommendations you may include them." All it did was rephrase the exact situation I had provided it, more verbosely.
This mirrors all of my experiences using LLMs to date (other than coding, which it performs slightly better at). I think these things are impressive in that they are doing things we've never seen computers do, but I have yet to find them do anything useful in a way that isn't fundamentally flawed.
I sometimes feel like I'm the only "AI" skeptic in the world, shouting into the void. This stuff just doesn't seem useful to me, and from what I have read about the technology there's no reason to believe this will dramatically change.