Hacker News Reader

Comment by LPisGood | Hacker News Reader

I recently tried this _exact_ same thing (except it was less of a meeting and more like a phone call) and I got stuck because it kept making up fake API endpoints OR I had to go manually login somewhere to get the API key.

View on HN

When? I've found over the last 6 months or so there has been a drastic change in quality of results. I used to have a ton of hallucinations like that. It still happens, but it is significantly less common now.

I also get better results when using a prompt window built into and ide than something detached like the openai website.

AFAICT, OpenAI o3-mini-high, a couple days ago completely hallucinated functionality within systemd homectl that would mount a home directory from another machine using sshfs. It certainly wasn't available on Ubuntu 24.04, but searching the Internet I can't find those options documented anywhere.

Though, I do agree that hallucinations have dropped dramatically.

I am not an expert in systemd homect but I have found that adding, "Show documentation for any methods or built in functions used" helps when I'm working on anything obscure or that is less likely to have a lot of publically availability code (that the LLM was likely to be trained on).

That helps a lot whenever I am working off something I know will likely exist but just needs to be pulled from a company's documentation.

That did provide some more interesting results, seemingly getting closer to something that would actually work, but not quite. Perplexity, Claude, and ChatGPT all provided very different approaches. That addition to the prompt definitely did provide some useful additional information.

Hm not my experience at all. No idea what the difference was. I gave it my API endpoint and told it the response shape. As far as the key was concerned, I told Claude to look at a given envar I defined and place that in a header. Claude did all the rest. Are you using 3.7? I don't think I used the Extended Thinking mode in the UI for it.

Which model?

Reply on news.ycombinator.com