Hacker News Reader

Comment by viraptor | Hacker News Reader

This works better for some languages then others. I tried something similar for Swift since I wanted a simple tool for MacOS. The process was not great, many methods were hallucinated, and the answers matched old/deprecated functions from previous Swift versions. The experience seems to scale comparably to what's available in stackoverflow.

View on HN

A friend of mine was trying to show me how surprisingly good ChatGPT is, and sent me an example prompt and response that produced working code. Just to check, I entered the prompt into Google and got back the StackOverflow page with verbatim the same complete block of code. It was even the first Google hit. The only thing ChatGPT had done was copy something easily findable on the web and scrub away the attribution, license, and source information.

"It's faster/less of a pain to ask the bot than to use StackOverflow" is a significant portion of what I use ChatGPT for. Sometimes, it's just a better search engine.

I recognize that there are questions about how that works in the longer-term, but for now, it's useful.

Maybe in some cases, but I’ve found the context of the original answer to be important.

Usually the original answer has some responses attached either clarifying something important or outlining why this approach is bad and linking to a better answer.

With GPT you only get one answer Saïd with complete confidence.

Even if all chatgpt did was save you two clicks per search, and maybe saved the time spent highlighting the portion that needs to be copied, isn't that still valuable?

Removing the attribution is a major negative, as it removes the incentive to contribute to stackoverflow. People are willing to do a lot of work for free, provided they can get credit for it.

In this particular case it didn’t save any clicks or do any highlighting that StackOverflow hadn’t already done. But even if it did save a little effort or time, which I could generally agree might have some value, I suspect it’s myopic to focus solely on the search effort. We lose something more important when AI inserts itself as the search engine; it prevents the searchers and the authors from having a conversation. Authors don’t get feedback, searchers can’t ask questions or follow up or seek out unrelated information from particularly knowledgeable or inspiring authors. I do want search to help me get to relevant and quality info, to sift out the things I wasn’t asking about, but I don’t want to trade my ability to interact with people for that or especially for slightly fewer mouse events. I’m also concerned about what happens when this starts to commodify after convincing people AI search is easier/faster… when ads and financial interests are allowed to guide the results — and that will happen — will it end up eventually being worse or harder to get true or relevant search results compared to StackOverflow or Google? Where do you go when the results don’t tell you where the information came from?

Yeah, the more you move away from standard tasks in top languages, the more errors you find. As a test, I asked ChatGPT to generate R code to run a T test for every group of my dataframe. It took the average for each group and then ran a T test on the resulting single value. It still spit out results, and if I didn't do stats for a living, I might not have noticed the error.

You're right that it does hallucinate a lot when it comes to MacOS API. I find that the hit rate is good enough to ask ChatGPT first, however, before diving deep into Apple documentation.

It's saved me quite a bit of time in terms of understanding how to use APIs as a client. I find the API docs that Apple provides are so bare bones and don't illustrate how the APIs are meant to be used, and you really have to search around for example projects or That One WWDC Video Where All Is Revealed. Asking ChatGPT first saves a bunch of time having to do all that searching.

Reply on news.ycombinator.com