Hacker News Reader

Comment by zrobotics | Hacker News Reader

Calling it a Landfill seems accurate. I just searched (on DDG) for the tap size for a 5/16-24 bolt. I got garbage like this:https://shuntool.com/article/what-size-drill-os-used-for-a-5...

This isn't even the worst example, since it does at least have the correct info buried amongst tons of Ai generated garbage, but I can't use this for reference, since it tells me 4 different drill sizes. I've had to switch back to a paper copy of the machinist's handbook, since I can't trust the internet to give me accurate information anymore. 10 years ago, I could easily search for the clearance hole for a 10-24 fastener, now I get AI junk that I can't trust.

How have we regressed to the point that I'm better off using a paper book than online charts for things that don't change?

View on HN

I find myself using yandex more and more. They’re like old Google, but obviously based in Russia.

https://www.americanfastener.com/tap-and-drill-size-chart/

That was the first result.

Unfortunately, yandex is destined to fade into irrelevance for the reasons that has nothing to do with the tech.

Can you elaborate?

Half of web in Russia is blocked. Literally, powers that be think of Russian tech companies as of their servants and nothing more. Yandex basically sold their main asset, domain name to other entity.

If there is any chance I’ll use some web content again, I generally copy and paste the bit I want into the notes app on iOS.

You know it’s bad when you trust Apple’s search function over Google.

This, I am a terrible note taker. For years a huge part my knowledge and skills relied on "if I found that information once, I'll find it again". My brain compressed the information by memorizing the path to retrieve it again.

Now that does not work anymore. You know some information is out there, you found it once when google worked, now it's lost in the noise.

I'm learning to take notes again and organize them so I can search them easily.

Yep, i print to pdf a lot now.

Googling "tap size for a 5/16-24 bolt" gives the drill size in the first line of the results page.

For queries like that I now turn to Gemini / ChatGPT first. Of course, this is only a good idea if I have some way of sanity checking the answer. If I doubt the answer I get back I try Google search instead.

You can ask a model to provided an analysis of its answer including a probability that it is correct as part of the prompt, helps with doublechecking a lot.

Is there any evidence that these probabilities are based on any real calculated heuristic?

They're consistent to the model, particularly if you ask the model to rationalize its rating. You will get plenty of hallucinated answers that the model can recognize as hallucinations and give a low rating to in the same response.

If the model can properly and consistently recognize hallucinations, why does it return said hallucinations in the first place?

Models can get caught by what they start to say early. So if they model goes down a path that seems like a likely answer early on, and that ends up being a false lead or dead end, they will end up making up something plausible sounding to try and finish that line of thought even if it's wrong. This is why chain of thought and other "pre-answer" techniques improve results.

Because of the way transformers work, they have very good hindsight, so they can realize that they've just said things that are incorrect much more often than they can avoid saying incorrect things.

You’re right back at square one hoping you can trust the analysis is correct.

No, you absolutely are not. It's like an extra bit of parity, so you have more information than before.

Does that extra information come from a separate process than the LLM network? If not then, assuming the same output is not guaranteed from the same input as per usual, then all bets are off correct?

Sorry for the late reply, but if you read this, there is research that shows that prompting a LLM to take variety of perspectives on a problem (IIRC it was demonstrated with code) then finding the most common ground answer improved benchmark scores significantly. So, for example if you ask it to provide a brief review and likelihood of the answer, and repeat that process from several different perspectives, you can get some very solid data.

I really like Kagi's approach to this, which is to give a list of references. There's still no guarantee that the answer is correct, but you can at least check the references :).

https://kagi.com/search?q=what+is+the+tap+size+for+a+5%2F16-...

> How have we regressed to the point that I'm better off using a paper book than online charts for things that don't change?

because products that require iteration lend themself to subscription models which in turn mean a recurring revenue which is deemed superior to onetime payments for a 'finished product'.

Reply on news.ycombinator.com