Comment by jraph | Hacker News Reader

Comment by jraph | Hacker News Reader

Parent Full thread

> Just knowing that there is such different ways of thinking is useful

We agree. People have different ways of thinking and interacting. Maybe that asker-vs-guesser thing made you / others realize that (and that's good! Possibly it made me realize that too, although having a flatmate years before had already done the trick tbh), but we didn't need it to know this.

> there clearly is a stark divide in behavior

How are you sure it's not confirmation bias [1]? When you have a hammer, everything looks like nails. When you have an asker-guesser theory, everybody look like askers and guessers, including yourself.

Odds are it is most likely, in fact, confirmation bias, since that theory was found to be unsubstantiated and underdeveloped, and since this is a sexy topic, it's hard to believe nobody tried to validate it rigorously (and the way scientific publishing is currently organized sadly doesn't encourage publishing negative results).

> Why not?

Because apparently, from what we actually know (robust, established knowledge), there's no good reason to think the following is actually true, even if it strongly feels like it for a host of reasons, which is my whole concern:

> this framework is good to understand how people think socially and have a better understanding towards one another

It's too easy to pick two half convincing categories that feel somewhat opposite and have the feeling that these two categories provide insight on how people work. Such theories are sugar for the brain.

I'd be most happy to be proven wrong in the future though! In the meantime, I'll pick cautiousness.

[1] https://en.wikipedia.org/wiki/Confirmation_bias

Oh, I see what you mean.

I agree with what you say regarding confirmation bias but then how do you separate that from what is considered the scientific consensus? What I mean is that Newton's Law is not scientifically accurate anymore (it's good enough, though) but the fact that it validated what we observed (i.e. gravity) is also confirmation bias.

What I'm getting at is that there is a fine line between confirmation bias and scientific theory. I hope I made sense, lol

Ooh that's a good question, how do you control for confirmation bias in studies?

I'm a bit embarrassed to have to admit that this goes beyond my knowledge. I'm sure there are answers to this, this must be well known in these areas of research. We also know that research itself can be biased too. I'll have to ask friends working on these topics! Thanks for the interesting discussion about this I'll probably live in the future.

On this topic specifically though, that meta analysis that concluded there was a lack of evidence was despite the potential confirmation bias (unless the authors of the meta-analysis where already suspicious about the theory… oh well… one can hope them following the scientific method provides strong enough guarantees. It's not completely bulletproof but it's the most reliable thing we have. I'll ask for sure!).

> but the fact that it validated what we observed (i.e. gravity) is also confirmation bias

Pretty sure that's wrong. The way it works is: we have this equation. It predicts where we expect such stuff to be in X seconds. In X seconds, we check it's indeed there. It's there: actual confirmation, not confirmation bias. That's how you check your hypothesis. Of course the initial hypothesis comes from intuition… formed by observing the world. Enough confirmations makes your model more reliable, and is the thing that will be used until a counter example shows its nose and a better model is found. Even then, the model can still be used for cases where we know it does the job; Newton's model is simpler to use than Einstein's so we keep using it.

I guess if you have a solid enough hypothesis, it also works like this in human sciences.

> Pretty sure that's wrong. The way it works is: we have this equation. It predicts where we expect such stuff to be in X seconds. In X seconds, we check it's indeed there. It's there: actual confirmation, not confirmation bias.

Exactly. My point is that since Einstein's theory, we know that Newton's Law is incomplete. Therefore proving that it was confirmation bias (i.e. that our equations just confirmed what we observed). Since we observed black holes, we knew that Newton's was incomplete as it couldn't fully explain their behaviors.

> i.e. that our equations just confirmed what we observed

No, no, it's the opposite, and it's key! What we had been observing kept matching what the equations gave us "so far". Without cherry-picking, or refusing to see the cases where the model doesn't apply (consciously or not), which would have been confirmation bias.

We did, in fact, question the model as soon as we noticed it didn't apply.

Confirmation bias implies "cognitive blinkers", I don't think this happened in this Newton vs Einstein stuff.

But I agree the confirmation bias risk is not very far away. It's an issue in the general population, it's also likely a big issue in research.

Don't we start the equations after observing a phenomenon? It wouldn't make sense to try to explain something before observing it..

For example, after observing black holes we understood that Newton's was not enough to explain them. Thus we had to find another theory that explained our observations. Now with quantum computing we know that Einstein's theory is insufficient, too (not very knowledgeable on quantum physics myself, though)

There's definitely a "Seeing an apple fall and intuiting a hypothesis" process which is early in the research process, which leads to formulating the equation as an hypothesis somehow.

So you observe stuff, intuit and formulate an hypothesis. The hypothesis is a model that you hope matches how the world works well enough. Developing a scientific hypothesis takes scientific rigor. Among other things:

- it needs to be testable (it needs to be possible to design some scientific protocol to check the hypothesis)

- it needs to be formulated before you start experimenting and collecting data (that doesn't mean you can't observe your world before, you just can't use these observations in the data that backs your thesis)

- it needs to be rooted in existing science, knowledge, it's not a simple "naive" guess. It certainly takes being deeply familiar with the research area.

Then you test your hypothesis with experiments. You design a significant number of them. You must not cherry pick here, that would be confirmation bias (but you can encode the limitations of the model in the previous step). You predict your expected results with the model you have in your hypothesis. You run your experiments, make your measurements, compute the deltas. Here too, you must not discard or tweak the results to your liking. That would be cherry-picking, or event outright falsification. If the deltas are small enough, and people review your work, and ideally reproduce it (same experiments, or other experiments), eventually there's a consensus that starts forming around your model. Congrats, the model is validated.

So, people start to use your model. They do exactly the opposite of what you did when you formulated your hypothesis: they don't try to come up with a model from preliminary observations, they assume the model works, and they use it to predict the future.

Until Einstein comes :-). And stumble upon a black hole. An observation that doesn't match. Then your model gets refined (with limits and restrictions) or even "deprecated".

But yeah, physicists model the world after the observations they make, not the contrary. Otherwise they are doing something else. Math, maybe, or philosophy, or whatever. It's just that designing the model is only the beginning, you have to check that it works… with carefully selected observations… before it can be validated.

With this asker-vs-guesser thing, we don't have convincing work that provides the validation step. This means asker-vs-guesser is an hypothesis, at best (at best, because we don't know if things required to formulate a scientific hypothesis have been respected).

Reply on news.ycombinator.com