craftsman

The alternative I've suggested is rather different than the suggested alternatives of accepting results or taking them for granted. In another comment I said:

> I'd suggest reading every paper critically, realizing (as most people do) that the results are tentative, and either letting it go at that, or dig in and do the hard work to figure out what may be flawed in the work presented. Then work with the authors or publish a paper of one's own pointing out the flaw and the new resulting analysis.

It's fine and quite legitimate to have a prior, as your example of a perpetual motion machine illustrates. Along with the prior it's also important to understand how typical cognitive biases can affect those priors. For example, you cite the "mere experience of other such announcements being repeatedly proven wrong" and "the probability of the team just being lax with their annoucement to get publicity and grants." These might be examples of availability bias. Clearly these things happen. The question is whether they actually happen with a frequency that would justify the statement:

> Finally, I know that "we found RNA building blocks and B3" on a meteorite is less likely that contamination, so the story seems fishy and premature.

Regardless of each of our own priors and biases, respectfully, you don't know this. You don't have evidence that shifts the probability in this direction. Or rather, you haven't presented any. You haven't pointed out a specific flaw in the experimental design, instrument design and operation, data collection and analysis. What you have done is state your strong prior suspicion that something is wrong. That's not knowledge, and it's nothing that anyone else could base their assessment upon. The statement is simply a statement of your prior, nothing else. It's not a material statement about the actual paper, its data, analysis, or its conclusion.

> Finally, I know that "we found RNA building blocks and B3" on a meteorite is less likely that contamination, so the story seems fishy and premature.

Since you rightly encourage everyone to question claims, I question this claim. How do you know this, specifically?

In other words, we have a situation where a team has done a bunch of experiments, data collection, and analysis, and then published a paper claiming "we found RNA building blocks and B3", right? And now your claim is "no, you're wrong about that," but without any experiments, data collection, and analysis. So if you're going to make a claim, provide the details, data, and evidence that substantiates your claim.

As I said in another comment, you may be reading too much into the statement to the press as opposed to the actual paper. It's more likely that the author quoted as saying that something could be "ruled out" meant, more precisely, that the possibility of contamination is very small, but non-zero, and that the overwhelmingly more likely conclusion is the existence of said compounds.

As I said,

> If your post was simply to say, "This paper could have flaws and its conclusions could be incorrect," then yes, almost all of us will agree with you. I'm not sure it's big news, however.

Disagree. It's trivial to question. It's non-trivial to do the work required to show evidence for or against some proposition. What is at "the root behind a lot of issues we've seen socially" is the easy path of questioning something without actually being able to do the work required to back up one's claims or counterclaims. So I believe the reaction which you think of as "trust the experts / science" is actually "do the work and show the evidence".

If a paper is published which claims that such-and-such drug shows effectiveness against shingles, it's easy and trivial to say, "Not so fast my friend" followed by any number of statements about Big Pharma, citing previous situations in which a claim was later shown to be false, etc. But while all of those things may, in fact, be true, they don't bring to bear any actual evidence that this particular drug study is flawed and exactly how.

> But merely been less confident (that they are) in their thoroughness and in their preclusion of contamination, is much easier to claim and be correct while doing so.

But it's a trivial observation, and one that isn't actionable or testable. For the team to have designed and performed the instruments, experiments, data collection, and analysis to say "We're confident in our results and we have ruled out contamination" is substantive and is backed by data and actual experiments. For someone else to say, "I'm not so sure, I think it could have been contaminated," and then provide no actual argument or data on what could have gone wrong, is probably not a novel observation or comment. In addition, it's an easy thing to say (as you correctly point out) without putting oneself on the hook to show how or why.

Here's what I mean: we can confidently say about every paper ever published that "I'm not so sure." Here are the possible outcomes: (a) the paper stands the test of time, the result holds, and nobody remembers anyone who said "I'm not so sure," or (b) someone finds a flaw in the paper, works with the authors and they publish a retraction or revised paper, and then whoever said "I'm not so sure," can easily (as you say) point out how "they called it," all without doing any actual work.

I'd suggest reading every paper critically, realizing (as most people do) that the results are tentative, and either letting it go at that, or dig in and do the hard work to figure out what may be flawed in the work presented. Then work with the authors or publish a paper of one's own pointing out the flaw and the new resulting analysis.

I think you're reading too much into the phrase "can be ruled out." I suspect that the leader author was saying "Contamination is possible, but based on our procedures, test methods, etc, the overwhelmingly larger probability is that we did detect <...> in the samples, and that the detection was not from pre- or post-contamination."

Could the paper's conclusions be incorrect? Yes, I think everyone--the authors included--will agree that "something could have gone wrong." But pointing out that "something could have gone wrong" is probably not a new idea to most readers.

If your post was simply to say, "This paper could have flaws and its conclusions could be incorrect," then yes, almost all of us will agree with you. I'm not sure it's big news, however. I would guess most of us understand that scientific findings are contingent on future experiments and results.

If your post was to point out a specific flaw, e.g., that the sample could have been pre- or post-contaminated, then it falls short of anything substantive and actionable. Nobody can do anything with it. Yes, you are correct. But no, there is not anything anyone can do with the critique. If you're going to point out a flaw, it needs to be something that you, the authors, or other PIs can check, perform, etc.

I think what the other poster is trying to say is something like the following:

There was a team / teams who planned the mission, designed and built the spacecraft and its systems & instruments, designed and performed tests on the samples, analyzed the data, and published a paper with some claims. The paper outlines their methods, test results, data, and conclusions.

Now, is it okay to question anything in the paper? Yes, absolutely.

How probable is it that something in the paper is wrong or could be improved? Definitely non-zero, it happens a lot. People who understand various aspects of the instruments used, the tests, data analysis, etc, often find some minor or major flaw in scientific papers. Could it happen with this one? Yes, absolutely.

How probable is it that someone on HN--knowing that there are lots of people on HN with diverse expertise--will find something wrong with the paper, and that the substance of the flaw they found is:

> I seriously doubt that it could be "ruled out" just like that.

> For one, the spacecraft (or the empty capsules) could have been pre-contaminated before their launch.

> Second, the capsules could have been contaminated after arrival even if they were "sealed" (perhaps the seal had issues, or maybe it happened when they opened them, since one assumes they couldn't test still sealed capsules, they'd have to open them).

Non-zero, but very low. Could it happen? Yes, it could very well be that the team(s) involved did not consider that contamination could occur before or after the sample was obtained and analyzed. How likely is it? Hard to say, but if it happened, the critique would not take the form of "Hey, it could have been contaminated!" Rather, it would take the form of "I notice in the sample handling procedure that such-and-such technique was applied. Further, in the data analysis, such-and-such test was not done, therefore one could not rule out ..."

You get the idea.

(a) they wanted to shoot it down in a location in which it would be easily recoverable, and (b) they didn’t want to risk damaging persons or property.

Not in the long run. If CO2 levels are increasing and the energy balance into the Earth system is positive, then the basic physics that the original poster referred to will result in warming. The net energy increase of the system will result in a higher equilibrium temperature.

I understood their point to be that that based on our understanding of basic physics and chemistry, the energy balance is such that the Earth system is gaining energy, and that this will lead to increased temperatures among other things. Certainly there are all kinds of complexities about how that energy will be distributed and what the effects will be, but just in terms of a simple energy balance model based on well understood physics, it would be difficult to make the case that warming won't happen.

You: "...Extreme left authoritarian political opinions, which is what Political Twitter was isolated to before elon"

If true, then we would not find much, if anything, that was not "extreme left authoritarian political opinions" amongst the detritus of "political twitter", right?

And yet, when shown that not only did there exist content from the political right, it was amplified more than content from the political left, you reply:

"you responded with an unrelated fact"

Let's try it this way. Let's say I make a statement that the birds at my feeder are isolated to crows. If you then point to my own videos which show not only that there are many other kinds of birds, but these other kinds of birds eat the most seed from it, what should I reply? "Oh, I'm sorry but that is an unrelated fact"? Or perhaps, "That proves nothing, how do we know the other birds were not brought there by the crows?" Or how about, "The fact that there were other kinds of birds at your feeder does not refute the fact that you prevent non-crows from coming to your feeder." Etc.

I infer then that you think the concept of a creator outside of our universe who caused it is a concept of no consequence. If so, then on that we agree.

The difference being what those beliefs are based upon.

I'd recommend a read through Plato's Socratic dialog Theaetetus. Then think through whether the progression of sensation, true belief, and justified true belief does or does not form a sequence of increasingly better ways to think about the world.

As for your last sentence, although I could say the same, I think you will agree that it would serve no purpose.

With the difference being that the former do not self correct when it happens, while the latter ultimately do self correct.

And yet, this mythical common man--who you say has to just choose who to trust--will find that if he chooses to build civilization by trusting the imperfect holy men, it will pale in significance to the civilization built by other common men who choose to trust the imperfect scientists.

The missing ingredient is that if the common men--who you think have no better option than to depend blindly on someone they choose to believe--build a civilization based exclusively on the meditations of these holy men who they have accepted as the repositories of truth, they will find that their society will fall to those societies which have a, shall we say, more empirical basis for their view of the world.

Correct. And over time we--both as individuals and collectively as a civilization--gradually cull those things which seemed to work at one time but have become clear that they do not, and gradually adopt those things which seem to work better than those. In the end, over long periods of time, what matters is what does work, not what seems to. A society which believes than matter is made of small blue regular polyhedra will fail to develop as well as a society which continually culls and refines its beliefs based on experimental evidence.

Yes, I have. The next step beyond thinking "Aha! Replication crisis!" is that the replication crisis is directing you towards its correction: the goal is replication, correct? And in many instances we fall short of that goal. The corrective action is to refine standards to which we hold scientific research. It is not correct to say, "Darnitall, we have a replication crisis, science is no better able to get us closer to things which are true than religion, divination, palm reading, astrology, and coin flipping." And that is exactly the sense in which the replication crisis is improving our ability to figure out which things are more likely to be true than others.

The claims that they performed miracles and various testaments to their good character are hearsay, not evidence.

If one man claims that god said to him “such and such”, he may think that god has spoken to him, but I’m not obliged to. All I can say with certainty is that he is a person who thinks god spoke to him and subsequently told me about it.

You seem to be focused on the so-called common man and his inability to do anything more than just believe things. I think you do this common man an injustice.

Let’s say this common man is religious (pick one) and is going about his day and is injured in some accident. He is taken to a hospital, and one which he knows is aligned with his religious beliefs. Upon arriving the doctors triage him and tell him he’s bleeding internally. But then they tell him that they have assembled a team of highly respected members of the religion to which he adheres and these persons are revered among its followers. They are about to lay hands on him and pray for his internal bleeding to stop. He asks if they will also be performing surgery, to which they reply that no, their god is loving, caring, and will hear the prayers of the faithful and heal him.

Now if you really are talking about the truly common man, this common man will, with whatever strength that remains, object quite strenuously! He will insist that a doctor perform surgery on him immediately and stop the bleeding.

And why does this common man do this? Because what he really believes is that while his religious beliefs may provide some type of assurance and comfort, when the rubber meets the road, he’d like some science, please.

Except you can find another group of holy men who disagree with that group. And a third group that disagrees with those two groups. The number of groups ends up quite large, and nobody can quite seem to figure out a way to test whether one group has it a bit more right than the others.

The fact that things seem to work does not mean that they actually do work. We all are subject to cognitive biases which can trip us up. Most people aren’t very good at recognizing when that happens. That’s why these things survive so long.

And no, that’s not moving goalposts.

Except when we test them, they don’t seem to work after all. Would you fly on an airplane engineered by someone trained in engineering, or a plane engineered by someone who relied only on inspiration from a deity? Would you go to the hospital with doctors who practice typical Western medicine, or a hospital in which the doctors only pray over you? Etc.

Ah, well in that case, the parent poster can just say that asking about the cause of the creator is a valid thing to talk about, because they mean a subtly different kind of ‘cause’ than the kind of cause you told them wasn’t valid to talk about outside of the universe.

The difference is not only in whether they are verifiable, whether they are verified, but also whether things which are produced from them actually seem to work. Religious claims fail at all three: they are not verifiable, they have not been verified, and they don’t seem to work.

See my sibling for my first reply.

My second reply is that it’s important whether things derived from two different claims actually work.

Make two hospitals: one which follows standard Western medical practice, and the other which does not and instead performs prayers over its patients. I can predict which hospital will have better patient outcomes.