Hacker News Reader

Comment by toast0 | Hacker News Reader

I understand some of the frustration though. I was trying to do some audio processing work once. Found the paper(s), which promised code available from websites that are no longer available. Dug through the internet archive to find the zip files with the matlab code; managed to tweak it to run with the matlab version I have; found it works as described with the sample inputs, but crashes horribly on my inputs.

View on HN

Source code availaility for academic papers is important for reproducibility, so other people can run additional experiments and demonstrate that the performance numbers in the paper weren't fudged.

It's not necessarily going to be useful for production use. There are exceptions to this; for example, there are papers were the authors claim that a flash/SSD emulator is suitable for use by other academics to experiment with their FTL ideas, or to grab network traces from NFS traffic so they can be replayed to test file system performance using real world data. In those cases, the point of the paper is to share a tool that can be used by other researchers (as well as the team that created the tool in the first place), and in that case, the code had better d*mned well work. (But even then, there might be buffer overrun bugs in the SSD emulator; which is fine, since the FTL is intended to be provided by an academic researcher, and it is not expected to accept arbitrary inputs including from a malicious attacker.)

I don't know whether the papers in your case were meant to be documentation for code that was meant to be shared, or to explore a particular research idea, the code was only meant for that particular purpose. Even if it was for the former, there's an awful lot of bitrotted, unreliable "abandonware" on sourceforge or github that can be pretty skanky; that's a problem which is not restricted to academically published papers.

I've been debating where the anti-science behavior stems from. From reasonable people at least. The best i can come up with is that most reasonable people recognize how the modern age is an information war. Product sales, articles on economy, articles on politics, even some well advertised miss-steps like the sugar industry pushing/funding pro-sugar anti-fat papers way back (which may or may not be true, but it is a common trope parroted).

I assert that all this leads to people being paranoid about information of subjects well outside their expertise. Which is a really scary place to be. The answer seems non-obvious to me, but is likely nuanced.. and the public doesn't do well with propagating nuance in my experience.

I'm really interested in tooling to help disseminate information.

When I encounter it, I feel it's often a hatred of the "doers vs the thinkers"

My career path was 25 years engineering, before migrating into a hybrid EE/PM role as sort of a natural progression from being "the engineer who knew how to run the project". Once I started learning the more formal approaches to PM, it uncovered an entire world of engineers who have an incredible hatred of any sort of planning of any kind, because all planning time is wasted and we should all just be doing.

The parent comment here feels the same way. Hatred towards research because it's all theoretical (I guess?). It seems clear as day that the best approach is a marriage between the two.

I think it's summed up by the mantra "those who can, do. Those who can't, teach."

I suspect "hatred of any sort of planning of any kind" is actually hatred for planning that fails to embody any actual strategy (and is therefor a waste of time because it doesn't help solve any actual problem except maybe alleviate non-technical vips anxiety with false hope). "formal approaches to PM" evokes just that sort of thing in my mind (kpis/goals masquerading as strategy, gantt charts, etc)

When I worked in a university lab that builds stellarators I learned that misgivings towards researchers is all bullshit. There are engineers and there are pencil pushers. Pencil pushers burn money and bark loudly. Real engineers can plan and execute on time and under budget.

I think one source of anti-science behaviour might be betrayed expectations. I know that I personally have stronger expectations for academia and scientists than for other people. So when someone from academia, a scientist, an engineer betrays these expectations, I feel worse than if a regular person did it. There's a feeling of "If we can't even count on those people, what are we even supposed to do?". For example, at the beginning of the COVID pandemic (around January 2020), I read a lot about it. Lots of very smart people were saying that this could be a big pandemic. I talked about it with a doctor in a non-professional setting that told me to basically not worry about it, that it wasn't going to be anything huge. This time I was right and he was wrong. Was it because I searched more about it? Was it just luck? I don't know. But I know that it made me lose a bit of trust with that person.

I think the origin of this might be on how I (or we) see those people. You're supposed to follow what the doctor tells you, what the scientists tell you. But in a way, since you're supposed to follow what they say, they have some kind of responsibility towards you. And when they say something wrong, it's way worse than when a regular person says something wrong. It's like when you're young and your teacher or your parents are wrong, it's very frustrating.

Your example about the sugar industry is also a great one. Try to understand a bit more about nutrition, and soon you'll hear all kind of conflicting advice and explanations from very different experts.

I know that personally I have to work on myself and accept that those people are humans, and make mistakes, just like me. But just like telling people to eat less and move more didn't solve the obesity epidemic, I'm not sure that this solution will scale to a large population.

This is interesting, and I've seen a bit of this sort of behavior, too.

Some people seem to confuse expertise for a claim of infallibility, and when some expert get something wrong, the reaction is to conclude that expert advice is worth no more than the guy on the teevee hawking vitamins and anti-expert bile.

It is a sort of Leveler belief wrapped in a search of an Oracle.

Something you may be more familiar with is people's concept that someone that "knows computers" is familiar with any and every sort of task that involves a computer whereas in fact this could encompass a wide variety of different skills that require an individual investment of time.

The same can be said of medicine where encompasses a very broad set of skills. Your doctor may have been an expert in sports medicine or brain surgery but it doesn't automatically make him competent and epidemiology. It also doesn't force him to pay attention to current developments in the news which is likely what informed your opinion. Personally I found it was completely obvious in January that we would be dealing with a crisis because I followed the situation and suspect strongly that your doctor friend did not.

There is also the issue of survivor-ship bias. We worry about many things and we will absolutely recall the times our worry was justified and forget when it we are mistaken. If Yellowstone ever blows there will be many people who knew it was just around the corner and this will be true if it blows now or in a century, whether or not we have any scientific basis for the thought process.

TLDR: A singular doctor of unknown specialty getting it wrong in January isn't a flaw in science. Science isn't expected to be very good at ensuring a single expert of only tangential expertise gives you the right answer whereas it is reasonable good groups sometimes slowly arriving at increasingly correct answers. If you want a more correct answer consider consulting or reading what several people of relevant expertise who are up to the minute on current information have to say.

Readers of Nassim Taleb, doubts raised over our blind trust of institutions like their reaction to coronavirus, any mainstream nutritional advice (fat is bad and causes heart disease, “the China study” was written by a nutritional biochemist), papers that p hack to be published, replication crisis of psychology, statistics and using it to lie.

There is no paranoia, an allegory is an ivory tower of studies about language from non native speakers with a PhD, and a native speaker who gets no recognition for using it daily but doesn’t have a fancy diploma or credentials so someone who speaks “proper” Spanish from Spain who has never been to Spain is more “credible” than the Mexican speaking “improper” Spanish daily.

Academia is to be ignored unless it’s relevant, Fritz Harber didn’t need the Nobel prize to have real world effects in nitrogen fixing to help farmers grow and sustain our population, Obama wasn’t more relevant because of his Nobel prize, and Perelman’s refusal of the Fields Medal doesn’t change his contributions.

Readers of Nassim will also recognize his critiques of academia are mainly targeted at social sciences and similar fields that can only “prove” their findings statistically, where p-hacking and incorrect use of models and wrong distributions and the like result in bad findings passed off as good.

That’s not the case with computer science, at least in systems subfields like filesystems, where theories can be implemented in isolation and shown to either work or not.

Disagree. Artificial benchmarks and p hacking for showing good performance in CS are also statistically proven. The best result in geekbench doesn’t mean anything to me.

Benchmarks and geekbench are not "where theories can be implemented in isolation and shown to either work or not."

Maybe I don’t understand your point, in psychology for example sterile lab tests are isolated and can be shown to work or not, is it not the same idea here?

CompSci theories are essentially mathematical proofs. You create a proof of something, then you build it to test it, to make sure your math is actually correct, and that the theory works in implementation.

Proof of correctness doesn't rely on having a large cohort of test subjects undergoing an experimental trial of some sort, and then interpreting the results with statistical models, distributions, p-values, etc.

I don't know psychology in depth, but if there are similar kinds proofs without requiring statistical analysis of a large experimental cohort, then I don't think Taleb's criticisms are aimed at those either.

It's the fundamental problem of knowledge - can truth be known via logic and reason, or via empiricism and observation? The answer to both is, sometimes, but with caveats.

Peter Norvig also wrote a good take on all the ways studies using experimental cohorts can go wrong: https://norvig.com/experiment-design.html

Reply on news.ycombinator.com