Hacker News Reader

> Student experiences of course vary widely, and even for those that do the menial slog version of a bio PhD, the attitude surrounding it varies. Some seem to be happy for the guidance when their advisor is like this while others resent the lack of intellectual freedom.

This section I think captures the problem well: most good (future) scientists will recent the lack of intellectual freedom, while the less capable will appreciate the guidance. This mechanism pushes academia more and more towards mediocre Cargo Cult Science, is my impression.

And I think we’re seeing the same theme playing out all through society. It’s convenient for the conformists at the top that need cheap compliant laborers, and it’s good for the (many) mediocre conventional-minded that enjoy it. But it’s not good for results.

> This mechanism pushes academia more and more towards mediocre Cargo Cult Science

Ultimately, a cargo cult is a scientifically sound application of empiricism that just happens to completely fail, whereupon it may be rejected. If scientific methodology is evaluated empirically (which I endorse) in terms of enabling scientific progess, you end up with a cargo cult.

For example, why is it a good idea to have falsifiable hypotheses? Most philosophers don't adhere to this standard. You might make a great argument on the grounds of what scientific progress means, how tangible it is and how it prevents discussions that are in some precise sense a waste of time. However, if you ground your epistemology on experience, the best and most convincing reason to adopt falsifiability is that it's historically been enabling science in ways that we like. One non-obvious prerequisite for this is that people seem to more easily agree on whether something is falsifiable than on whether it is true. I think this is an empirical psychological fact.

Concerning what is a good scientist, I consider as the important metric their individual contribution to humanity's knowledge. It may be that the biggest marginal contributions are made by people who identify epistemological deficiencies in their field and initiate paradigm shifts. However, I believe most of the total contributions to scientific knowledge are made in ways that barely benefit from deep epistemological insights.

I think it's very hard to argue "it's not good for results" that we have too many compliant laborers, if "results" is advancing human knowledge. Successfully teaching philosophy of science to people such that they can meaningfully apply it in practice is hard and requires significant investment. I'm not convinced such efforts are the best way to advance science. Rather, I think a more effective measure would be better aligning the incentives of scientists with the goal of advancing humanity's knowledge.

You need both types for sure. The major problem is that they've been confounded, and the more creative ones are subsequently being driven out at an alarming rate.

There are also not enough spots for the worker bee types, because even though many do reach PI status, the majority don't. The majority of people in general do not become a tenured professor, and the entire academic system is essentially like minor league baseball where either you make it all the way or you exit.

There should be way more laborers working on bio research, a lot of which doesn't even require understanding the science on the level of the "tradesman" PhD, let alone understanding the science on the level of being able to break new ground. This could involve both new viable long term positions to be a laborer in a university bio lab and a greater diversity in the type of biology research that can occur in industry. But government funding agencies needs to get their shit together first.

>To think deeply about meta science in any official capacity as part of PhD training requires you to go out of your way.

Yup. Not just life sciences, this applies to physics as well.

>I also imagine that even in less expensive/slow fields there is an increasingly unapproachable body of prior knowledge to learn, and people don't want to be trainees forever.

This is where AI might come in handy, for compressing the time it takes to ingest the prior work by providing tailored educational material, or even obviating the need for it by augmenting our intelligence (esp. when it comes to recall/search). Otherwise, we'll eventually be in the situation of Schlemiel the painter [0].

[0] https://en.wikipedia.org/wiki/Joel_Spolsky#Schlemiel_the_Pai...

I'd not heard of that problem but it's a funny story, thanks for the analogy!

> a large philosophical paradigm shift might be needed too

Why? The way to learn a lot of prior knowledge is focused study and good teaching. Philosophy has little to do with it. I agree that it would make sense having more specialized lab workers (which definitely is a trade and should be learned while doing like one) assisting scientists whose education must be broader.

Another thing concerning the philosophy is that the current environment and competition for resources at research institutions rather discourages good scientific practice. E.g., it encourages overselling results while never attempting to reproduce them. This is getting better in my opinion, as e.g. in the open science initiative. However, I find it very hard to agrue that teaching more philosophy of science would do much good here. It's a problem of incentives.

Another question is whether science would actually progress faster (or in some other way better) if the philosophical ideals were followed more closely. I don't see how the situation in bio labs that you describe would benefit from all PhD students having heared three lectures on philosophy of science. The occasions where the philosophy actually informs decisions in the lab are very rare (though sometimes very important) and there seem to be enough people who know these things well enough for their purposes when this knowledge benefits the lab. When leaving academia, few students will have a chance to apply these concepts and most at least know that scientific methodology is a thing and will learn it if actually needed for what they do. Furthermore, most fields have their own somewhat incompatible folklore methodologies that do not easily generalize to unrelated problems.

No as fields get deeper and deeper there is a point at which it is no longer humanly possible to cover the prior material exhaustively in a vaguely reasonable timeframe. We can keep digging narrower and narrower niches at the expense of basic scientific literacy, or we can realize that at some point the system cannot keep scaling in a one dimensional way. Or I guess we can make people go to school until they're 60.

If you think what I'm proposing is three lectures on philosophy of science than either you have zero idea how a PhD program works or you're arguing in bad faith. The fact that someone can graduate from a top 5 biology PhD program without ever demonstrating the ability to come up with a decent research question nor to independently plan a decent study to address a given research question is appalling. The fact the programs actively discourage students from exploring anything intellectual outside of the narrow framework of their chosen lab, even projects that would clearly fit under the program's broader umbrella, is also appalling.

The primary point isn't whether students can learn those things on their own later. The primary point is that this is a huge indictment of the culture of graduate training. It is creating a legion of fucking drones as our next set of profs. Many of them are truly not aware that they lack this expertise.

Also, a PhD is explicitly designed as training to be in academia. There are a small set of industry jobs that require a PhD, and there are more industry jobs that appreciate a PhD, but really the only reason that PhDs have shifted so much towards industry is because the academic job market is completely fucked. It is only recently that biology professors will entertain the idea that someone should choose to leave academia after a PhD and that is not considered failure. Yet at the same time they do a bad job of preparing others for academic research.

> If you think what I'm proposing is three lectures on philosophy of science than either you have zero idea how a PhD program works or you're arguing in bad faith.

Talking about three lectures was meant as a question of what you actually propose (you might say that's bad faith). However, I believe you will find many people who you can convince that this is an important problem and they will suggest the "lecture approach" in full seriousness. That's also what I would expect from administrators who can actually implement some change in response to the problem.

You still haven't said what measures you would actually endorse. I will try to guess a second time: Do you want to make each PhD student design their own experiment, perhaps with individual hands-off supervision to teach methodology? Either these experiments will be "toy experiments" to teach them methodology, or they would have to make a real contribution to science, in which case you would have to massively increase research budgets (or massively reduce numbers of PhD students) and add a LOT of non-PhD student researchers who do all the work for your newly made PhD-PI. In the latter case, these experiments might not actually deliver on their promise of generating much knowledge and, I believe, would prove an investment in the PhD's education at the cost of less scientific output.

You will not convince administrators making economic decisions to implement either of these, even if they agree with you about the urgency of the problem. I think it's quite instructive to think about why this is so. Which brings me back to my main claim: the important problem in modern science is the incentive structure, not the lack of scientific methodology knowledge in PhD students.

You are right about the incentive structure being a main problem in science.

But at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work in the first place.

Remarably, nearly all the commenters here betray this very lack of knowledge by talking about it as "philosophy". As if it was some practically irrelevant jibberjabber.

Science does not work by magic though, with people going through some motions of an occult ritual. It's an algorithm, implemented insanely badly currently.

> at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work

I don't think people's lack of understanding scientific methods is the main factor shaping the incentive structure, though it's very interesting how big a role it plays and I would love to hear more detailed arguments.

In order to change the system, there must be enough alignment on what the actual goals are and what incentives might benefit them, ways to effectively organize around it, and ways to actually enforce the new incentives. (Perhaps even more things...)

I think there are big hurdles on each of the above. First, concerning alignment on the goal, many people don't agree that faster scientific progress is actually good. You might argue that trying to teach people epistemology might change that. I neither think it would nor that it should, but one could argue about it a lot. "Indoctrinating" people starting at a young age that "science and knowledge = good" as a moral value might work (it is being done and I mostly endorse that) but that's different from teaching the scientific method.

The really important important factor here is, of course, that many individuals and organizations benefit from the current system and don't want to change it.

Concerning organization, political intersts and competition hinder it somewhat, but co-operation between scientists is quite high and increasing. Hopefully that trend will continue and not sucumb to geopolitical pressures.

Concerning enforcing better incentives, that's very hard. For example, we can't objectively and reliably evaluate someone's scientific contributions. Any metric you set up is going to be gamed, etc. I think the incentives have improved over the (very short) history of collaborative science and the trend looks positive. Take for example the trend towards open access publication and growing awareness of many issues (replication crisis, etc.).

Generally, I would say that understanding how science (the system) works and how it should work (or would, given certain changes) is very hard. Similarly, I also think really deep understanding how science (the process) should work is also hard and applying that knowledge in practice is much harder still. At some point you get diminishing returns for the efforts of teaching it. You still haven't proposed how you would promote this knowledge. I can't think of anything that seems likely to do much good but would love to hear about it.

> Remarably, nearly all the commenters here betray this very lack of knowledge by talking about it as "philosophy"

I believe epistemology and philosophy of science are philosophy. I also believe that having and improving these things is very useful and one cannot overstate their role in getting science where it is now. This is in my mind perfectly consistent with there not being a great need (way?) to teach more of it to students.

Perhaps an explanation of how you would teach it would make me realize we haven't been talking about quite the same thing?

It's harder to game metrics when there is a wider diversity of them and the way in which they are weighted in different decision making committees is different. Because it's hard to simultaneously game a bunch of different metrics, and the benefits from gaming a single one are probably worse than the benefits of just organically doing what you think is best.

But yeah I totally agree that these are extremely hard problems with no silver bullet. Philosophically and from a "how do systems of humans work together" perspective. Teaching methodology of science like a cookbook I also agree is the wrong play and just digging in further to what got us where we are.

I'm curious what makes you say scientists are collaborating more? Stats or has that been your experience? I've seen a lot of semi political collaborations where people are mostly just posturing to get on more papers. I have also seen a decent number of good faith interdisciplinary collaborations, though it seems to be a bit of a coin flip whether they flame out over disagreements. I've personally not seen much cooperation within the same field though that I'd consider genuine.

Thanks for sparking the interesting discussion in any event, it's not often I end up debating people on both sides of a debate so this has been fun.

In my field there's lots of honest and fruitful collaboration. It's not all perfect but definitely a good situation. Technology has been making collaboration easier and faster as well. Globalization has led to international scientific collaboration being ever more expected and normal.

Look into the organization of CERN for example. It's a huge project, highly politicized in many ways, with many parties that have their own goals and motives. Yet they all realize that a huge and sufficiently effective collaboration is the only way to do this science at all. And they are doing it.

I'm not sure why you think "philosophy" and "practically irrelevant jibberjabber" are synonymous. Or maybe you just think that's what commenters are implying? That's certainly not what I was trying to imply. Scientific thinking can be super practical, and it need not involve meta thinking for everyone that uses it. But there absolutely are philosophical discussions that remain of relevance today, discussions that some part of the scientific community ought to be having.

Besides, academic science has become more formalized than it used to be, not less. People are running the algorithm badly IMO not because they are trying to be philosophical or eschew the previous fundamentals of science, but because they lack a deep understanding of what they are attempting to implement. The guardrails that were put up to try and force things to be more formalized have backfired into becoming a bad crutch for shitty science.

It's also not surprising that certain metrics and incentive structures would break in a career environment where resources have become scarce. Things implemented in good faith 25 years ago may not have even been a bad idea at the time, but they definitely were reacting to a different era with different job expectations and different open questions.

Anyway, I definitely agree with most of your broader concerns but I'm not sure if I'm misinterpreting your statement about philosophy. In my experience present grad students are hyperfixated on highly unimportant (from a training perspective) technical details. When I've been around PhD students chatting about science at lunch or house parties or whatever I more often hear discussions at the level of abstraction of mouse perfusion techniques than philosophy of science. It is certainly theoretically possible to have too many eggheads, but that's been the opposite of my experience.

Most decisions made about the structure of PhD training are not made by random administrators like undergrad. The head of a program will be a faculty member in that department actively doing research and advising students, usually a faculty member that has been very successful in their research output (at least by official metrics). The decisions on how to structure first year curriculum, what to require for graduation, etc. are all made by PIs in that department.

Of course you are 100% right that those PIs are short term incentivized to keep the cheap labor coming. I think there's a decent chance that this utterly backfires on them in the next few decades, but I guess we will see. It has already started to backfire as far as obtaining postdocs goes.

Anyway, a small scale example of a better academic requirement IMO that would definitely be implementable by those making program decisions (because it already has at UCSD neurobio, perhaps elsewhere) is to change qualifying exams. In engineering programs I know quals involve grad level course work exams too so they're already a big time commitment, but in many bio programs it is simply a project proposal presentation and you get asked some questions on the background lit.

Because the project proposal is just a project you are planning to do in your lab for your PhD, it is not uncommon for it to be pre-scaffolded by the PI, not requiring original thought. At UCSD they require that you write your quals proposal/presentation on a different neurobio topic entirely from anything to do with your chosen lab. This makes it far more likely that students will put together an in depth scientific proposal themselves before embarking on executing what is often someone else's. Of course their proposal does not get implemented, it is just evaluated and given feedback by a committee of established faculty members.

However what I was getting at on a bigger level is that not all PhDs need specialized meta science skills, but some do. It doesn't have to happen overnight, but I'd like to see the process started by having some sort of branching in PhD requirements, perhaps like the quals change I described along with differing year 1 curriculums being optional. This shouldn't require many resources because at the graduate level it is completely fair game to stick some students in a room and give them some suggested references and a problem, and check back a month later. The important part is having not only the clear encouragement but also the protected time.

Oh and I think more opportunities to work with fellow students on meaningful projects is sorely needed too. PhD students from different labs mostly only help each other on a very surface level, it is rare to see students work together across labs in a way that each would be contributing authors on the other's paper. This has the opportunity to still be productive from a traditional viewpoint but also to naturally produce more thinking about a range of science.

Anyway, at a minimum I think professors can stop telling PhD students that e.g. wanting to attend a meta science reading group is a waste of time. Having that as a component of PhD education need not be compulsory or even expected, but for it to be culturally frowned upon is bizarre to me (though the selfish reasoning behind the attitude is obvious).

So the incentive structures part is again true, but incentive structures aren't going to change without an attitude shift and a willingness to diversify intellectually. And it kind of is chicken/egg - the incentive structures created a bad educational environment for some time now, and some of those people are already early career profs. They're in an even worse position to be trying to fix things because they're more ignorant of the fundamental problems to begin with. Seriously there are bad actors but there are also people that are just remarkably unaware of how to be remotely statistically rigorous for example. Either that or they are really good actors who are willing to look dumb in front of certain peers to further their long term goals, but academics don't strike me as the type to be content with playing dumb (if it's being perceived by others in any event).

Regardless, I'd be happy to start from a place of changing incentive structures if that were possible, and work from there. But at the end of the day I think this is an important factor for the incentive structures to consider. Bad training systems can create a real telephone game effect after a few generations.

> there are also people that are just remarkably unaware of how to be remotely statistically rigorous

That reminds me of D. Kahneman's self-righteous book where he recounts teaching statistics for years "without it ever occuring to him that it might be used in his research". Claiming to have learned this lesson, he proceeds to promote his book while half of it's claims have failed to reproduce. He's still a celebrity im his field.

My point being: you really think this situation ever used to be better?