Science Is Getting Harder
mattsclancy.substack.com
mattsclancy.substack.com
Psychology is sorta the opposite: there is no limit to the number of experiments you can run on people or possible associations between causes and effects. It's not like psychology , literature, philosophy, or history has gotten harder over the past century, unlike math, physics, or economics. Sure, there are more advanced statistical methods, but running experiments hasn't gotten harder. This is also why the vast majority of physicists and mathematicians are teachers rather than researchers, and why econ papers have gotten much longer and are full of dense stats methods. There are always going to be new discoveries in biology and medicine, and same for applied math and applied physics, such as engineering or astronomy, but theoretical math and probably also theoretical physics are as saturated as can be.
This hasn't been my experience in fluid dynamics. There are so many places where one could apply a standard approach that people haven't. It takes some effort and experience to recognize that, but it's still true in my experience.
I think what you described is probably true for sexier areas like high-energy physics, but not true for more "boring" areas like turbulent flows. Yes, turbulence is "unsolved", but that doesn't mean that useful information couldn't come from applying standard approaches to particular turbulent flows. The variety of industrially-relevant flows is quite large.
Although less famous I would argue that prediction was just as wrong as "640kB should be enough for everyone", lots of extremely fundamental discoveries had been made since then, and we don't seem to significantly closer to a theory of everything.
At any rate, that is how I understood him. Not that we're nearly there, but that our current path of discovery seems finite.
However, as you note, in Applied Maths there are more unsolved problems than you could address with a million researchers.
A physicist or pure Mather social looks at Navier-Stokes and says, “Oh, we know how that works, nothing to do there I guess”, whereas the applied mathematician looks at it and goes, “holy cow, this could keep my entire department busy for the rest of our natural lives”.
I don't know if that's the best example, it's one of the Millennium Prize problems to prove that smooth solutions always exist to the Navier-Stokes equations. Pure mathematicians do, by and large, still consider proving the existence of things to be "something".
Maybe I'm misunderstanding what you're saying here.
>> no matter what problem you can think of, either it has already been solved to the highest level of abstraction, or it's an unsolved and famous problem, or not worthwhile/trivial. <<
So, Millennium Prize is an unsolved and famous problem like Reimann Hypothesis, etc..
If it works well, you get to push clinical science forward.
Then about half the time, "This seems like a boring classification problem" turns out to be way harder than expected.
My guess is this slowdown is happening for a few reasons:
1. Increased number of researchers = increased team size = less innovation. The article shows increase in team size but doesn't ponder the implications. Teams shy away from bold ideas, in my experience. If you want innovation it has to come from individuals empowered to work alone and recruit slowly. The moment you're put in a team situation you are suddenly expected to pitch and convince others to take a risk on an idea that perhaps you aren't even sure about yourself yet, which is a high bar to meet. And the team won't want to try it because if it works the glory will associate with the individual who came up with the idea and drove it forwards, leaving the others in the shade. So teamwork puts pressure on people to propose 'safe' ideas that were found outside the group, which nobody will object to and which everyone can share equally.
NB: non tech firms struggle to create new tech partly for this reason. They have a culture of creating so-called innovation teams. This practice is rampant in finance for example. I never saw an innovation team do anything truly surprising. You could always guess up front what topics they'd be "researching" before learning anything about them because the range of topics was so narrow.
2. State subsidies. We know these kill worker efficiency. If that weren't true the USSR would never have fallen behind the USA in terms of wealth. What the article refers to as science is really academia, and academia is dominated by ever increasing amounts of government money. Whilst the article phrases this as science getting "harder" it can also be seen as researchers simply becoming less efficient than they were in the past, which is exactly what we'd expect to happen given that academia is a parallel planned economy. Efficient here means in terms of discovery production not paper production, of course.
3. Falling paper quality. Another way to view (2). I feel like half my HN comments are about this problem these days but the quality of papers in some research fields is staggeringly low, sometimes junk quality. As in, you could throw out 90%+ of the papers and the field would get better not worse. In a few fields like "social bot" research or epidemiology I'd struggle to name any recent papers that weren't intellectually fraudulent in some way. If you join a field as a researcher because you feel like it's an important topic, and then discover that the papers published in the last 10-20 years are much more likely to be non-replicable or have nonsense methodologies than those published 50 years ago, then you'll probably end up reading older papers because you feel you get more out of them. Then you'll end up citing them more often as a result.
I definitely feel I saw this when reading the epidemiology literature. Papers from the 1950-1990 period were quite different to modern papers. Way less fancy maths, much easier to read, more obvious and logical questions being asked and no WTF moments. You definitely got a feeling that the authors were intellectually curious and wanted to understand epidemics. From 2000 onwards the papers became nearly always useless.
A big part of this is the post-2000s era explosion in the use of advanced statistical methods and, especially, the acceptance of unvalidated models as "science". The creation of free tools like R and STAN made it much easier and so many papers now are just people playing around with R and random arbitrary datasets. They plot some regressions and publish a paper. Unvalidated modelling seems to have destroyed a lot of fields, because to people who are rewarded for publishing it's basically crack cocaine. If your papers have to describe factual things about reality then you're limited in how much you can write about by cost of experimentation, difficulty of discovering new things etc. If your papers can describe arbitrary scenarios invented in a computer with no care given to validity, then those caps are removed and you can publish an infinite number of papers. Yet the actual value of such papers can be zero or lower.
So yeah. He shows graphs of discovery fall in exactly the same way across every single field he looks at, whilst output grows exponentially, and then concludes that "science" in general is getting harder. But then that's clearly not been true of some fields like AI lately. It seems more likely that such a strongly correlated trend is caused by structural issues in academia rather than a true general effect.
This isn't actually true.
"For the first time in the post–World War II era, the federal government no longer funds a majority of the basic research carried out in the United States. Data from ongoing surveys by the National Science Foundation (NSF) show that federal agencies provided only 44% of the $86 billion spent on basic research in 2015. The federal share, which topped 70% throughout the 1960s and '70s, stood at 61% as recently as 2004 before falling below 50% in 2013."
https://www.science.org/content/article/data-check-us-govern...
https://www.science.org/content/article/massive-2021-us-spen...
"Small boost this year completes 4 years of substantial growth despite Trump ... NIH's budget now stands at $42.9 billion, a 33% rise over its 2016 level of $32.3 billion. Similarly, spending by DOE science tops $7 billion, compared with $5.4 billion in 2017, a boost of 30%. NASA science programs rose by 8% and 11% in 2018 and 2019, respectively, before slowing in 2020 and 2021. NSF's budget, now nearly $8.5 billion, has grown the least among the four biggest federal science agencies. But even so, a 14% rise since 2017 compares favorably with an overall increase of only 4% during the second term of former President Barack Obama."
So these are massive spending increases, regardless of what industry is doing.
There's something else we need to watch out for here. Corporate research isn't famous for its high production of papers, but the evidence that "science" in general is getting harder is based on paper-related metrics. It's possible that science hasn't really been getting harder, but that corporate R&D has been sucking the best science out of the academic realm into a place where fewer papers are published overall, so there's less to cite. The fact that the most cited paper of all time is an obscure biology paper and that pharma spending is such a large component of "basic science", is suggestive of this.
The USSR never did fall behind the USA in terms of wealth; it started very far behind when it came into existence in the ashes of czarist Russia, and caught up quite a bit through one of the most rapid industrializations the world has seen. Even after the initial phase of industrialization, from the 1950s throughout early 1980s, the Soviet economy grew faster than the US economy.
At any rate, a full discussion of this might be off-topic, but let's say that it's a pretty uncontroversial idea in economics that a state run economy (like science) tends to be less efficient and produce lower quality outputs
For example, the current infant formula crisis was entirely the result of government management of production, distribution, and pricing of it. But the suggested fix is to have the government launch a criminal investigation of Abbot.
I did not experience this in my 8 years in academic math (PhD + postdoc). Maybe it differs a lot from field to field, but mine was positively teeming with unsolved problems. More importantly, it's teeming with paths that will clearly lead to more unsolved problems (of all difficulty and importance levels).
In fact, the thing I miss the most from academia is probably helping new students blaze some of those new paths. Both for their sake, and as a way for me to discover new problems for myself.
More scientist will mean that overall we have more time to spend in the dusty cobweb filled corners that have yet to be studied in fine detail. And while many of those corners contain nothing interesting. If even one of them contains something truly undiscovered it's worth it right?
I don’t think this system will last for too long. The environment is increasingly filled with status hungry people. Balaji has a good prediction on that.
>I don’t think this system will last for too long. Balaji has a good prediction on that.
link?
He was asked about his views on academia vs industry, having been in both himself. He said capable people in academia will gradually realize that it’s better to start companies or do the same work in industry, rather than writing these grants.
Unless the CEO actually was one of the inventors, that is grounds to invalidate the patents and there may be other consequences.
Really compared to the shenanigans with Newton, Leibnitz or Hooke and Newton, much of what we see today is quite harmless. Or take for a more recent example Teller and Oppenheimer. I think in science (but not only in science) we tend to idealise the old days and scientists, because we remember the discoveries not the politics and their personal flaws. I would even argue that this greatly demonstrates the success of the scientific process. In the end the best science won, despite all the political infighting.
Even looking at the peer review process: it was much more informal in the distant past, and maybe more akin to what happens today with invited papers in my experience, or with small open source software projects.
There were certaintly very petty but intense personal squabbles in the past but that's a bit different from the pervasive structural issues in science today.
At some level too, I don't care what it was like 100 years ago. I can still see the problems today.
Watson and Crick's famous 1953 paper where they presented the double helix of DNA is one such example.
I wish things like the research in this blog post got more attention, because it's important in understanding how things have and have not changed.
To some extent it depends on the weakness of humans for status and competition, because they are powerful motivators to drive advances.
I personally left science because I could see it had 98% overlap with what I'd do as an engineer (or entrepreneur, in the case of a PI), with a tiny fraction of the monetary benefit. Being a winner in that system is a great life, but it rests upon a huge pyramid of people who are working their butts off at low pay and lower prestige, for a brass ring that few will ever grasp.
I said exactly the same thing when I first left academia for tech, and people would look at me suspiciously and ask why I'd ever do such a thing. I looked at every PI I knew, and saw a task essentially identical to running a startup, but with no financial upside -- and the downside risk of being jobless and adrift if you don't get tenure.
I am a PI.
What I spent my day doing was dealing with managerial tasks, fiddly stuff for funding applications, etc.
I got to actually code for a whole day last week, and it was marvelous. It was also only possible because it was an all-hands-on-deck, everything else can wait situation.
Politics, of course, is part of the humanity, but it manifests differently in different environments (academia vs industry, or then vs now). There is undoubtedly politics in industry, however, there are also products and reality checks. The companies won’t survive if they operate entirely based on politics. In academia, on the other hand, the environmental feedback can be weaker depending on the subject; there, politics could come to the fore. Amusingly, Kissinger said, politics in academia is specially vicious precisely because the stakes are so low.
As for then versus now, there used to be a lot of low-hanging fruit that could be taken until 1980s, and there weren’t many researchers. But now there are fewer accessible opportunities, with far more researchers. If you consider that the size of the cake has not increased proportionally (due to factors mentioned in the article), while the number of players has increased substantially, you may conclude that the environment has become much more competitive. Increased competition indicates increased politics, emphasis on selling, networking, and other non-scientific factors relevant to industry, except there you actually get paid. The conditions are nowadays totally different. There all sorts of performance metrics (H-index, citations, number of papers etc), that have become target, in place of good science, which are gamed.
Injecting a ton of money creates perverse incentives and attracts people who are not interested pursuing something out of curiosity and interest, but for the money.
I think it probably creates a hostile environment for people who are there out of pure interest, because if there is a way to earn more money/status/power/whatever by doing something other than good work people will do it. So you effectively get punished for doing what you're supposed to instead of playing the game.
I think the "professional money extractor" explanation here is an apt description: https://news.ycombinator.com/item?id=18003253.
If you only give money and positions to people with "passion" when there is a clear demand for more, that's called exploitation. Maybe consider becoming an aerospace investor instead, that field is full of talent chasing after lost dreams for pennies. If you are a software engineer on this forum, you know damn well what your net worth is in the current North American market compared to the other legacy engineering fields. It is patronizing to expect the other scientific fields to be worth "less" than CS.
That's just an excuse to exploit people. The same nonsense as with teachers. "We want teachers that are passionate, not there just for the money". Bah.
In the end we're trying to get stuff done, and it doesn't really matter whether the pipes are kept clean by somebody really passionate for sanitation, or somebody who just thinks the pay is good enough to tolerate the smell.
The other irony is that Teller himself was part (came out) of one of the biggest counter examples. The Los Alamos project pushed unprecedented funding into science, and it let to many discoveries that would have taken decades longer otherwise.
Salaries for PhDs and PostDocs are a different story of course. They are alright in theory, but often you can only get a part time job, which in reality means a > full time job with a part time salary.
Around 80k €/year as W2 professor base salary
Post-tax 55k €
Guess that might be considered incredibly well paid in Germany
I can only speak from personal experience and observations in my environment, but to me this statement sound like a dark, cynical joke.
There is plenty of scientists who would love to work on interesting things, are hired as scientists, but once they are hired, no one actually lets them work on scientific problems.
1) On number of papers published, citation value, etc. There may be an inflation in the numbers of PhD's granted for little more than serving as a PI's lab tech for a few years. That requires a few publications. Often the publication count is inflated by splitting a paper that would be more coherent as a single publication into several publications to fluff a CV. Such papers may not get many citations. This is about politics and careers, not about the quality of science being done.
2) Universities are in the market trying to patent discoveries and license them to big corporations. This is very bad for basic scientific discovery in many fields, as funding is directed away from those fields. Basic science underlies later applied science applications. Shutting off basic research is killing the goose that laid the golden eggs, but the administrators want those sweet patent license payments more than anything else.
3) People with the potential to do difficult science and math, but who happen to come from a background where they can't afford college without large loans, are much less likely to go into academic research than in prior eras in the USA when college was less expensive. Many would like to do research but the economics of current education systems are incompatible with that goal.
4) Some areas of research are simply not being funded anymore due to pressure from entrenched interests in areas like energy, pharmaceuticals, etc. Renewable energy research is a threat to fossil fuel interests. One-shot cures for diseases like HIV/AIDS are a threat to the profits of pharmaceutical companies selling lifetime maintenance programs. Research into environmental pollution is a threat to a wide variety of interests, as is research into declining biodiversity.
The corporatization and financialization of literally everything is really out of control, and these are the side effects.
In general, I think the paper needs a stronger argument about what a null hypothesis should be and why that is violated.
But most of the graphs can be explained by the sheer number of papers being published.
Academics have the perverse incentive to publish as many papers as possible, instead of discovering as many interesting things as possible, because items published is way easier to measure than real progress made.
Therefore there is a significant paper inflation problem. People might today write 10 papers with LPUs (least publishable unit) with stuff that would have been a single technical report 30 years back.
This is much worse than it sounds, because it affects productivity significantly in subtle ways: it takes effort to publish those 10 papers, it takes effort to read up and review the 10x papers of others, and it takes time to think about LPUs instead of concentrating on the bigger picture.
Unfortunately, due to the way the research system is set up, it is not likely things will improve.
In my field, the latest papers are a bit "click-baity". They have interesting titles with many trending words but when you read them it is hard to find either a coherent vision or a single result that one can build on top of.
Of course, I am generalizing. There is great research being done today and low-quality papers in the past. But it is completely true that the need to publish has distorted the goal of research communication.
There might be a lot of space for applied math/physics, but perhaps this is closer to engineering than research though the distinction is often blurry.
The focus is quite different in private companies and research institutions.
It is not obvious that fundamental research does not “increase our collective wealth of lead to some useful application”. Just not in the short term, usually. Despite a lot of economics theories, companies being directed by their self-interest or that of their shareholders does not lead to good outcomes over the long term. Someone needs to do it, who’s not going to be limited by ROI over the next 5 years. In practical terms, this can only be public funding.
Taxpayer-funded institutions justify it by pointing out that if there are short-term applications, companies tend to already do the R&D themselves, therefore public money is not needed to have good outcomes. The alternatives are:
- research in private institutions with public money (the big pharma model), which has obvious ethical issues when said private institutions make billions off of publicly-funded research.
- public institutions funding short-term research through things like patent licensing, which is quite impopular (there is a strong argument that public institutions’ output should be public; governments are not corporations).
- governments not funding research at all, which quickly leads to countries falling hopelessly behind as you cannot count on corporations to give a fuck about public good.
Overall, it seems like a good idea to let everyone do what they do best: corporations focus on bringing stuff to market and applications developments, whilst institutions with a longer outlook focus on things that could be used in applications 20 to 100 years from now.
Deep learning is beyond what VLSI was for the transistor, for example.
But even STEM research itself wastes too much time with costly self-serving objectives, rather than shooting for breakthroughs that lead to actual applications. Take CERNs large hadron collider, which has produced preciously few new insights, despite costing a fortune in taxpayer money. The Higgs boson is all well and good, but it’s hardly a new finding.
Or take mathematics’ famous millennium problems, where only a subset (eg. P vs NP) would lead to practically useful new insights. By contrast, solving some obscure numbers theory problem would benefit humanity how exactly?
For a few decades, the prevailing school of thought in public administration has focused on justifications. You must always justify how the public will benefit from the proposed use of money. Then you must report that you used the money for justified reasons. The administrators will also audit you to ensure that you didn't use the money for frivolous purposes.
As a consequence, you can't get funding for studying something you find interesting. Instead, you must always justify how your research will benefit the public. If you manage to get funding, you must spend a lot of time reporting how you used that money and what it did enable you to do. Those reporting requirements are also one of the major reasons for the administrative bloat in the academia.
The general public does not want to fund BS research, and science stagnates largely as an unintended consequence of that.
Having a position that doesn't require me to peg everything to a practical deliverable has done wonders for my ability to actually chase down ideas.
Nothing worth doing is easy. We have cleared the low hanging fruit and can now do the actual work. This is so exciting :)
(also I wonder sometimes if as a society we are teaching people less to be critical thinkers, who are not afraid to disagree with the mainstream)
Breathing is pretty easy.
But more seriously, I think what you said only applies in iterations of revolutions. A new breakthrough (e.g. new physics, new invention) could create a lot more low hanging fruit.
I think a major problem is ever since general relativity, all of our (existing and potential) new revolutions are very abstract and hard to relate back to human perception.
Take dark matter/energy for example - if we get a better model to understand that, it will revolutionize how we think about the universe, but it will (hopefully) have less impact on day-to-day society than the nuclear model of atoms did.
>Take dark matter/energy for example - if we get a better model to understand that, it will revolutionize how we think about the universe, but it will (hopefully) have less impact on day-to-day society than the nuclear model of atoms did.
Is unrelated to:
>I would argue that general relativity barely affects humans day to day lives, whereas the nuclear model of the atom completely changed the global political system.
Maybe to help you, I was using General relativity as a time marker (i.e. it was the first sea-change scientific discovery that had limited impact on day-to-day human life)
You then argued that my premise was incorrect, because before general relativity was formalized, people could have believed that all future discoveries were too abstract to affect day-to-day human life.
I then argued that you were correct in that general relativity in fact was too abstract to affect humans' day-to-day life.
Then you made a personal attack against me.
Now we are here.
I just want to state this as context, not as an invitation to tangent into an argument: I think general purpose AI is going to fade from consumer software again, as it has so many times before. But I suspect that some of the tools may find a home in areas where all of the problems are multivariate, and things like advanced techniques in linear algebra can help find signals in the noise when you can't control an environment.
I recall years ago someone discovering that chemo works better on an empty stomach, and not just for the obvious reason of not having anything to throw up. Normal cells in "starvation mode" absorb toxins slower, while many tumors ignore this signal. If we can nail down things like "If you have these genes and your serum vitamin D is > 120 and you fast for >8 hours and ingest 20-40 mg of caffeine an hour before infusion, tumor shrinking is increased by 40%" by mining through mountains of telemetry and then working backward from there to find the causal link.
Also, many major historical scientific breakthroughs have been just a smart (usually rich) dude/dudette with a lot of free time and not a member of the conventional 'academic' system.
So somehow figure out a way for public to get in on the research instead of government restrictions on who can purchase supplies which limits research to the .0001% who have PhD's.
This opens the door to a lot more mind power going into research.
See the book "the structure of scientific revolutions".
Most scientific breakthroughs through out history were just private citizens outside of the conventional academic establishment.
We need to put an end to these government restrictions on buying supplies and equipment which keeps the research down to just a tiny fraction of the population.
Allowing private citizens to do research would put a fire under those old codgers asses.
I have a few published papers in peer-reviewed journals with 30+ citations and I'm a college dropout, but I run a company that can fund/produce this kind of research as part of our mission statement. It's not really restrictions, but rather money/funding, and willingness of private companies to publish publicly (which is a big blocker for most).
It makes sense that science picks “low hanging fruit” early on. Then, on average, later discoveries require more effort.
But the rate of progress depends on both how much effort a discovery takes AND how much effort is available. The progress of society has made it possible to aim exponentially more resources at solving problems. Computers let us automate things. Medicine & farming mean more humans can do higher value work. Better politics means fewer people dying in wars.
So I don’t care if each discovery takes more effort than the last. As long as we get exponentially more resources to go along with it, we can keep creating exponentially more knowledge for a looong time.
The 20th century saw massive increases in knowledge with relativity and then quantum mechanics (ie QM/QFT/QCD). There have been accomplishments in the 21st century but a lot of then (eg the Higgs boson) were confirming things theorized 50+ years ago. Despites decades of trying we're still to find a crack in the Standard Model. Yes we've eliminated a lot of options and that's important in the process of elimination but we don't really seem to be any closer to resolving the fundamental contradictions of QM and GR.
There's a fun list of unsolved problems in physics [1]. My favourite is the so-called vacuum catastrophe [2] where the predicted (from QFT) and actual vacuum energy diverge by roughly 120 orders of magnitude. At the other end of the spectrum is the magnetic moment prediction vs actual [3] that differ by less than 10 parts per billion.
I'm actually losing faith we'll see meaningful progres sin a lot of these areas in my lifetime and I never used to think that.
[1]: https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_p...
[2]: https://en.wikipedia.org/wiki/Cosmological_constant_problem
If I, with my current education, was sent back 200 years... I could probably do OK with inventing stuff (engineering education, so I have some good answers backed by a nice model, much of which I'd have to work backwards to derive because who remembers 200 level engineering classes?). It took Marconi a while to realize that he had to ground the antennas, so there's a pretty good discovery for free.
On the other hand, if I was actually born then... I think I'm a pretty clever guy, but not a genius. Someone like me 200 years ago would probably... I dunno, be trusted to fix mill equipment. Or maybe work on watches, depending on where I was born. Maybe I'd be a town pharmacist or something.
I haven't found a fair way to assess quality of papers - I think it depends highly on individual author motivations. If we are able to identify the right patterns among authors' motivations, it may be possible to come up with a better rating system for papers published. Current rating systems focus too much on individual papers/journals, and too less on authors.
To your other point: I get what you mean, and I don't even disagree, but I could also make the exact opposite point. Papers today often are these big, overly structured manuscripts that actually cointain a multitude of ideas so that it can become hard to see what exactly the point of the paper is. In the past, papers often were much more atomic. They presented one idea with some relevant context. There was no need to prove you had surveyed the entire field just to present an idea. No useless context, more relevance, more focus.
Making matters worse, population count in highly educated countries is shrinking, inevitably leading up to fewer brains trying to solve hard problems than before. For now, I don’t see the remaining countries with a growing population making up for this loss in collective cognitive capacity devoted to science.
Perhaps we shouldn’t fear universal AI as much as we should cheer it on as the future saviour of science.
https://twitter.com/ThomasPHI2/status/1516079802184343560?s=...
I've used as a general theory for why change in the world seems to be slowing down, since although it's stayed constant, it has slowed down on a % basis.
The whole piece is, imo, pure speculation masquerading as analysis.
There is less chance of escaping past mistakes, or more chance that a mistake will ruin you. There are more things you have to stay on top of benefits and finance wise. There are more options for many things, which leads to more analysis of what's best. The systems we use every day, from cars to computers, are more complex.
Somewhat related, but I believe this is the reason why academia will continue to lag behind industry. Academia has a bias for fundamental knowledge, but as information grows, fundamental knowledge represents a smaller proportion of all knowledge[2]. Anecdotally I have observed a 2-5 year gap between innovation in industry vs academia (CS). Academia often far behind looking to monetize the success of industry via newly minted formulaic programs. This was not always the case, e.g. databases came mostly from academia into industry in the 1970s, but nowadays innovation in the field seems industry-driven. To be fair, perhaps academia has already explored much of the relevant concepts, and it's easier to innovate by way of implementing these same ideas in different programming languages and business models.
[1] What are we left with then? More efficient but also less robust systems? Being more efficient, i.e. generating more output, seems to be the chosen trade-off today, when you can afford extra human intervention, but will it ever be the case when we reach an inflexion point and economics favor robustness/stability over output?
[2] Not to mention the issues with defining something as fundamental. Should fundamental knowledge be frozen in time, or instead act a sliding window? I.e. when, if at all, does something go from fundamental to obsolete? and when does something go from innovation to fundamental? Is it merely popularity driven?
When the R&D workforce expands at the rate it has over the last few decades you will invariably see less talented and less talented people entering the field. Also, various forms of overhead will grow with the size of the workforce.
As an example about 20 people attended the 1911 Solvay Conference on Physics. How many attend a conference today?
With the growing size of the R&D workforce also comes the most destructive force in any human endeavor: politics.
Volumes goes up, quality goes down.
Also, an average Earthling is now much richer than before. A random Korean or Turk of 1911 had much smaller chance to study physics, much less money to even buy a ticket to travel to the Solvay Conference. Both is now easier and cheaper.
On the downside, there is a lot more prestigious and well paying jobs today. Instead of tackling physics, best minds of today may be sitting in Facebook, trying to discover yet more ingenious ways to push ads on the rest of us.
I imagine you were being tongue-in-cheek, but just to unreasonably latch onto that statement a bit: politics is certainly infuriating, and it's easy to treat it as some unalloyed Bad Thing that we wish would just go away, but politics is essentially communication, and so it can be a good thing, too. The bigger the group, the more communication you need to keep everything (somewhat) together.
For instance, on a larger "human endeavor" scale, I am grateful that WW3 has not happened yet — thank you politics (:
As with many things, it's a tool that can be used poorly or used well. But the tool itself is not inherently bad or destructive, I don't think. Or maybe it is both inherently good *and* bad, and you can't entirely separate the two.
"Politics" in this context means a form of communication, consensus and decision making in a group.
Perhaps it is impossible to nip talk of the latter type of politics in the bud, though.
My intent in bringing up the issue originally was that when you have larger groups of people interacting, you get larger-scale patterns of communication. People form groups and sub-groups; people give support to groups based on trust of that group or one of its representatives, rather than getting to know everyone individually; higher-level conglomerations of ideals form because tracking every issue in detail is too much, etc.
There's a lot of gunk that comes along with that. But it is not all bad, or destructive. It can even be _con_structive.
I think that generally applies to institutional politics as well as national/world politics.
People don't (usually) get pissed off about idle chitchat. They do when their project is spiked.
Economics is about resource allocation using money and markets. Politics is about resource allocation using... influence? Connections? Coercion?
If we had an open, transparent market where sponsors could bid on research, it might work better. (Hey, HN: Anybody want to set up such a market, funding it by taking a small cut of the bid?)
Of course we can only speculate, but to me it seems self evident that there would be effectively 0 overlap. The issue is that politics invariably turns into a viral team sport where people pick some side, adopt that side's views wholesale, convince themselves they're the most important thing ever, and then set out to convert everybody else to their team, beat the other teams in any way possible, and of course always fanatically cheer on their own team regardless of whether or not its deserved.
And then when things get hot enough, you get violence. Politics is precisely what started WW1. An otherwise irrelevant Archduke was assassinated and then that politically snowballed into everybody killing everybody everywhere, because politics. Then we chose to engage in extreme political myopia imposing absolutely harsh punishments on the losers of the war which, shockingly enough, didn't really lead to them rejoining the "world order" but instead declaring war on it again, and nearly winning.
And the only reason we haven't yet had WW3 is nuclear weapons. War doesn't really work when you can guarantee that your country (and you, for that matter) will most likely not exist at the end, "win" or lose. But now political frenzies are overriding even that most basic aspect of self preservation and inching us closer to WW3 than we've ever been. So no, I do not think he was being tongue in cheek.
I realize "politicking" is often used in a derogatory sense, and perhaps that's how you mean it here, but there is a whole lot of plain-old-politics that takes place on a regular basis which has good overlap with those issues you hinted at. Things like: planning and building infrastructure, making sure people do not go hungry or thirsty and that they have places to live, having common laws, etc. A list kind of like "what have the Romans ever done for us?"[1].
Certainly, politics/politicking can also have an amplifying effect on the bad stuff too, as you mention.
I suppose my main issue with what you say is the bit starting: "politics invariably turns into a viral team sport ..." I don't see it as invariable. The pendulum has been swinging that direction recently, and I don't like it either, but I don't see it as a one-way road to doom.
Though I respect if you do feel that way, of course!
----
Aside:
Your comment led me to read a bit about history, and I found on Wikipedia[2]:
"According to a 2021 study, Franz Ferdinand's absence was key to the breakdown of diplomacy and escalation into war, as Ferdinand had been the most powerful and effective proponent for peace in Vienna."
Like you, I had the "an otherwise irrelevant Archduke" idea in my head, but I guess now I'm not so sure. So, thanks for the indirect new info (:
[1]https://www.youtube.com/watch?v=Qc7HmhrgTuQ
[2]https://en.wikipedia.org/wiki/Assassination_of_Archduke_Fran...
"I’m claiming that science is getting harder, in the sense that it is increasingly challenging to make discoveries that have comparable impact to the ones in the past".
So, unless you're arguing that there aren't an equivalent 20 people alive today as prestigious as the 20 people that went to the 1911 Solvay conference. Then the fact that there are greater percentage of less talented people now, shouldn't make it harder for the greater number of talented people to make "discoveries <of> comparable impact". (With some caveats of course.)
The Solvay conference was invite only, but I think your point still stands.
When was this golden age in the past when the world was magically easier and “better”? I see complaints about social media brainwashing people but what is religion and nation state populism?
I have a hard time taking contemporary opinions on “life is getting harder” seriously or sincerely. It’s just another clickbait trope.
No one wants to tax rich people who are of the age to have benefited from an era of high taxes. We let them pull the ladder up behind them.
There is no real reason to follow their orders or coddle their sensibilities. What are they going to do? Instigate a civil war from their Lay Z Boy? 60-70% of the population want to reverse their political policies across the spectrum of contexts. Just do it.
Maybe just spit balling. Maybe it isn't that science has gotten harder, but science needs more engineering support than what it currently has to develop the technologies to allow more a more rapid discovery.
Also any physics paper you read today is about one of those major discoveries made over past 120 years, except maybe extromagantism or optics