Hacker News Reader

>There's no guarantee that the phenomenon we're interested in will be expressible in an F=ma manner. Complexity exists.

That's a fair point. But having a good theory doesn't necessarily mean that it's expressible as a succinct equation. The theory of how metabolism works is insanely complicated, but it's a good theory nonetheless, because every part of it plays a functional role, it explains a wide range of phenomena and it's logically compatible with other successful theories. None of that holds for deep learning models.

Good points.

Besides being an overly provocative term, I don't think "post-theory science" is a belligerent to "theory-based" science. It's not logically compatible or incompatible with other theories, because it's not logically based.

It may be contradict theoretical predictions, and that's where empiricism comes in. Long term, if successful, deep learning models of metabolism may be useful or irrelevant to the development of a theoretical understanding.

A lot things in practice are using empirical methods and are not looking at F=ma like representations.

The basic issue with going just empirical is that there is no way of knowing if the relationship actually exists or if what is observed is a symptom of something else. (Or, in the extreme case: random. Some bits in finance have that problem, for example, finding an edge in markets).

Most successful application of ML etc. are not relying in areas where there are no ideas about any relationships, but rather where there are a lot of confounders and the relationships are complex/difficult to formulate.

Sticking with on of the examples, we know that proteins fold in somewhat predictable ways, because we do have theories around the "mechanics" there. So good applications are not strictly in the realm of scientific theory but rather in being able to apply and use such theories in previously inaccessible ways.

> Sticking with on of the examples, we know that proteins fold in somewhat predictable ways, because we do have theories around the "mechanics" there

But we were broadly unable to leverage those theories and knowledge into a predictive system. AlphaFold largely dropped all that (IIUC), and works much better without it.

(I was once a PhD candidate in protein folding, and left in part because I felt the field was unlikely to make any substantive progress. It didn't, for at least 30 years, until NN-based "modelling" started to replace all approaches up till then)

The point is that given a solution to protein folding, we can then compare that solution with current theory and even physically test it. The search space is enormous but individual solutions are physically deterministic.

This is a good application of ML since there is an independent “check” on its work. And it’s hard to call it “the end of theory” when theory is still tightly coupled with the system, just at the back end instead of the front end.

That is not true of many applications of ML, like Facebook, which simply spit out solutions at scale with no way to independently check them. We should not think the same way about all ML systems; there is a lot more to it than just the neural network.

How is theory still involved? You don't use theory to check the predictions AlphaFold makes, you use molecular structure determination (x-ray crystalography, MRI, other techniques). Of course, those involve "theory", but not theories about protein folding.

For me, theory enables this to actually work in the sense that we know that structures are not random, i.e. there is some way to predict them and it is inherent in the composition (hand waving). If there is no theory linking input and output then predictions could just randomly stop working, for example.

This is just simply incorrect.

Theory is not involved in this. We know that a given protein will always (more or less) fold into the same configuration every time, generally on their own. We know this by doing experiments in which the protein is denatured (heated to break internal covalent and some ionic bonds, and then left to anneal (cool down), and then inspected at the atomic level for the resulting structure. They always fold back to the same shape. This is an experimental determination, not a theoretical one.

Nobody was ever successful at devising any comprehensive theory that explained how they did this, though they of course identified many different elements that are likely involved in some way (hydrophobicity, local charge, possible compactness and many more).

But there is (currently) no theory that connects protein amino acid sequence to structure, which is why AlphaFold was such a huge deal. It makes the connection (somehow), without any theory builtin, and without any theory to explain.

In fact, there is not even a theory that requires that they always fold into the same shape.

You are misunderstanding me. There is theory about how atoms etc. interact. We cannot use that to predict shape (at present) but at the same time there is no force etc. missing as far as our current understanding is concerned (at most basic level no physics seem to missing). So predictions seem at least possible or rather are not blocked.

On the other hand, trying to use an AI to make predictions which atom of a radioactive element was to decay next would run afoul of current theories.

People have worked on "theory-based" protein structure prediction for 35-40 years. It essentially went nowhere.

It is theoretically possible? Probably. Does it have anything to do with how AlphaFold works? no. Does it have anything to do with verifying AlphaFold's predictions? no.

I never claimed it has anything to do with verifying the predictions. I am saying it allows such predictions.

But it doesn't!

It isn't theory that led us to notice that (almost all) proteins fold naturally into the same shape every time they are denatured and annealed. It was simple experimentation.

The attempts to create a theory of how that happens have been measured by the success of those attempted theories to make successful structure predictions. They have failed (pretty dismally in most cases).

So there is no theory that allows us to make such predictions. We simply have experimental evidence that proteins can adopt their "normal" structure by themselves, essentially every time. We have no functioning theory of how they do this, and no way to use this non-existent theory to make predictions.

You are still misunderstanding me. I am not saying we have a theory of how proteins fold. We have theories on force interactions between atoms. Those theories are pretty complete and do not say that predictions of protein shapes are impossible. We are not blocked from trying, like, for example, in predicting decaying atoms. So trying stuff like AlphaFold is not going against current understanding.

My point is much more basic, if you will. So I meant "allow" as in there is no theory saying that we cannot do it. Not that we have a theory how to do it.

Well, but wasn't AlphaFold at least in part trained based on the existing mechanical models? Or was AlphaFold only trained on ground truth imaging of the folding process?

It was trained on experimentally-determined physical structures. Not predicted models based on various folding engines.

> This has resolved to a lot of frustrating modeling and unfalsifiable theories. P-hacking, replication crisis & such.

It's entirely possible we haven't found such models because of the p-hacking, poor replication and publications with low signal to noise ratios, and not vice versa. Psychology's replication rate was in the mid 30% range. It would be hard to see any general trends with such unreliable data.

You can create a complex system to make predictions about complex systems. But without underlying theory, you can’t put bounds on its accuracy or utility. Such a system will work right up until it doesn’t, with no way to predict in advance where that point is.

We see this with people all the time. Jane thinks she knows how Joe will react to X but Joe does something unexpected.

A system that makes accurate predictions an unpredictable percentage of the time is not that useful. But it might be great for fooling ourselves or others.

Jane being able to predict how Joe will react most of the time turns out to be super useful in real life.

Sure it fails sometimes, but nobody would voluntarily give up that ability.

This is a great comment and what holds a lot of today's AI back is indeed complexity. The only question I would add is whether it's still science (hypothesize, predict and test as stated) if it's not a human performing the expected steps - e.g. is training a neutral network through feedback iterating through these steps (with short cycle times?)

Probably depends on how you define science. If you put science on a spectrum with it on one end and magic on the other (so the spectrum is more of explainable knowledge versus useful but inexplicable rituals), AI definitely moves towards the magic end of the spectrum and maybe into a field commonly called "engineering" for the fact that it bastardizes knowledge with utility.

My question above would definitely not extend to the word engineering (and by association I probably have to withdraw the idea that AIs could do science.)

>> The only question I would add is whether it's still science (hypothesize, predict and test as stated) if it's not a human performing the expected steps

This is possibly just a semantic question. If so, I have no problem calling it something else.

this is my question with a lot of this "science" - how are we ensuring that we're not incorporating researcher biases when setting up and training a new model?

To take a reasonably provocative example: if a new climate model predicted extremely low rates of change, would it be tuned so it showed change in the accepted range? If so, then how useful is this model to science?

> If it's not a human performing the expected steps

Nothing is performing those steps in a NN.

Not sure why you attack economics?

It works really well to eg predict and understand the impact of tariffs or taxes or deregulation.

Yes, economists can't predict recessions in advance. And economics also tells you exactly why: the efficient market hypothesis tells you that if people knew that stock prices were going down next year, prices would already react to that knowledge today.

> Yes, economists can't predict recessions in advance

Aren't they actually pretty good at predicting recessions? Thought the joke is that they predicted 12 of the last 7.

pretty sure the parent is doing the exact opposite. other forms of inquiry aren't valueless because they aren't falsifiable.

cheers. Dead on.

It's not an attack.

OK! I might have misread your comment.