TBH, I am waiting for Mathocalypse-level progress in biology and medicine, but I guess that's not happening as fast, because it's the real world, human lives, etc., mostly etc.
Biology is the complete opposite of that.
Seeing tiny stuff doesn't magically give you enough info for Claude to reason with the system predictably (the way it can with source code).
There’s no well-labeled dataset for all this because we don’t know everything we are looking at.
Take a neuron. You can stain it so you can tell apart an axon and a dendrite. But if you don’t it’s very possible to miss or mislabel the axon, because unlike the textbook neuron, the axon isn’t always obviously oriented in a way that promotes ID.
And that’s just one example. With more complex samples, you also have a lot of edges which AI models don’t segment very well.
So, I'd say it's a pretty similar process, and yet we have the AI models now almost completely integrated into the SWE domain remediating significant parts of its complexity.
At peak, we had 1-3 million xeon cores running our tasks (protein folding, protein design, drug docking), 24/7. This was before GPUs were really a thing. Each mini-task was basically a rosetta or gromacs or autodock binary running on local inputs and writing local outputs for ~1 hour using ~1GB RAM and no network. Input and output was via colossus, or distributed through a global torrent network (literally all 1-3 million cores formed a single data transfer group, with data passed between clusters at low probability).
I built most of the code of the system (which depended on Google infra) including the scheduler and the machine worker, and also ran the system as on-call "SRE" for 3 years. During that time, we nearly almost didn't take down the Google global network, and had small latency impacts on many services. Managing to run the system for so long without antagonizing ads or search or youtube was a major challenge.
I would say the complexity of the system I built (considering everything down to the OS) was roughly the level of a few human cells in a plate, in terms of interacting components and the complexity of their interactions.
Here are some of the publications that came from running exacycle (we provided strong evidence that using more compute improves outcomes in molecular simulations)
https://pubmed.ncbi.nlm.nih.gov/24345941/ https://pubmed.ncbi.nlm.nih.gov/24265211/
Anyway, my point was not in that I would equate software complexity with the complexity found in biology but that I see the same trend and similar type of challenges that can be found in both fields and therefore I see both fields amenable to the AI advancements whereas many people from the thread do not, and I disagree with that.
I used to work in the MPI world on supercomputer jobs and it was miserable. I also worked on other high concurrency systems at Google (helped run the Ads DB, did training and engineering of Sibyl). I continue to insist these systems are incredibly simple compared to even the most basic human cells interactin g.
THe primary bottleneck for a drug is getting it through clinical trials. No matter how much you speed up lead detection, development, and in vivo studies, the bottleneck will still be clinical trials. Now, if we could make lead detection better at generating molecules that get through clinical trials, that's great- but it would take decades to know for sure. If we could make development less likely to produce toxic molecules that trigger the immune system, sure that's great- but again, we're talking decades before we know if it worked.
Attempting to improve clinical trials- well, those are already fairly well designed and run, but it's likely you could make improvements there too that increased efficacy. But again, you're still just dealing with the FDA, filing tons of paperwork, and tons of people looking at the paperwork. Unless you fix those systematic issues, we're not going to accelerate drug approvals.
That is a bit of a semantic stretch and calling it substantial improvement for a 50-year (?) old computational bottleneck in structural biology is a bit of an understatement too. It has been a true breakthrough solving a problem humanity as of then thought it was impossible to tackle.
But I don’t see how any current AI system will have the curiosity and drive. You need to create conditions for meaningful experiments to be done and a mechanistic way of doing this doesn’t work. The most elegant experiments find ways to cut to the core of the emergent behavior, around all the simultaneous changes happening in different levels of the system from which said behavior arises.
You simply cannot see it all. Not every AI can, we are measurement limited. We can track and measure some dynamics, and the art of the experiment lies in thinking through the connections, then picking the right places to look to catch the system in the act. And you better do it so the failure guides the next experiment, and the next and the next, till you finally have sliced through the complexity and found some key nodes that hold the keys.
And usually after all that, you find your key node is regulated by the key nodes of a few other grad students. The complexity here is in part because the entire system is in constant conversation across levels of organization. The causal flow is looped and is also very sensitive to the environment.
AI has a big role to play, but I doubt it’ll parse it without humans at play. It simply cannot invent a slight modification of a microscope and physically connect the new lens element in the right place, calibrate it, all in time for the biology grad student next door to bring some cells to image that are a mix of 40 days old (no contamination! Magic!) vs 30, 20, and so on, to do a very expensive experiment (in terms of labor and time and rats and materials).
So much can go wrong on so many vectors. Which is also what pushes humans to their creative best. And AI will help a lot. But till you have a robot that can drive, but also do mouse dissections and staining and manipulate a microscope with faulty parts to get a clear image, you’re going to need humans to study biology.
When you invent these super robots that somehow slide through all this physical labor without consuming a whole lot of power, we can talk.
Meanwhile, at the cutting edge, where neither procedure nor mass produced machinery exists to do the whole thing, the reality is, we already use a lot of fancy tech in biology, and can use more, and AI can help keep the focus on the actual physical experiments that take so much care and attention.
AI already can do in-silico simulation
add microscopes and tiny precision actuators, and we can build closed feedback loop in-vitro system
once we have in-vitro, it will be easier to go in-vivo
For instance, let's say we are doing biology on plants, which is still much easier than doing human trials. No matter how motivated claude might get, you aren't getting a corn plant to grow in 40 seconds. Real world research pipelines involve stages where even greenhouses don't do, and sacks of grain are shipped to Hawaii, or Brazil, or wherever, to try to get practical data for a given test.
If your computer simulation is so accurate you can skip a step now done with biology, great, but it gets harder the larger your problem is. I might see if a molecule kills bacteria kind of quickly, but what does a molecule do to a real human being? Not so easy.
Humans are just organic simulation environments for these drug trials.
Despite the improvements, though, we still see plenty of code that is "slop", because it's directed by someone with little idea of what to do, but also because it's harder to test for (or even measure) code quality than correctness. The "solved problem" rhetoric depends very strongly on your standards.
In other fields the problem is similar, and typically worse. Plenty of AI music sounds acceptable, not clearly flawed, and even mildly "interesting" to me; but I think this is much more a commentary on music than it is on AI. Prose continues to be abhorrent in nearly every application; while I've seen examples of people getting really impressive output, it takes a level of effort that defeats the purpose of almost everyone who wants AI to assist their writing (i.e., just generate a comprehensible report on some findings and get on with their day). Art is a whole different world; the quality of results clearly strongly depends on the quality of prompting, it's much better/worse in some genres than others, humans vary widely in their ability to notice "mistakes", etc.
I've always thought openai could actually create an agent that can control various physical robotic arms and things like that, and then continuously iterate on solutions that can participate in the real physical environment. This could accelerate automatic iteration for many things that require human operations, such as complicated experiments, assembly adjustments, and testing.
Otherwise, AI is always trapped in virtual systems and still needs human intervention. But even though AI is already very powerful now, there is still a limit to how much humans can handle or assign tasks. Many times, AI has already finished processing something, but I don't even have time to assign the next deeper level of tasks.
People hope AI can improve productivity, but at the same time they're afraid of AI actually getting involved in reality, while the valuation of the entire AI industry is so high.
This kind of wanting to move forward but not moving forward makes humans especially uncomfortable.
Since sequential steps are slow, I think we're going to see massive increases in parallelization. I'm not sure how the regulatory frameworks will keep up honestly.
Why? This is not an AI problem, it's an industry one. And if it was simple, it would already exist.
Not necessarily. Many things are simply gatekept behind the door.
We see it happening in SWE as we speak - people from and outside the tech who had no or very little experience with programming are now starting to deliver their ideas into products without devs or very few of them. We also see this within the SWE where strong domain experts are now able to deliver ideas into code easier than ever, and by themselves, and for which they would previously need a handful of programming experts. We also see it in other way too, strong programming experts are now able to pick up other domain easier than ever.
This process is something that I believe it will (and hope so) inevitably be picked up by the pharma too so the myth about the complexity will become broken.
Software is utterly trivial compared to the life sciences. The reason is quite simple: reproducibility and explainability.
Let me guess. You have never stepped in a lab.
I wonder what the unlock would be, if any, of having simulations substitute any (or all, in time) of these steps. Akin to the method used to train FSD cars, or robots.-
But I'm no biologist so if someone else is, feel free to correct me.
I wonder if, at some point, AI's will perform to such degree they are able to construct "black box" simulations as needed, that nobody understands BUT that are sufficient to empirically experiment on ...
... much in the same way LLM's function now: In an aggregate sense, they are bloack boxes.-
Process improvements come from failing faster, so you spend less money on your test. Finding the actual answer without testing would also be wonderful, but LLMs might be fast, but they aren't magic. Something like Alphafold saves time in some steps, but it's not just "hand me a candidate", but find completely novel verification mechanisms, which would be just as useful for humans. Either way, parallelization is unlikely to increase.
finally, us