danielmarkbruce

Alphafold solved a waayyy narrower problem that had lots of experimental structural data. Guessing how a drug interacts with an entire human is a whole different scale of thing. Even knowing protein interactions wouldn't tell you how those interactions propagate through an entire human system or whether a drug will be safe and/or effective.

Drug trials in humans.

Who said anything about easy and fun?

It's quite a leap to go from the study you linked to thinking AI will create models that will be useful for new trials. The big leaps in AI have required enormous amounts of data. There is no such analogous data sets in biology and no foreseeable way to get it.

It really is hard to see AI improving the trial process meaningfully. We are more likely to see progress with better trial design and regulatory regimes.

It sounds lazy tbh. No one said privatize. It can be open source can't it?

This is a largely agreed upon problem. What isn't clear is why it seems to get sidestepped in discussion so frequently. No one knows how a molecule will interact with human biology. It can't be simulated computationally to generate data (too slow), there isn't a lot of data in the grand scheme of things, so you can't train a model to actually predict the interaction of a molecule with a human.

And nothing I've said here is an original thought or piece of insight.

why not just do it?

Under examined? No. This is a very common talking point.

A glut of models doesn't mean there is no moat. There are several dimensions on which to compete, and it might turn out that being able to produce models that optimize on different dimensions is a difficult and valuable thing to do. It also might not, but the "there is obviously no moat" only comes from people who don't use these things systematically.

Lol, it's a bad deal huh. Tell that to the guys who bought the IPO of Facebook, Google, Amazon, MSFT, Nvidia.

Just because something might be not great in aggregate, it doesn't mean any single IPO is a bad idea. Some people have insight into the future of a specific industry/business. If you can pick one, good for you.

People said similar things about facebook, and similar things about Nvidia stock recently (not ipo, but similar "why buy at this price"). Global GDP is over $100 trillion every year, there is a lot of business to do in the world, AI might be able to do a lot, it might be very profitable.

The number might be too high, but doesn't take more than a little imagination and some basic facts about the world to see how it could be worth 20 trillion in 10 years.

The comment below/above is right. Pre-training isn't the path (but you really do need to learn how it's done). Learn how it works, learn the architecture, then take models and do post training. Fine tuning and RL on them. There was a fun/clever/useless project recently posted here about building a terminal emulating model thing, as an example.

You don't need to code to follow it, just read along

You are completely off base. His initial book was released almost two years ago. I post train models, for work, and I learned more from his book on LLMs than any other source on LLMs. There is a lot more to it than that book, but you have to start somewhere, even if you have a decent theoretical foundation. There are a bunch of people who learned how to build them from his book.

How do you think people learn to build/train models? It comes down from the sky?

You can take any open source model and continue training it. It's nothing like building a CPU.

Get Sebastian's book and build an LLM from scratch. The problem with most textbooks (on any subject, it's not unique to ML/AI) is they usually try to build the theoretical foundations first. In many cases things are better taught top down.

There are some cases where that is still the case - medical diagnosis where you have a lot of text information that you can pass in can be pretty mindblowing with LLMs. If it's text inputs, it's going to be pretty darn good with an LLM. If not, probably not.

This seems silly. It's analogous to an electricity market where every megawatt is a different voltage, in a different city with no transmission lines... might stop working halfway through the job, and only works properly if you buy another 500 megawatts from the same supplier... good luck getting this market off the ground.

Correct, latency is not critical.

Yes, it can get expensive. We do something in financial markets where the cost of a single end to end task could be $100. That doesn't mean cost doesn't matter though, it matters a lot - if we screw up and it propagates, it's money down the toilet. And if we could get the task down to $20, it would make a difference.

There are all manner of documented architectures in papers and books, I don't know that there is anything new under the sun at this point, the devil is always in the details. That said, some of the things we use off the top of my head: generate multiple responses using different models and have a third model judge the best answer; generate an answer and have another model critique the answer and then a third take the answer and critique and generate a new answer; multi-model debate; iterate in a loop either x times or until some condition is satisfied; pipelines of specific single calls to check for specific conditions.

I'd say in general, with respect to errors/low quality responses etc you have to be extremely clear about what you are doing because once you get into the weeds, what is and is not an error is very grey in many cases.

To answer your question - we use cheaper models to just flag if something looks obviously flawed. That covers the vast majority of things because it's the propagation of obviously stupid stuff that causes a problem for our system. To the extent we are doing something sophisticated, we use an appropriate architecture, like it's built into the system.

Copper is a valuable metal with many uses, and it can't be printed. You can make diamonds, but not copper.

However, if you made half the copper disappear, you'd have a big problem.

What's crazy about it is if that ship sunk, as long as nobody knew, it wouldn't matter one bit. Shows just how much of a waste gold mining is.

Sure. I guess the reason some people don't feel so bad for the math folks is the people in those careers you highlight tend to not have the asset a math researcher has - a giant brain that is valuable outside their little world of math proofs.

It's a bad analogy. Math proofs lend themselves to just throwing more compute at the problem both in training and at inference, you can verify when you are done. Most tasks don't lend themselves well to training or inference.

Robotics isn't really a coherent grouping. Some things that would be termed "robotics" are already "solved", other things aren't even close and need actuating and sensing capabilities that are miles off or so cost prohibitive that it doesn't make any sense.

Perhaps. But as someone who uses AI to automate something that seems in the bucket of things you are talking about - it's harder than it seems, the edge cases are endless, claude code is worse at building real systems than people think and humans are doing a lot more than people realize once you start trying to automate.

All I'm saying is the math advances of late really hasn't changed much outside of math as far as AI is concerned.

We are miles away from that.

Math proofs are right in the sweet spot for training LLMs with RLVR. It isn't some general "LLMs are getting insanely smart!". Even most software development itself is miles away from being as well suited to training as math proofs are.

The majority of math research is akin to professional chess. Completely useless and the folks doing it are using their giant brains to play a game that no one outside themselves benefits from.

Fair enough. It just makes him and the foundation sound so soulless. We need more folks like him, not less.

Read "the man who solved the market". It's a remarkable story. He was a remarkable dude, extremely smart, has given a lot of money to promote math and science.

Respectfully, you didn't answer either. Jim Simons was a brilliant mathematician, has done a lot for math and people involved in math. If you assume they are "pro math and pro math people", the why is simple too.

I see you saying a lot of intelligent things - do you really not know much about the Simons Foundation?