bjornsing

Can you really rent a cluster of 75 H100s for 75*2 USD per hour? Individual H100s, yes. But with sufficient interconnect to run these huge models?

24 hours? Wouldn’t it be better to do shorter bouts of scheduled unavailability so unknowing people’s systems will boot up without manual intervention, but still generate lots of nasty logs / alerts?

In 1998 when I was in high school I dreamt up this kind of power plant and asked my physics teacher where the power really came from. Not happy with his answer I asked a university physics Q&A service [1]. Not happy with their answer either I did a masters in engineering physics and kept asking this question whenever I got the chance. Never got a really good answer. But I’m glad to see it works.

1. https://fragelada.fysik.org/index.php?amne=Energi&stage=&key...

I think it will be hard to expand psychiatry to that level while keeping it professional. The fundamental issue is that people ascribe personality flaws to others instinctually and also have strong feelings around being subjected to such treatment, in a way that they don’t have around sprained ankles. In everyday life it’s called badmouthing or trash-talking. It’s a part of human nature.

If you look into the history of psychiatry I think you’ll find quite a lot of examples when diagnosis and treatment was forced on people who didn’t want it. It’s not hard to find contemporary such examples either.

Thanks for the extensive answer.

> I don't think the idea of a zero null hypothesis is necessarily mathematically unsound. In cases like the difference in means, a zero null hypothesis is well-posed. However, I agree with you that there are better practices, like a null hypothesis incorporating a nonzero effect.

I don’t think a zero null hypothesis is mathematically unsound of course. But I think it is unsound to do one and then look at the effect size as a known quantity. It’s not a known quantity, it’s a point estimate with a lot of uncertainty. The real underlying correlation may well be a lot lower than the point estimate.

And of course it’s hard to get people in charge interested in better hypothesis testing. That testing will result in fewer conclusions being drawn / fewer papers being published. It’s just another symptom of the core issue: it’s quite convenient to be able to buy the conclusions you want with money.

I asked Claude to dream it up instead. It was good.

Seems to be the press release for this article: https://www.nature.com/articles/s41928-025-01422-1

Anyone know where I can read it?

I’m not arguing that there’s something fundamentally wrong with mathematics or the scientific method. I’m arguing that the social norms around how we do science in practice have some serious flaws. Gwern points out one of them. One that IMHO is quite interesting.

EDIT: I also get the feeling that you think it’s okay to do an incorrect hypothesis test (c > 0), as long as you also look at the effect size. I don’t think it is. You need to test the c > 0.3 hypothesis to get a mathematically sound hypothesis test. How many papers do that?

Sure (and of course). But did you see the effect size histogram in the OP?

The problem is basically that you can always buy a significant result with money (large enough N always leads to ”significant” result). That’s a serious issue if you see research as pursuit of truth.

Also, even if you can prove the legitimate origin of the funds you can’t be sure that your bank will accept them. If you can’t find a bank that will accept them then formally you may still have the ”asset” but you can’t really use the money.

Ok. I may have read a bit too much into this paragraph:

> Order the items above, smallest fraction first, whilst taking into account the uncertainty in the number of trials to bound the probability of over-estimating each fraction.

But why mention ordering if you’re not looking for statistical reasoning around the ordering in particular?

The chain rule was explored by Gottfried Wilhelm Leibniz and Isaac Newton in the 17th century. Either of them would have ”invented” backpropagation in an instant. It’s obvious.

The answer looks a bit simplistic compared to the question (as I interpreted it at least). In order to estimate the risk of incorrect ordering you have to calculate P(p2 > p1) where p1 and p2 are drawn from different beta distributions. AFAIK there’s no closed form expression for that probability (so Monte Carlo is one possible approach).

Is there a better principaled approach to #1 than Monte Carlo sampling from beta distributions?

Interesting, thanks!

Reference sounds interesting but I’m getting 404 there.

Why the hell do all those ”renowned institutions” need this (admittedly brilliant) guy to turn their crawl into a usable search engine? What’s wrong with this continent…?

You could train that architecture end-to-end though. You just have to run both models and backprop through both of them in training. Sort of like mixture of experts but with two very different experts.

If you are only capable of blind trust or distrust then there’s a huge difference between human experts and LLMs (and it’s of course wise to put your blind trust in a human, not an LLM). But if you have more of a ”trust no one” or ”trust but verify” mentality in general then it’s not so clear cut. The LLMs have their advantages. For example every chat with an LLM is an independent sample, whereas once a doctor has diagnosed you it’s very hard to get them to consider evidence contradictory to that diagnosis.

To be clear I have nothing against the prevailing psychotherapeutic industry. I’m very thankful for that part of the mental health community. I think it’s rather well-aligned with TikTok too, btw.

> You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved.

Well… On a conceptual level I’d call that a feature branch.

Sometimes it’s exactly the right medicine. :)

I fully agree that the night seems dark. But having recovered from a state that many institutionalized professionals deemed hopeless I have a different perspective.

Have you seen One Flew Over the Cuckoo's Nest? I think it serves as a good model to explain why we can both be right. You’re looking at the average effect of the institution across every minor character that appears in frame and deem it positive, and it was. I’m looking at what happened to Jack Nicholson and see the institution as deeply dysfunctional. If you think about it I think you’ll agree that I’m not wrong about that.

I’m well aware of the risks. But I think you have an unrealistically rosy picture of professionals’ diagnoses.

Or maybe it just feels that way because the world is being fixed at unprecedented rates?

For example I had the misfortune to needing help from the mental health community a decade ago. I sometimes think about how much easier that part of my life would have been if TikTok and Claude had existed back then. It’s easy to forget, but many of the institutions we’ve built over centuries are deeply dysfunctional and very much in need of disruption.

The night often seems darkest just before dawn.

(With that said, I agree there are risks of the kind you describe.)

Is it really true that Google has no feature branches? Sounds unlikely to me. How do you do code reviews for example if everything goes straight to main?

The current state of affairs is concerning. But what really concerns me is how much of the innovation in the infrastructure layer is American. I can browse innovative infra layer startups all day and not come across a single European company. Why is that?

Sure, I can spend my days doing that. But I appreciate the help (from the LLMs). And I think we actually have the same goal function: we want to make naming more compressible, less unexpected. You can call that culture (and it is) but you can also see it as pure information theory.