rsfern

I also found your post lacked coherence, but I agree GP’s feedback is delivered a bit harshly

- the way you jump right in is kind of off topic with tons of details, your connecting sentence about Kepler would have been better framing context to lead with (but it also needs revision because it reads like you’re elbowing into the conversation)

- HN is a general (technical) audience, your opening statement and the rest of your post assumes a lot of specialized knowledge. It needs some scaffolding and some distillation for a broad audience to understand what’s important about what you did

- that opening sentence isn’t grammatical? Your solution is faster than some quantum theory of gravity? Or maybe you missed a word and mean it’s more accurate than QED? And it’s a theory of gravity that predicts orbital resonance?

- you’re mixing conceptual and implementation details with the effect of information overload. Is the interesting part the theory of gravitation or the fact that you used Rust and WASM? I would lead with the scientific part, then add a separate sentence on the computational tools you used so they both get the focus they deserve

- the rest of your post is long and kind of alternates between what looks like pasted autogenerated link summaries, jargon-filled descriptions of the research, and some descriptions of visualization tools that don’t clearly connect (to someone without expertise in this field) to the rest of the post

- there’s a flood of crap science being cranked out with agents. Maybe your project is not one of these, but you’re fighting an uphill battle against everyone’s heuristics to filter out the noise. If you want to be engaging you have to distill, clarify, and adapt to your audience

Counterpoint, I’ve been one those surface level emacs users for years and coding agents have been super helpful for me to learn elisp and some emacs internals by hacking on some org mode and elfeed extensions

Doesn’t this mess up the context though? I use ruff and my agents get confused if I edit and run ruff out of band. How do people deal with that? I’m always having to remind agents to re-read files because they’ve changed

How does one develop strong judgement without strong technical knowledge?

I think this is a difficult counterfactual to assess, we didn’t develop clock accurate enough to notice the time dilation until 50 plus years after the theory of relativity was published.

Still, how long would it take someone to work it out empirically enough to have confidence that there wasn’t just some bug somewhere? Maybe they would mothball the GPS project for not working before figuring it out? It’s hard to know, 50 years in an alternate timeline allows for a lot of domino effect

It can still be a good long term investment for society even if individual researchers are primarily motivated more by understanding than application. Arguably it’s the best approach we have because reliably predicting which ideas will be fruitful is super hard

I don’t work in drug discovery, but isn’t this thought experiment skipping a lot of steps? My impression is AI is accelerating a lot of the preclinical screening and design work, but that’s pretty far from establishing “tremendously improved outcomes for patients” isn’t it? That’s still really difficult to verify, are people getting real acceleration of this part of the work with AI?

> if openai had provided really readable proofs with utmost care but still dropped 400 at once, would there have been less outrage?

Less criticism, yes

There’s always going to be outraged people, but outrage isn’t the word I would choose to describe the positions of the mathematicians I’ve read on this topic, including TFA. There’s a lot of optimism mixed with frustration that something important is missing

I think you’re out of touch with the discourse in the mathematical community, it’s quite relevant because clear communication is an important aspect of intelligence

Proofs can be unreadable for more than one reason. Are these ones unreadable because the math is super advanced or because current agents suck at clear writing? Maybe a bit of both?

The progress is clearly not uniform though, it’s shaped by how amenable things are to collecting data and verifying computation. I think writing clear and cogent prose is harder to quantify than formally verifiable logic, and this is why progress in coherent communication has been slower than raw problem solving ability

Maybe this is just a matter of what model developers choose to invest training resources in, but I don’t think it’s inevitable unless clarity is made a higher priority

Are they fully “doing math” in an aligned way though? They are finding mathematical proofs, which is important, but contextualizing results in the prior literature and clearly communicating the approach and implications is just as much part of mathematical research.

You might argue that these aspects of math are less important in the new AI accelerated math world, because agents will inevitably be smarter than humans, but I think clear framing and communication is even more important than before because with this technology we can choose to augment our intelligence instead of defer it

Proof of what? There is a sign error in one of the proofs, OpenAI acknowledged it and withdrew three papers (two relied on the result).

I agree LLM review is also fallible (as is human review) but the interesting part to me is that finding this sign error before publication should have been table stakes for OpenAI, it’s their own model that found the sign error.

I’m curious what was in the original prompt and what was in the prompt that led to finding the sign error, I think it matters a lot for understanding the dynamics here

There’s a somewhat famous lecture [0] by Wigner (one of the greats of 20th century physics if you’re not familiar) on exactly this topic. One of his points is that new tools and ways of thinking developed on the way to solving mathematical problems with no apparent application often find downstream applications in science and engineering. If we’re skipping the part where we identify and understand the new math, will we still reap the unreasonable effectiveness? Tao’s position in this post suggests that the current wave of LLM successes is not conducive to this dynamic

0: https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...

Modeling superconductivity with DFT is tricky, there are plenty of DFT reports from reputable groups explaining why LK-99 should be superconducting. It’s a limitation of the theory, DFT can’t model correlated electron states well, and it’s not great at finite temperature, and both of those are important for superconductivity.

Edit: I somehow missed that this is about magnetic semiconductors (not superconductivity) so DFT is a bit on better footing here. I still think it’s a bit challenging predicting magnetic ordering at elevated temperature, but maybe not as difficult as superconductivity

> a few standard deviations

I’ve been wondering recently if it’s time for someone to revisit this study to see how it holds up with agents https://arxiv.org/abs/2404.10002

I agree that accessibility is a big problem, and take your point about ignoring technology that could help, but if the technology if not there yet then I think more research and discourse is warranted before deploying it widely. As a society we need to decide what kind of requirements we expect of these systems, then agree on standards of performance

Is it ok for models to offer specific mental health interventions, or just provide neutral information? To what degree is that technically achievable? Does the answer depend on the scenario?

Health care workers have mandated reporting responsibilities (in the US, I assume other countries are similar) for some situations. Should models be bound by this responsibility as well? (I think yes but it is a thorny question). How to achieve that while preserving patient privacy for issues that don’t fall under mandated reporting rules?

We also have standards of care for health workers. Benchmarks are good, but that’s similar to a licensing requirement, and we have accountability mechanisms when people don’t adhere to their ethical and professional duties. What should be the liability situation when a model responds out of standard resulting in harm (to the standard of evidence we would hold a human health worker)?

Maybe I am not plugged in to this space enough, but my impression is these questions aren’t really in the foreground

I find this troubling, there are so many potential ethical issues with this application of language models. Maybe it would be more appropriate for models to issue safety refusals and help the user learn how to get help from a licensed mental health professional

I think this is dramatized to the point it’s talking past the article, the math community isn’t really making any of these arguments from what I can tell.

The discourse is (1) models are capable of making really impressive mathematical advances, usefulness is not in dispute, (2) the frontier AI companies aren’t being super transparent about information sources so it’s hard to know exactly how to evaluate the level of capability that was demonstrated, and (3) there are lots of kinds of math that is interesting and there are open questions about how to get there.

In particular this article highlights a particular open question I’ve seen discussed on HN before, which is that the particular proof strategy of finding a counterexample might be more amenable to RL than other strategies of proof that might be needed to resolve the other branches of the Navier Stokes problem (and probably other similar areas of math)

According to OpenAI, but they haven’t exactly been transparent about what information the prompt entailed.

The bigger question is to what extent did expert mathematicians metaprompt the model with fruitful solution strategies through their sessions finding their way into training data. Answering that question definitively is kind of important for understanding the models contribution/capability. But I feel like people want to turn this into a debate about priority and credit which is sort of secondary

I think that’s a lot of risk of anchoring reviewer bias. I’d be more comfortable with a triaged review where the editor’s office uses models to score whether a human editor should evaluate a paper to potentially send out for review, then the editor makes their own assessment, and the reviewers continue to do their job unassisted

For me the convenience of “hey remember this fact” is outweighed by a desire not to get stuck in a search or context bubble.

It might be nice to have better UI to control which bits of history get added to the context of a chat, but then just use a coding harness instead of web UI

It should resonate here. The thesis is that good design principles are transferable between humans and agents. So focus on solving important problems and build well designed tools and documentation to get there, you don’t need special design considerations just because agents

I don’t know about the startup space, but in AI for science people are spending time building MCP wrappers around poorly designed APIs instead of redesigning the API or building an abstraction layer that humans can also use. That seems like a mis-allocation of effort

There’s no reason it has to be like that. We could change course one more time back to a stable and non-partisan science funding landscape and then stick with it if we choose to. Even some of the alarm in this article over the proposed funding cuts isn’t set in stone, the presidential budget request from last year aimed to cut NSF by a similar amount and Congress didn’t go for it

On the contrary, I think the chess comparison is on point. We’re discussing observations that even the strongest models devolve into making invalid moves without scaffolding. For me that raises the question of whether these models are learning the rules and generalizing from them, or of they’re just pattern matching and flailing on this task. Maybe the reality is somewhere in between, but the benchmarks don’t seem to directly measure conceptual generalization, they measure task completion. They can disrupt a lot of people and industries by pattern matching and flailing without being AGI.

I’m sure these models know the rules and can explain them when prompted, but that doesn’t seem to be the way they actually complete this task. Will they get there? Maybe

Maybe it depends on the person? My six year old isn’t great at strategy but they can pretty consistently make valid moves. Sometimes they ask for confirmation on a move which is also not a trait I see in language models (at least unprompted)

the discussion isn’t really about whether language models can become strong chess players though, the point is they seem to struggle to consistently make valid moves. Most humans don’t need to read two books to pick that up, just a couple lines of basic instructions

For numerical code I like einops.reduce more than numpy/pytorch sum reductions because you can reduce over named dimensions. It’s much more readable than having to reason through axis indexing again every time you come back to the code

I agree (and so does Buckmaster based on his written statement) that we are better having solved this.

But I disagree that which humans were credited is the heart of the issue in this particular controversy. The question is what do you need to bring to the table for a result like this. A pre-release frontier model trained on the open literature and $15 million of inference? Or all that plus a year of the experts finding the path to the solution for the model to run with?

I think it makes a huge difference in terms of what we think the future of mathematical research will be like, and whether we should still encourage students to go into this field, which was the original topic of this thread

That’s the prevailing narrative, but I think this controversy calls it into question to some extent. If the OpenAI result wouldn’t have been possible without experts seeding the training data with feedback on promising solution routes, there’s less reason to believe this, IMO. More information and transparency is needed

I think you’re missing an important distinction. “Major damage” to the talent pipeline because models become capable of original end-to-end mathematics is what the community has been discussing. But if the models rely on sniping nearly complete work then this damage is antisocial without a lot of upside, it would be destroying a talent pipeline that would still necessary for continued progress.

Which is it? I don’t think OpenAI is being transparent enough for us to really understand whether these results would have been possible without relying on unpublished information from the solution strategies of the experts