14,641 karma · joined August 5, 2008
gareth full_stop mccaughan at_sign pobox.com
Evidence?
Tufte's "data-ink ratio" is a pretty good heuristic, but e.g. it would probably become a better match for actual visual clarity if what you counted wasn't "amount of ink" but something that measures spatial change in ink. It's the transitions that feel like noise, much more than patches of constant colour. But "data-ink ratio" is easier to say and easier to read than "ratio of integral of norm of gradient of colour".
Also, I think this sort of graph is better with very light gridlines than with no gridlines, though I think Tufte would disagree.
Also also, in Tufte's book, immediately before the section about "data-ink" there is an copy of a graph drawn by Playfair showing a country's balance of trade over time. Tufte gives it as an example of a good informational graphic. You can see it at https://en.wikipedia.org/wiki/William_Playfair#/media/File:P... if you like. This graph uses shading to indicate which direction the balance of trade goes in; that information is completely redundant with the curves by which the shading is bounded; but it would very much not be improved by removing that "redundant" ink even though according to his principles as interpreted by the page linked here that would make a huge improvement in the data-ink ratio.
Physician, heal thyself.
I'm sorry, could you possibly patronize me a little harder? Thanks.
Yes, any fact has significance. No, that doesn't mean that one can get from "you can keep asking why like a toddler" to any deep truths about the nature of reality. Descarte's cogito is a clever bit of sophistry but not actually a valid argument.
(In your other recent comment in this thread you inform me that I am not smarter than Aristotle and Descartes. Probably correct. The advantage I have over Aristotle and Descartes isn't a better brain but centuries of progress in understanding made by countless smart people; it doesn't even particularly matter whether any of them were smarter than Aristotle and/or Descartes.)
[EDITED to add:] Oops, I didn't read carefully enough; your earlier comment was addressed to someone else. I'm fairly sure you intended to include me among "the other reddit atheists", but it wasn't me you were informing and it's technically possible that I wasn't one of the people you were saying it about. Anyway, obviously it doesn't particularly matter; I just don't like leaving something inaccurate uncorrected in something I wrote.
> There is no way to use science to prove "literally nothing came before that".
Quite correct; there isn't. Which is one reason why I never said that there was. What I said was that "literally nothing came before that" might be the answer to one of those questions, by which I meant it might be the correct answer, not that science would be able to prove it.
(Science is never actually in the business of proving things. Every scientific discovery is in principle provisional.)
I think you might be able to save some of the effort you put into being condescending[1], if you refrained from giving everything I say the stupidest possible interpretation.
[1] Of course I'm joking; I am sure being condescending to me is a source of satisfaction to you and you'd rather do it than not. But I like to maintain at least the polite pretence that my interlocutor is arguing in good faith.
> "Everything" being cyclic is explicitly a teaching of Kabbalah
Sure. And of Nietzsche and of the ancient Mayans, if I recall correctly. (I might not; I am not an expert on either Nietzsche or the Mayans.) That has little bearing on how plausible it is that everything actually is cyclic, of course. Anyway, to whatever extent you find cyclic cosmologies more plausible on account of the idea being found in Kabbalah (or indeed in Penrose), you should be a little less impressed by first-cause arguments.
> You're still talking about "everything" i.e. "something" i.e. existence
I assume the point you're getting at here is that even if someone gives a complete account of the temporal and causal structure of everything in the universe it's still possible to say "OK, fine, and where did all that come from, then?". (And so ad infinitum.) It's rather galling to be accused of obtusely failing to recognize this when in fact I explicitly acknowledged it more than once in the comment you're replying to.
> for the sake of discussion let's call this first cause "G-d".
No, let's not. Because giving it that name smuggles in all sorts of completely unjustified implications. If any of these first-cause arguments worked (which, I repeat, they do not) there would be no justification at all for attaching any such label to the thing they purport to find.
Contrary to your earlier claim, no "first cause" argument shows that "God is". At most it shows that "some thing beyond which it isn't useful to keep asking why, is". And, as already observed, in fact it very much doesn't show even that, as evidenced by the fact that anyone can if they please keep asking "why" even if the last answer they were given is "God".
> That there is [...] at least Divinity [...] is a direct logical consequence of existence.
No it isn't.
The sort of infinite-regress argument you describe has, as you say, been around since the time of Aristotle. It's been a bad argument since the time of Aristotle too.
1. Obviously you can keep asking "OK, and what came before that?" or "why?" or whatever. Equally obviously this doesn't prove anything about the ultimate nature of reality, because the mere fact that one can keep asking such a question obviously isn't the kind of thing that could possibly prove anything about the ultimate nature of reality.
It might be a different matter if there were some guarantee that we could sensibly always keep asking such a question, but there isn't. At some point the answer might be "literally nothing came before that" or "for no reason". We might find that unsatisfying but reality is not obliged to satisfy us.
2. You haven't actually made your argument explicit, so let me try to do so. "We can always ask what came before. This can't really go on for ever, so at some point we will have to land on something about which we can no longer ask that, quod dicimus deum." There are several serious problems with this.
(a) It could, in fact, really go on for ever. Maybe there just is a neverending supply of predecessors when we ask "what was before that?" (including when we ask "what was before that whole chain of predecessors?" and "what was before that?" and so on). You might find that unsatisfying but, again, reality is not obliged to satisfy us.
(a') It could go on for ever without being infinite. Maybe everything is cyclic in some sense. Again, arguably unsatisfying but reality is not obliged to satisfy us.
(b) It isn't actually true that if we land on "God" as an answer then we can no longer ask "what came before that?". We might choose not to, but only because there's a tradition of not asking that question or of shutting it down as illegitimate when asked (as e.g. it commonly is by small children when they're told that God made everything). There are religious traditions in which gods were made by or born from earlier gods or earlier proto-gods. The "Abrahamic" tradition is very much not one of those, but just as you can ask "so where did the multiverse come from?" you can ask "so where did God come from?".
(c) There could be a point beyond which there is no answer to "so what came before that?" without that last (or rather first) thing being anything remotely like a god. For instance, maybe (i) when you keep asking "what came before that" literally, referring to time as we experience it, you eventually get to some point that's "earliest" in the same way as the south pole is "southernmost" and there is nothing "earlier" in the universe, and (ii) if you then say "so what came before the universe?" in some broader sense the answer is "nothing did". Then the universe plays the role of "first mover". Once again, you might find this less satisfying than putting a god in that role, but reality is not obliged to satisfy you. Or maybe you might say that in-some-sense-prior to the universe are the laws of physics but those are even less godlike than the universe. (But also less like the sort of thing that needs anything to have "come before". Yet again, you might find it unsatisfying for there to be laws without a lawgiver but yet again reality is not obliged to satisfy you.)
(d) No argument I've ever seen for why a god is a better candidate for "thing we stop asking 'what came before' about" than the universe has seemed any good to me. Maybe you've got a better one than I've seen, but that's not the way I would bet.
> Note that this does not at all describe or inform or prove the particular nature of G-d
(I remark that the paragraph beginning with what I've just quoted was edited into monideas's comment while I was writing this one, without any acknowledgement that anything had been changed. It's only by chance that I happened to see the change.)
That grotesquely understates how little any first-cause argument says about the nature of the thing it allegedly proves that there is. You can call your "first cause" by any name you please, "God" included, but calling something a god doesn't make it one.
(I reiterate that I don't actually think that any first-cause argument manages to establish the existence[1] of anything. But even if somehow it did, it would tell you far far too little about that thing to justify attaching a name like "God" to it.)
[1] I know that you prefer to use "exists" more narrowly and to say that "God is" rather than "God exists". Feel free to substitute "being" or some more complicated phrase; what I mean by "existence" here is "the thing that theistic first-cause arguments are claimed to establish about God".
> by modernity I basically mean post-Enlightenment
Then it is hilariously false to say that in modernity atheism is "the default class of religion for the masses". The great majority of the masses in the "Western" world were (at least in name) Christian at the time of the Enlightenment and at every time since until very recently.
> atheists are so boorish and conceited that they think that they just believe what is true because they deny that assaying metaphysical truth is even a thing you need to do or can do at all.
What, all of them? You don't think it might be possible, e.g., that someone might grow up holding some sort of theistic beliefs, and be more intentional about those beliefs than the people around them and decide that actually there are probably no gods? That some people might arrive at an atheistic position precisely by "assaying metaphysical truth"?
(Or does something only count as "metaphysical truth" if it's not atheistic?)
> in modernity atheism is the default class of religion for the masses.
I'm not sure what you mean by "modernity" to make that true. Something like 60% of US adults describe themselves as Christian. Likewise for people in Europe. No doubt a lot of those people aren't very actively and enthusiastically Christian, but that doesn't seem to me to be what "atheism is the default class of religion" would look like. And the figures were much higher, say, 50 years ago, and if you're reckoning "modernity" to start after then then you've got a very eccentric definition of that term.
The average Wikipedia contributor (even ignoring the possibility that they might be a robot) is not an exceptionally good writer, because very few people are exceptionally good writers, because that's what "exceptionally" means.
You wrote, earlier:
> For the n'th time: the recent successes of AI in mathematics are the result of a brute-force attack. [...] ~100 GPU years. How many human-years were invested in solving the same problem, before they were overtaken in the last few days by an AI?
I don't see how this makes sense except as a comparison between total AI-agent effort and total human effort. And the result of that comparison is not that the AI agents have put dramatically more effort into attacking the problem than human mathematicians have. Any one of those 10k AI agents has spent a lot less time on Navier-Stokes than a human who's been working hard on it has. All of them collectively have spent less time on Navier-Stokes than the mathematical community at large.
The AIs put an amount of agent-time into the problem that's in something like the same ballpark as the amount of human-time the mathematical community has put in, and (I think) in fact clearly quite a bit smaller. The AIs got to the solution, which the humans hadn't. I don't claim that this makes the AIs more impressive than the humans -- it's plausible that humans were already substantially more than half-way there. But it means that the AIs are at least comparable to the humans when it comes to attacking this sort of problem.
I agree that it doesn't seem like OpenAI's models are as good at mathematics as Grigori Perelman. But a year ago they weren't as good at mathematics as me[1], and a year before that they weren't as good at mathematics as a typical university undergraduate, and a year before that they weren't as good at mathematics as a typical secondary school student. (Don't take any of those timings/levels too seriously; I haven't checked exactly how the progress went. But it's something along those lines.) How do you expect things to stand a year or two from now?
[1] I was an IMO silver medallist[2] and have a mathematics PhD from a good university, but I'm in my fifties and necessarily a bit rusty at this sort of thing, and while I spent a couple of years trying to be a pure mathematics researcher I wasn't terribly good at it and escaped to the Real World.
[2] For those who don't know how it works, this doesn't mean "second place"; they give out a lot of medals of each type. Roughly speaking the proportions nothing:bronze:silver:gold are 6:3:2:1, IIRC. I was a fairly mediocre silver medallist, so maybe somewhere around the 20th percentile of IMO team members worldwide.)
> they look at lot like monkeys on typewriters.
It is hard to take your arguments seriously when you say this sort of thing. 10k monkeys on typewriters, typing one letter per second per monkey, would take something like 300,000 (elapsed) years to get as far as typing "NAVIERSTOKES". (I am assuming typewriters with only letters, case-insensitive.)
The AIs made an amount of progress that was at least broadly comparable to the amount of progress humans had made, with a number of agent-hours of work that was not dramatically greater than the amount humans had taken, to complete the solution of the Navier-Stokes thing. So the AIs were, individually, in something like the same class of ability as the humans.
You may be as impressed or unimpressed by that as you please, of course. You may think it unwise to extrapolate superhuman performance in the future, perhaps e.g. on the grounds that the smarter we try to make them the less relevant training data there is from what humans have done before. You may absolutely have moral or legal or political objections to what the AI companies are doing or what you expect them to do in future. But whatever they are, these systems are a long long long way from being monkeys on typewriters now.
I think this might still be in the realm of "maybe about as stochastic-parrot-y as human beings are"; I can easily imagine doing a worse job of remembering relevant stuff that happened to be in English if I had to do my work in some other language. Human memory is surprisingly context-dependent. But it would be interesting to see what happens if you ask an LLM to write code while talking to it in some language that has waaaaay less programming-related stuff on the internet. Swahili, perhaps. Is it much worse than when you prompt it in English, or a little worse, or what?
If the LLMs are very stochastic-parrot-y -- just piecing together bits of code associated with the words you wrote in English-or-Chinese-or-Swahili -- then I would expect them to get catastrophically worse when prompted in a language in which there's very little programming content on the internet. (For what it's worth, I think I also think this degree of stochastic-parrot-ness seems rather incompatible with what they are able to do. But others may disagree.) On the other hand, if they're more like humans -- somewhat better at remembering relevant things when they're in the same language as they're working in, etc., but operating at a conceptual level as well as pushing words around -- then I would expect the loss to be much more moderate.
(It seems somewhat relevant that the insides of a transformer network operate on embedding vectors rather than literal tokens; presumably those embedding vectors are much less language-specific.)
The only thing there that seems like it's relevant is the finding that the models they used had worse results when prompted in Chinese than when prompted in English. But ...
I am fairly sure I would do substantially worse work if I had to do it in French rather than in English, even if the work itself was all mathematics and programming and the like. Maybe that in some sense indicates that I am a stochastic parrot but it clearly doesn't indicate that I'm a stochastic parrot in some way worse than human beings are since I happen to be a human being myself.
... so what am I missing, that makes the models' worse performance in Chinese an indication that they're stochastic parrots in any interesting sense?
(I take it that "they're stochastic parrots just like we are" is not a very interesting sense. I mean, actually it would be quite interesting to understand better how much of human thinking can be reasonably described as stochastic-parroting -- fairly clearly the amount isn't zero -- but I think it would be interesting as a finding in human psychology, not as a fact about LLMs.)
What's the difference between voting against anyone who acts against your interests "on principle", and voting against anyone who acts against your interests "out of self-interest"?
(I am not suggesting that you shouldn't vote self-interestedly; arguably democracy works best when everyone does.)
The problem isn't that you didn't show your work, it's that your work was wrong.
I entirely agree with your calculation that 10k agents for 88 hours is about 100 agent-years if we assume 24/7/365 operation. That's not what I was disagreeing with.
But then you said "How many human-years ...?" followed by estimating not the number of human-years that have gone into the problem but merely the number of years.
You can compare elapsed years for humans (26) and elapsed years for AI systems (about 0.01). You can compare agent-years (about 100) and human-years (26 times the average number of humans working on Navier-Stokes at any given time). Either of those is defensible.
But it makes absolutely no sense at all to compare agent-years for the AIs and elapsed years for the humans. Which is what you did.
If a typical human mathematician works 2000 hours a year (actual human mathematicians generally find that they can't do 8 hours a day of focused hard intellectual work, but I think we should count some of their "percolation time" too) then that's about 6 human-years per mathematician. So to get the same amount of mathematician-work as agent-work the average number of mathematicians you need to have been on the job is about 100/6, or about 16.
So when you wrote
> How many mathematicians have worked on the problem? 10k? Not likely.
the 10k figure was a total irrelevance. The number it would actually have to have been is about 16.
(My earlier "as many as four" ignored the fact that humans don't work 24/7/365, as you point out. But my point is that however you slice it the relevant number is more like four than it is like 10,000.)
My guess, for what it's worth is that that is roughly the order of magnitude of the number of human mathematicians working primarily on things that could be classified as "trying to make progress toward resolving the Navier-Stokes problem" during that time. I wouldn't be surprised if the actual figure were 3x bigger or 3x smaller. It probably depends on how broadly you interpret "trying to make progress toward resolving the Navier-Stokes problem", and one important difference is that all those human mathematicians leave behind them a trail of papers proving things that, whether or not they end up on the path to Navier-Stokes, may turn out to be useful later, whereas if OpenAI's agent swarm proved a lot of useful theorems along the way most of them never got published.
I don't, of course, disagree that it's possible for an AI company to put a lot of AI agents to work on a problem, but I'm not sure how that makes what they can do less impressive. The fact that you can do that has always been a major part of why AI could be such a big deal. "A country of geniuses in a datacentre" is the kind of thing people have said; we aren't quite there yet, but the "country" part is as important as the "geniuses" part.
> Math has very little funding compared to other science domains.
True. But to whatever extent the numbers I've seen are correct, for the whole mathematical community to have spent less on Navier-Stokes than OpenAI did -- even if we value the tokens they spent at something like market rate rather than at what the compute actually costs them (which might be right since any capacity they use internally can't be sold to customers) -- the average number of mathematicians working on Navier-Stokes since 2000 would need to be somewhere around four (depending of course on how well paid they are), and that seems too low to me.
> I feel like this is not the correct way of thinking about it.
It seems to me that if you say "It is absurd to waste time discussing whether it is intelligent or not. It is just an algorithm, we know how it works, and it does exactly what we expect it to do." then this only makes any sense if your "knowing how it works" and "what we expect it to do" enable you to predict what it can and can't do.
(I repeat that I agree that what matters is what it can do, not whether we choose to apply the term "intelligent" to it. But unless I misunderstood you were saying somewhat more than that.)
> Anthropomorphizing LLMs has added so much noise to this discussion.
I think sometimes it helps, sometimes it hurts, and sometimes it's indifferent, because LLMs are like us in some ways and unlike us in some ways. (The same goes for many other things, but LLMs are much more like us in some important ways than any other human-made artefacts.)
We know what calculations it does. We have some hazy idea of some bits of how those calculations lead to something that at least somewhat resembles intelligent behaviour. But that's a far cry from actually knowing how it works.
For instance, suppose you give one of today's frontier models some of those chain-of-cubes rotation puzzles (the sort that infamously men are about 1sd better at than women, statistically speaking). How well will it do? I have absolutely no idea and I'm quite sure that a more detailed understanding of the transformer architecture would not make my guesses any better. (Actually, I do kinda have some guesses but they're based on a vague notion about how the models might be partitioned between vision-y bits and language-y bits, and it's very possible that that notion is out of date.)
> it does exactly what we expect it to do
Were you, let's say 6 months ago, expecting it to resolve one of the Millennium Prize problems?
(I do agree that it is more productive to ask "what can and can't they do?" than "should we classify that as intelligent or not?".)
> for Navier-Stokes they spent in 3 days more money than the whole mathematical community over the last 20 years easily.
Are you sure?
(The numbers I've heard, which I admittedly have no very strong reason to trust, don't seem that way to me.)
I hope this isn't actually news to you, but: There is more than one human. There is even more than one mathematician.
If there happen to have been as many as four humans working on Navier-Stokes at any given time since the year 2000, then that's more human-years applied to the problem than agent-years.
> How many mathematicians have worked on the problem? 10k? Not likely.
You don't get to count the factor of 10k once when working out how many agent-years OpenAI gave to the problem and again when demanding that for parity there would need to have been 10k mathematicians on it.
> And all that's without even considering whether the AI based its proof on carelessly shared work by the humans. Or rather, yes, let's consider that: it totally did.
Let's suppose that indeed what Buckmaster and Alpöge had done was in the model's training data. Well, it didn't enable Buckmaster and Alpöge to solve the problem for Navier-Stokes (they could only do Euler), and it did enable OpenAI's model to do that.
Also: we don't actually know that what they'd done was in the training data; the latest bits of what they'd done that could plausibly have been in the training data were from before when Buckmaster said they progressed from preliminaries ("We worked through the literature and upgraded various preliminary results") to actually making substantial progress on the problem ("This was until about a month ago, when we had real progress"); and from what Buckmaster wrote it sure seems like a lot of the Buckmaster/Alpöge progress was in fact done by LLMs. (E.g., Buckmaster says that he and Alpöge have been working frantically to try to understand the proof for their Euler solution. That sounds to me much more like "an LLM did this thing" than "we figured out all the hard bits and the LLM did nothing more than filling in a few details".)
Buckmaster's own account of things is that all the really clever ideas were those of Córdoba and Martínez-Zoroa. (Which are already out there in the open literature, and there is nothing remotely improper about making use of them.) And my understanding (but, note, I am not an expert on fluid dynamics or PDEs and I could be wrong) is that actually the OpenAI model's construction is quite different from that of C&MZ. On what basis are you confident that "the AI based its proof on" what B&A did?
(For the avoidance of doubt: I am not arguing that what OpenAI did was OK. Even if they actually didn't train at all on any of the Buckmaster/Alpöge chats, it's very much not good professional ethics to hear that someone else is working on something and rush to try to scoop them, and there is absolutely no question that they did that. The question here is how impressed we should be by the model's mathematical prowess.)
> A success rate of 17%?
A success rate of 17% on problems of this difficulty and significance is something that for any human being would be a career-defining triumph.
> We have thousands of problems solved by humans in the last 2k years and how many by AI? A couple dozen? Oooh scary!
That would be a more convincing argument if the AIs, like the humans, had been around and trying to solve those problems for the last 2k years. However, as you might have noticed, the state of the art in AI was rather primitive 2000 years ago.
(Evidently not, since humans hadn't managed to do it even with the entirety of human knowledge available to them.)
First: the construction of the real numbers from (traditionally) the rational numbers by means of "Dedekind cuts" (sometimes called "Dedekind sections"). The idea is that if you're trying to build up the machinery of mathematics from scratch, it's not too hard to go step by step from (say) sets to nonnegative integers to integers to rational numbers, but it's harder to get from there to the real numbers, and Dedekind's idea is to say that e.g. the square root of 2 is the way of chopping the rational numbers into "things less than the square root of 2" and "things greater than the square root of 2".
Second: the construction of the ordinal numbers (a sort of generalization of the notion of "nonnegative integer" that allows the numbers to get very infinite) due to von Neumann: you start off saying that zero "is" the empty set, and then you repeatedly say: the next ordinal "is" the set of all the ordinals you've constructed so far. So, e.g., 1 = {0}, and then 2 = {0,1}, etc. -- but once you've constructed all the nonnegative integers you can then look at {0,1,2,...} and that's a new ordinal typically called ω, and then you can take {0,1,2,...,ω} and call it ω+1, and so on and so forth.
Both of these are special cases of what Conway does: Dedekind's is the case where all the numbers are rational numbers and you don't allow either set to be empty, and von Neumann's is where you _require_ the right-hand set to be empty.
There's a further connection, which I believe is how Conway found these things in the first place: if in the definition of surreal numbers you delete the requirement that everything in L has to be less than everything in R, then what you've got is (more or less) the definition of a position in a two-player game. L is the set of positions one player can move to, R is the set of positions the other player can move to. (I say "more or less" because e.g. in many games you're allowed to repeat positions, and games may have complicated winning conditions or involve chance or whatever.) And there's a whole rather nice thing called "combinatorial game theory" that's all about these, and from that perspective numbers are just one particular kind of (position in a) game. (Specifically, a number is a game in which at no point in the subsequent gameplay can it ever make your position better for you to make a move: you'd always rather pass if you could.)
You can think of the "surreal numbers" as being built up step by step. We start out with no numbers at all, and then we repeatedly do a construction that makes some new numbers.
A surreal number is made from two sets of (pre-existing) surreal numbers. We typically call them L and R, for "left" and "right", and sometimes write it as L|R or {L|R} or something like that. The "left" numbers have to be smaller than the "right" numbers. The resulting number will turn out to be, in a certain sense, the "simplest" number in between all the left numbers and all the right numbers.
Now, as I said, we start out with no numbers at all. It might seem like that gives us no way to proceed, but it does: even given no numbers at all, we can still make a set of numbers, namely the empty set! So we can use that for both L and R, getting ∅|∅. Empty sets on both sides. We call this 0, and it will turn out to behave in the way you'd expect the number 0 to behave.
Now we suddenly have another set available, namely {0}, the set containing only zero. Which means that instead of being able to make one number, maybe we can make four: ∅|∅, ∅|{0}, {0}|∅, {0}|{0}. The first of these we already knew about. The last isn't actually admissible -- remember that the "left" numbers have to be smaller than the "right" numbers, which is "vacuously" true when one of those sets is empty (it means "if you have a number x in the left set, and a number y in the right set, then x<y", and if there are no numbers in the left set or no numbers in the right set then that's trivially true) but isn't true when both sets contain 0 because 0<0 is false.
So actually we get two new numbers: ∅|{0} and {0}|∅. The first fits into what OP calls the gap "between nothing and zero". The second first into what OP calls "the gap between zero and nothing". In both cases, "zero" means a number and "nothing" means a space where we don't yet have any numbers.
The number ∅|{0} is called -1 (it has to lie to the left of 0, and there's no constraint on its left, and -1 is "the simplest number less than 0") and the number {0}|∅ is called +1 (it has to lie to the right of 0, and there's no constraint on its right, and +1 is "the simplest number greater than 0").
I should explicitly acknowledge that I haven't defined what "less than" and "greater than" actually mean for these numbers, nor anything else about how they relate to one another that could possibly justify giving these things the specific names 0, -1, and +1. But there are definitions for "less than" and "greater than" and "plus" and "minus" and so forth, and the whole thing does turn out to work very nicely.
Anyway, once we've got these numbers we have eight possible sets that can go on the left or on the right. The requirement for left-things to be smaller than right-things reduces the possibilities somewhat, and the actual new numbers we get next time around are: ∅|{-1}, which turns out to be -2; {-1}|{0} which turns out to be -1/2; {0}|{+1} which turns out to be +1/2; {+1}|∅ which turns out to be +2. We also get some already-existing numbers in new ways; for instance, {-1}|{+1} is actually equal to 0 ("0 is the simplest number between -1 and +1"). Again, I should explicitly acknowlege that I haven't said anything about how you determine when two of these things are actually equal; again, it does all turn out to work properly.
If you keep going with this construction, you produce all the integers, two at a time, and also all the "dyadic rationals", meaning fractions where the denominator is a power of 2. And then, once you've got all those, at the next stage of construction you abruptly get all the real numbers -- e.g., the square root of 2 is L|R where L = {dyadic rational numbers that are negative or have a square smaller than 2} and R = {dyadic rational numbers that are positive and have a square larger than 2} -- and you also get {0,1,2,3,4,...}|∅, conventionally written as a lower-case Greek letter omega, which is an infinite number, larger than all the integers. (And its negation.) And {0}|{1,1/2,1/3,1/4,...} which is an infinitesimal number, positive but smaller than any ratio of positive integers. And you can then proceed further and construct a vast infinitude of numbers, including all the real numbers (which we've already made) and all of the so-called infinite ordinals (which you can kinda think of as being a sort of "infinite positive integer", though there's more to them than that) and much more, all in a system that lets you do arithmetic and suchlike. It's very elegant, if your brain has been twisted into the mathematician-y shape that finds such things elegant.
The article says "A single firm, Irregular, is responsible for hacking done by all three companies" but I can't see anything in the article that actually justifies this claim. The nearest to that is the sentence immediately after that one: "Anthropic disclosed that Irregular was responsible for creating the tests ...". This is not, in fact, the same thing.
(Especially as, as aesthesia mentions, the article just happens not to mention that by "hacking done by all three companies" it doesn't mean, e.g., the most famous recent examples of such hacking: Irregular wasn't involved in the OpenAI/HuggingFace incident.)
So, so far as I can tell, the story is: OpenAI and Anthropic make AI models. Irregular does AI model evaluations. In some of Irregular's model evaluations, in which supposedly-sandboxed models attempted to break into simulated targets, the models got out of the sandbox and did bad things in the external world.
The article talks about "firms which instruct AI models to commit cyberattacks", which is a very neat bit of dishonest framing. It's true, in a sense, that Irregular instructed the models to commit cyberattacks -- inside their sandbox, against fictitious hosts. It's also true that the models actually did commit cyberattacks (e.g., the Hugging Face incident, though once again the attacks described by the article don't actually include this one). But it's not at all true that Irregular instructed the models to do anything like the bad things they actually did.
The article says '[Anthropic's] later disclosure shows that exactly zero percent of the agents went "rogue"'. Once again, the disclosure does not in fact show that. It shows that one variety of going-rogue could have been prevented by telling the models explicitly "this thing is real, not part of any kind of test, leave it alone". That is not the same thing.
The article claims that 'In the wake of these attacks, Anthropic and Irregular have deployed a swarm of AI Safety influencers paid by Anthropic-connected foundations to distract from their culpability and towards the baseless “rogue agent” theory.' It offers no actual evidence for this.
And the article seems very keen to highlight links between the companies involved and "effective altruism", though it is -- I assume deliberately -- rather vague about whether it's saying "of course we all know that EA is evil, so that shows that these companies connected to EA are evil" or "this incident shows how evil EA is".
The "Effort News" website has a number of other look-at-the-scary-Effective-Altruists stories on it. They also strike me as rather bullshitty.
... And then I look a bit further, and I see that Effort News's "about" page says "It all started when I was experimenting with using AI for financial auditing. I found stories that were crucial to the public’s right to know, including several of the stories now available at /investigations. I knew we had to sprint to the launch and launch a publication, directly applying this technology." and "The scope of what we can investigate has massively expanded, because we can chase 1,000 misses for one hit. But the final product cannot be slop. There’s plenty of slop on the internet. The way to surpass that, and what really matters, is manual curation and review of every finalized story."
Manual curation and review? I think the people behind Effort News are admitting that this is AI-generated "journalism". I expect that one day AI systems will be trustworthy journalists, but I personally am not very convinced that that day has yet come. And I don't see much reason why I should trust Brian Chau, the guy behind Effort News, to be doing everything possible to make his AI systems trustworthy journalists. It looks to me as if maybe they've been given instructions along the lines of "dig up things that make Effective Altruism look bad" for some reason.
(I don't mean to imply that EA is their only target. It's just one that jumped out at me.)
I think there's a bit less to that than meets the eye. Yes, OpenAI's result builds on human work. It's possible that it builds on more human work than OpenAI admitted. But even if we suppose that everything Buckmaster and Alpöge did (which, btw, was itself very heavily LLM-assisted/generated work) was a necessary precursor to what OpenAI released, it's still the case that OpenAI's clankers completed the solution and Buckmaster and Alpöge didn't.
My understanding from what Buckmaster has written about this is that the deep mathematical ideas behind their work (and presumably OpenAI's) are due to Córdoba and Martínez-Zoroa. Those ideas are in the published literature, and human mathematicians and AI systems alike are allowed to use them, and doing so doesn't mean they didn't actually do something impressive. Mathematicians build on one another's work; that's how mathematics progresses and always has been.
It may very well be that OpenAI's announcement has a serious problem of professional ethics, especially as their first version of it didn't even list Córdoba and Martínez-Zoroa in its references. (On the specific question of what if anything they learned from B&A's work before that was published: OpenAI are now claiming that after investigating carefully they are confident that the model was not trained on anything Buckmaster and Alpöge did after early July. B&A had been working on this thing for much longer than that. However, on Buckmaster's account of things it wasn't until mid-August that they got beyond what he calls "preliminary results".)
But! The results of B&A were themselves largely AI-generated. (From Buckmaster's statement: "on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable." That is: the LLMs found the proof, and B&A had to work to understand what the LLMs had done. It's not that humans did the thinking and AIs just did the gruntwork. (Except in so far as one might want to give all the credit for Real Deep Cleverness to C&MZ.)
And! What OpenAI say their model has proved goes well beyond what B&A did.
I don't see any way of slicing this that makes it unreasonable to say (unless it turns out that there's an error in the proof -- unlikely, given that it comes with Lean verification, but there have been misformalizations and Lean bugs in the past and there surely will be in the future) that AIs solved the N-S problem. No, they couldn't have done it without the work of C&MZ, but again: important mathematical work almost always builds on earlier important mathematical work, that's just how it is. Yes, if OpenAI are lying through their teeth their model might have had early access to B&A's ideas -- but it seems like most of the B&A work was actually done by AI systems anyway.
It is (I think -- I am not an expert and in particular I have not so much as looked at OpenAI's publication) reasonable to say that the deepest ideas here came from humans, and that it was already widely expected that the N-S problem would be solved in the not-impossibly-distant future in something like the way it has been. So, sure, what the AIs have done here is much less impressive than if they'd settled the Riemann Hypothesis or (probably even harder) PvNP. But it's still a resolution of a famous mathematical problem that any human mathematician would have been very proud to have achieved.