That's true!
635 karma · joined February 25, 2016
I think we all know the game has changed. The question would be whether the CEO is handling that effectively. On that score I don't think he is. It sounds like a slow-moving mutiny is going on, and to have it blow up in the press means you've already failed.
I'm not sure he's wrong. I'm speculating here but I can imagine the kind of developers Proton attracts--likely people who care a lot about the code and who care about the craft. That's fine, but when you're running a company in this environment, and that's what things look like internally, you do have to be a little brutal to shake people out of old ways. A manifesto titled "Your old job is gone" may actually be warranted.
It sounds a lot like there are people internally who are still fighting the shift, and they've taken to leaking things that hurt the company's security image. Part of the CEO's job, if he's going to push hard in the direction he's decided on, is to fire those people who will undermine it before they do. That this has all now backfired is entirely on him. Maybe he thought he was "motivating" people by calling them losers, or shaking them out of an outmoded way of thinking about what we do, but none of that matters when you have employees attacking your main selling point and the core image of the company in public.
He's not wrong that he needs to move fast, or that processes need to change, or that the job has changed, but his execution isn't so great.
He'll probably survive just fine though.
Now that things are settling a bit after the Navier-Stokes and theorem-dump shocks, something like what mathematics will look like post-AI is starting to emerge. Of course, things should be conditioned on the assumption that capabilities will continue to improve, but imagine something like this--
An institute with a tremendous amount of compute, either rented or owned, built on the Humboldtian model of the unity of teaching and research. The main draw for big names would be the access to compute, free of the fear of being scooped. New forms of attribution and collaboration would develop, and we'd gain some clarity for students and PhDs on what a future career looks like.
The current melange of math university departments all following their own paths through the crisis without consolidating resources makes the transition more difficult. Think of the path physics took beginning with scattered cyclotrons (the basement of the Niels Bohr institute was even dug out to put one in), then the larger labs that followed. That transition was gradual because physics led it from within, whereas with mathematics every university department is suddenly at a massive compute disadvantage with respect to labs that aren't following the same model, and are driven by different incentives.
It could be this simple--maybe we don't need to completely throw away the old model, attribution practices, and so on, and maybe we can preserve the transfer of knowledge to the next generation of mathematicians, if we simply give them lots of compute.
An analogy to physics to highlight how the divergent interests of the labs could be managed in a state of "peaceful coexistence" might be: physicists got well-funded accelerators and labs that followed standard academic practices. These were separate from "war work" (Los Alamos) that followed government priorities, even with the crossover of personnel. You could have the same thing with a "math accelerator" working on the academic model, while the AI labs follow their own priorities, and some subset of people cross between.
It might be objected that this is merely a conservative attempt to preserve the existing academic conventions and practices currently under threat, but tenure itself already does that, and is arguably the very thing that exacerbates the problem for young mathematicians.
Since the Greeks we've had the idea that "Being and thinking are one," or that Being (in the sense of all of existence as such) has some essential unity with thought, and therefore can be thought, and expressed or submitted to the logos or reason. Being is in some sense fundamentally intelligible, and mathematics is the most developed, exacting, and articulate expression of Being.
Logic was understood in this older sense up to roughly the the mid to late 19th century. This is why a work like Hegel's Science of Logic begins not with syllogisms or propositions but with Being and Nothing. But this was forgotten after logic was mathematized by the English around the time of Russell, and its connection to ontology was gradually overshadowed by a focus on epistemology (still, it should be remembered, originally as a means of getting back to ontology).
There may be truth in art, but it's always haunted by its own historicity or contingency, which is to say untruth. Mathematics seems on the contrary the only really timeless, absolute thing we have. Part of what makes it captivating is stumbling on a construction or concept or proposition or theorem that simply must be, independent of us.
The AIs are certainly now more than automatic theorem provers, mechanically traversing some space of true propositions. They are able to push things forward and connect seemingly disparate domains to get to a proof, but to my mind it remains to be seen how well they will be able to form new concepts and definitions.
Imagine the controversy surrounding Cantor, for example, but put an AI in the place of Cantor. If an AI proposed something like the (infinite) hierarchy of infinity, would we have accepted it? What would the intuitionism debates have looked like? Would they even have taken place? And aside from that, has it actually been shown conclusively that an AI could propose such a thing?
There are lots of attempts right now to recover a humanism for mathematics, or restore man's pride of place with respect to it, but maybe we don't need to worry about that. Tao's attempts to preserve the mathematical community, while allowing for practices to change through the crisis may look like a kind of rearguard action, but seems reasonable to me and not really dependent on any kind of humanism. It's a way to avoid the question for now while things play out (and not conservative/reactionary like Scholze and others), which may be exactly what we need, because after all, perhaps we still don't understand why we do mathematics, what it's really for, and what our relation is to it. Whether it's enough to preserve funding is another issue.
Except they multiplied it by 10000, and didn't watch what was happening.
As far as I can tell Nvidia takes a longer-term view on the diffusion and proliferation of hardware and intelligence, and sees the labs' attempts to impose a regulatory structure to save their business models in the short term as contradicting that longer-term view.
No, the incident has been blown way out of proportion by interested parties. They gave a swarm of agents an impossible task in an ExploitGym Benchmark setting, then didn't monitor it even after they discovered the initial breach of Artifactory.
Everything has been fishy, starting from the initial presentation at the blackhat conference, where things were framed like, "we've entered a new world of security," as an accomplishment, rather than what it really was: massive negligence.
They've been raising the issue bi-monthly through mini-scandals that have until now been consistently slapped down by Jensen Huang, but it seems an alliance with Democrats, just prior to an election where they're poised to take power in the Senate and the House, might finally be how they crack their "problem."
It won't be long before a massive incident is blamed on an open model in the wild, not from inside the labs, and none of us will be able to leverage open models to run private business workflows for the cost of electricity and hardware.
It's worth noting as well that if the Democrats imagine they'll get a "slow down" to protect one of their main constituencies, the professional class of credentialed knowledge workers (or however you slice it), they're dreaming--the labs have stated openly again and again that their business model is to capture the 10T TAM that represents the sum of wages of that very class of workers.
This is honestly sleazy, just admit you were basically way off.
> A few reflections on my "LLMs Can’t Jump" paper:
> My position paper recently got some traction here, so I wanted to share a few thoughts and clarify a few things.
> First things first: some people are framing this as "DeepMind is throwing cold water on AI for science" or claiming the paper argues LLMs can never make real scientific discoveries. This is NOT the case.
> This is a personal position paper, not the company's view on AI for science. This is also not my position. As a core contributor to AlphaProof (the first AI system to win an IMO medal), I know firsthand that my colleagues at DeepMind, other frontier labs, and academia have made amazing discoveries with LLMs and will continue to do so. This paper is NOT an "LLMs are a dead end" kind of thing.
> Rather, the paper is the result of a deep dive I took to study the invention of General Relativity. I wanted to explore what it would take for a modern AI system to make that exact kind of jump. Specifically, I focused on the equivalence principle—a key axiom that Einstein formulated through thought experiments grounded in his physical intuition. I was trying to figure out what it would take to give modern AI systems that sort of thinking.
> Giving AI this specific capability isn't necessarily the most urgent thing to do next. It is very likely that improving our current recipes will lead to many exciting discoveries in the near future. In fact, that is what I am personally working on these days (sorry to disappoint you!). It is also quite possible that I am wrong, and that simply scaling our current systems will lead to new inventions in physics and elsewhere.
> Nevertheless, this was my position last winter when I wrote the paper, and I'm sticking to it. I think that there are a few interesting ideas to explore in this space which could influence the next generation of AI systems. I was very lucky to receive a lot of interesting feedback about this position—thank you for all the messages!
This is a good example of what I was pointing to with "fully transforms the division of labor."
I'm personally in a position similar to yours, but as I watch the different moves the labs make, I see the edge we've been handed (for now) also constantly under attack from different angles. This is why it seems to me that so many who can make the most of things for the moment also feel the clock is ticking.
The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process."
That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place.
I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor.
One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
The labs have to be told NO--the problem they're dealing with, that model outputs give the game away, and that in turn the distiller becomes the distilled, is a fundamental problem they have to figure out how to deal with without going to the State.
It's just a form of rhetoric.
To dismiss them, saying, "ah ha! they're making the same error as all failed prophets!" misses the point.
What they are really talking about is AI governance.