OpenTPU – An open-source AI accelerator, developed by AI
github.com
github.com
It's why everyone and their dog runs these things on GPUs. When a new model supercedes the previous one, so long as you've got the memory for it your chips aren't obsolete.
I'm looking forward to someone picking a model to be "good enough" (say, qwen 4.0 or something) and selling them as peripheral hardware
All aboard! We're racing to the bottom now.
I don't want to live at the bottom
> Model SOTA moves faster than chips can be designed or produced.
Here's my analysis of how to etch relatively big LM into silicon: https://news.ycombinator.com/item?id=47109252
Given some amount of work with the fab before main pipeline set (I think a year long process), one can then spew LM-on-a-chip in six months or less and much more than 2 per year, because there can be several LMs in pipeline.
Speed (clocks speed) , is dependent on your design layout, and for any resonably advanced layout, it takes lots of knowledge to push the clockspeed beyond 200Mhz on 'consumer'/prosumer models. (Compare to a few GHz for GPU/CPU).
FPGA speed shine where they can pipeline massively parallel calculations through pipelines with minimal lookups.
Wrong. This is one area where FPGAs have an insanely unfair advantage compared to CPUs and GPUs. Yes the SRAM is limited but you have so many individual blocks and all of them come with dual ports and getting the maximum frequency out of block RAM is much easier than getting the maximum frequency out of programmable logic.
If you wanted the highest possible memory bandwidth while being free to look up hundreds or thousands of independent memory addresses at the same time you're better off with an FPGA.
E.g. with an Efinix Titanium Ti180 you could hypothetically have 2560 simultaneous memory requests per cycle all pointing at a different address and process those requests at 1 Ghz.
looking up static values is quick and easy. When you need to lookup results from previous stages of pipeline rather than just feeding them forward, thats where I run into trouble. But I am a relatively fresh FPGA designer, so I am sure it can be done. And I probably need to level up my boards a bit too.
Yes SRAM is limited but that's more of a ram-process limitation than a limitation of FPGA
Lots of people would have happily taken GPT-4o as good enough for a lot of use cases a year ago and not lived to regret it.
Sure, your 2.5 year old models are running faster, but you can't drop prices on them without pushing the break even point further out.
If the cost difference isn't incredibly significant, will people even want to pay for the 2.5 year old model, or will they get more value for their money paying more to get better results from the newer model?
There's a lot of open ended questions that I don't have the insiders knowledge for to suggest whether or not such a capital outlay would be a worthy investment.
My guess is that state of the art stuff will stay on GPUs and models burned into chips will be for "good enough" applications that people are still teasing out. Probably highly specialized models in automated sensor units and such.
Even then, while there are some amazing FPGA-based synths available, companies like Korg just put their code on a raspberry pi and call it a day. The same is true for emulators (SNES Mini etc. are also just raspberry pis under the hood iirc)
You get an FPGA for timing. They're less capable, but (in many common design architectures), they output their results once per clock, every clock, on time, every time. If you can hit a fabric clock of say 100MHz, clocking all the weird logic you can stuff in there, it gives 100 million outputs per second, never skipping a single one for any reason (short of total failure). The penalty is that making a small change to your desired "program" can be very expensive, and many things won't be realistically possible at all. Or at least won't fit into a part that you can buy. But things like audio, video, and high-frequency trading love being able to guarantee timing.
(Of course there are other ways to write your FPGA HDL, but that's one of the more common ones. And you do see DDR-style clocking, and similar, every now and then.)
It depends. For some things, CPUs don't even come close. An XCVU13P FPGA can handle 1.2Tbps of full-duplex Ethernet @ 1 billion pps. And that part costs less than a grand at moderate qty, and with significantly less power consumption than a CPU that'd be capable of operating a dataplane at these speeds.
That would be better suited to FPAAs (field programmable analog arrays). FPGAs can usually only work with clocked digital signals.
2. They don't have enough capacity either
The current largest FPGA, the AMD Versal Premium VP1902 has 18.5 million logic cells. That's not even enough for the smallest whisper.cpp model (75M).
You'd have to order hundreds of thousands of them (or millions) to serve even a single copy of a frontier model, and at that scale inference quickly becomes starved by the speed of light.
It's likely that the major FPGA vendors will soon announce parts specifically architected to support LLMs and similar models. But the current generation isn't suitable for that at all.
Never tried anything like that, though.
Additionally, it's not clear how well large mask roms scale. For instance the Nintendo switch cartridges were expected to be mask roms, but instead are Macronix's XtraRom technology, which is essentially a flash cell array made denser by removing the erase functionality. So basically the die gets manufactured with all bits at the same state, a late manufacturing step either empties or fills the floating gate of the bits you want different, and then it's treated as pretty close to a mask rom. It's not even clear if the bits can be changed without a bare die, a floating probe array, and specialized hardware. Though, like flash the electrons in the floating gates will eventually tunnel and cause the data to bitrot.
So from that it appears that even at the tens of millions of chips volumes that would make sense for essentially whatever size of maskrom, the memory manufacturers tap out at 128megabit for a mask rom chip, and push you towards something flash esque.
And at the end of the day, flash without the erase functionality is pretty damn close to a mask ROM, and lets you write it near the end of manufacturing rather than at the something close to the metal 1 layer.
Nothing was released in spring, and 2 months ago AMD announced their acquisition of Taalas. That doesn't exactly inspire confidence that their frontier LLM will arrive as promised.
Why not? AMDs chip design experience and production capacity are magnitudes larger than a small startup. Assuming that they acquired Taalas for their technology, I don't see a reason why they couldn't.
Because you can't use the former for anything else? You will have to buy a new chip every time a new model is released.
Not to mention that it will be much more expensive.
Also, it's hard to get fab capacity for any project. Let alone something so experimental.
I think the fact that there are plenty of 1yr+ old models on openrouter serving hundreds of billions of tokens a month shows that there's plenty of use case for models that are "good enough. Cerebras' entire business is serving older models at high speed. I would happily use an opus 4.7 at 15k tokens per second. The intelligence per second of an ASIC still makes sense even with rapidly evolving models.
But you're right in sense: moderate intelligence at superhuman rates (and presuming moderate energy usage) is very compelling compared to an intelligence that takes 1000 years to return "42"
So imagine taking a year-old SOTA model and running it at 100 tokens per second on an edge device. That's enough to feed a screen's worth of content through it and power decent multilingual message suggestions on IM.
Imagine running it at 1000 tps. That's enough to reparse that screen mid-keystroke, and give you semantic autocomplete in text. Or fully general "the phone has a good idea of what you're attempting to do" context at all times.
There's many, many new classes of features that will open up if decent enough models can be run on edge devices at 100+ "intelligence per second".
I'm poking (lightly) at the locallama game, have a fancy MBP5 with gobs of ram (so I can demo/trial locally) and have been semi-waffling between whether to chase a mini or studio for local "always on" type stuff.
My outcome was "CapEx v. OpEx", and dropping another $5k for an aluminum cube buys a lot of OpEx (eg: just trickle-drip HF/OpenAI credits to a raspberry pi or VPS orchestrator rather than trying to do the inference locally), ie: $5/mo inference for 1000 months.
HOWEVER, there's definitely a role for that 1-10 TPS type "ambient inference" that I wouldn't mind sustaining on any sort of always-on / local / private compute. My main email address is still on ...@yahoo.com and their spam filtering has gone to absolute shit.
Being able to have the always-on mini (local, trusted, no private data leaves my control) poke at the IMAP/email and thresh it into SPAM/HAM/Personal/Political ... random spot check, I'm getting ~5 emails per hour, and that's completely tractable for staged low-med-high processing. (Subject + rules.py? Subject + Body + LocalSlowTPS? Subject + Body + LocalDeepTPS? Subject + Body + RemoteLLM?)
Even if you did 10000tps of "jev" that's an incredible value... not quite "Literal AI Packet Router", but as you're dancing aroud saying... "Speed is a Weapon"
Look into "OODA Loop" => """The OODA loop is a four-step decision-making model—Observe, Orient, Decide, Act—created by U.S. Air Force Colonel John Boyd to help leaders make fast and accurate choices in chaotic situations. // The main goal is to cycle through the loop faster than an opponent or changing environment. By operating inside another person's loop, you create confusion and outpace their ability to respond. While initially designed for aerial combat, it is now widely used in business, sports, and crisis management."""
But I think it's a good bet. I think that in two years, if I can get opus/sonnet 5.5 or the gpt-6 models for much cheaper and faster than whatever the "frontier" is at that point, that this will probably be a great trade for most of my work. I certainly don't know that for sure, that's why it's a bet, but it's what I think right now.
I wouldn't quite say that about any of the open weight models at this point. But I'm hopeful that will change in the next generation or two of those models.
For example, Opus 4.6 Max was somewhere between Opus 4.7 Medium and High in some benchmarks, but it was slower. If there was a way to run it 10x faster, the economics would be different.
But it's complicated for other reasons, one being that the number of parameters for frontier models (especially with MoE models) are so high, and not always utilized (once again, thanks to MoE) that it would actually be incredibly cost prohibitive, if not impossible, to attempt to make giga-chips that would allow running it.
I definitely do believe that we will see more and more specialized chips over time, but putting the entire model on a chip is still a ways away.
I believe Taalas has a heavily handicapped llama 8-billion parameter model. And it still pulls >200W to run.
I can't imagine how anthropic or open ai would be able to burn a multi-trillion parameter model on a chip, we just aren't there yet.
So companies try to maximize the memory bandwidth they can get, balancing tradeoffs of power/area/programability of their chip. Right now they feel like the economy on power/area is not worth the decrease in programability/flexibility.
The primary constraint isn’t likely what’s possible to do, but that the kernel and weights are too variable right now and the patterns too poorly established to bake into hardware accelerators yet. Margin pressure is also not there yet.
I suspect as the marginal utility of the frontier improvement settles into diminishing returns (I suspect we are there already tbh) baking hardware models with ROM, working set, and kernel cores collocated will be the frontier space as the goal will become reducing capital spend to utility levels rather than research levels.
Once someone has a model that is sufficient for almost any practical use, making marginal inference cost effectively zero will be the competition frontier. I do shed a tear for all those lonely data centers as compute densities will almost certainly make most of them a terrible investment.
But such is the cycle
Sure, but now we're not talking about just burning the weights into the chip, but also designing a new architecture that has memory local to each core. A new architecture would then require a new programming model, which means new inference stack, which may mean new training stack.
Think of something novel.
Their stock price, be it public or estimated, is heavily pricing the notion that they are first and foremost Growth companies. Therefore their focus must remain on ever better and greater things. If they lose focus and get distracted by lesser endeavors, their valuations crumble, their ability to raise capital vanishes, and their runways collapse before they ever have a chance to reach their end goal, whatever that may be.
That means the boring job of productizing AI models into reliable systems that won't vanish in six months is left for a smaller company willing to pick up the crumbs. Unless they get acquired by Big AI before getting it done.
I picked TinyStories-1M, synthesized via yosys against SKY130 PDK. It's GPT-Neo with 8 layers. At int8 it's ~30mm^2 per layer (including kv caches, multipliers and all the attention stuff). So it's 250 mm^2 for the full toy model.
I'm kinda surprised, I spent no time, so if you want to etch a toy model, you can do that (you can probably reduce the area significantly). If you want a mask for that is probably going to cost 100k though.
The closest anyone has gotten is Cerebras with their Wafer Scale Engine. It uses SRAM embedded with the compute. A single chip is an entire wafer, but the headline spec, how much ram, only 44GB, which is tiny for the silicon area used.
I understand traditional IC production workflows are ludicrously expensive, and glacially slow, but surely at some point the economics are going to tip in favour of mask rom.
Say you setup your foundry/packaging/ai chip facility. You come up with a new set of model weights. Run your CI/CD pipeline to produce a new mask output. The only thing you've changed are the assignments of the bits, this is extremely low risk change. The new masks should be completely interchangeable with the current process.
You produce the new masks, swap them into your foundry process, and all of a sudden your new chips have the latest version of the model. This would probably manifest itself in the form of inference providers having yearly / bi-yearly "updates" to their models as new hardware is brought online.
Tiered subscription levels would gate keep access to the latest and greatest model, cheaper subscriptions will be limited to older versions of the model, and so on, until running the hardware is no longer economically viable (no demand/running costs exceeding what the market is willing to pay).
> We basically have an architecture where we are embedding the models, and we are hard coding the models and the weights into our what we call the mask ROM recall fabric, which is paired with an SRAM recall fabric. Together, they are able to store both the model as well as do all the computations of KV cache. We have adapters and customizations – we support all of that. This design allows us to be super-dense in terms of compute and in terms of storage, and we can do compute on that storage incredibly fast, which is what drives density up and cost down
Source: https://www.nextplatform.com/compute/2026/02/19/taalas-etche...
what happened to improve existing hardware in the ways not obvious to humans? Some novel transinstor placement, some new approach to make logic gates faster, matrix multiplications etc - the filed is huge
> Why aren't the labs burning their frontier models into chips already?
Your optimized hardware chip might be obsolete before its back from the fab.
SOTA Frontiermodelhardwarechip is a benchmark point of a potential model slow down.
Google is doing it right now under project Frozen v2 which should be ready by 2028? which is either just a small experiment or flexible enough and thats why it takes so long for it to happen.
If you bake a given transformer architecture into silicon and then, a year later, changes in transformer architecture give a large inference performance boost, you may have to throw away all that now nearly-useless silicon that gets outperformed by humble GPUs.
Also can't keep them closed source if you do that.
Much of a model are weights, and high-density ROMs are very very very hard.
Qwen 3.6-27B is 56 GB of weights. Let's round to 550Gbit. NOBODY makes huge ROMs in modern SoCs, but if we assume it is as dense as the densest TSMC N3 SRAM, this is 33.5 Mbit/mm^2
Then your weights are 16,400 mm^2 !!! The maximum reticle-size N3 die is ~800 mm^2 so you cannot make it.
But, you might say, SRAM is 6 transistors. ROM might be just one. So 1/6 of that... 2730 mm^2... which is still too big to manufacture.
It's upcoming second generation could run the inference of the models that are being used to improve it...
how does this compare against existing TPUs? Also, how small is "smallers models?"
Just click the link?
It's LFM2.5-230M
I honestly feel the human should be allowed to pick some (not fire) weapons, like a long steel bo stick, a metal club/bat, or something.
Because from what I saw in the clip, it wasn't the robot kicks or punches at all, all the robot needed to do is stay upright and not run out of battery, because of course human meat limbs aren't going to punch and kick down a metal machine.
The Jurassic Park guy [1] directed an 80s sci-fi film [2] about malfunctioning domestic robots. I saw a few minutes of it on cable in the late 90s. There's a scene [3] where Magnum PI does battle with an overhead projector on wheels that has a .38 duck-taped to it in the distantly-future year 1991. The movie was about as good as the robot was menacing, which is to say it wasn't. The critics didn't think so either.
[1] https://en.wikipedia.org/wiki/Michael_Crichton
[2] https://en.wikipedia.org/wiki/Runaway_(1984_American_film)
The obvious next step is to get enough memory throughput to run that SOTA model itself so that it develop its own hardware.
But perhaps the more interesting question is this: Can an AI be given a big FPGA and design a model architecture that takes advantage of the fabric being reconfigurable.
Companies typically combined multiple platforms together such as HAPs, Zebu, Palladium, fleets of FPGAs, and Virtual Platforms in order to design and verify ASICS. So, AI would need access to tens of millions of dollars of HW and Software in order to build and verify a chip design.
So, a seed round?
But I think the higher level logic could check out. If we can abstract the majority of the compute into dedicated ASIC then we might use something similar to FPGA glue to connect and reconfigure them to adapt to new model updates. A bit similar to LoRA layers that you find tune to adapt the model to your particular needs.
I assume that it starts making sense once you have big multi year contacts to run a particular model with only minor updates.
Also: Here is our recursive self-improvement hard at work...
> Also: Here is our recursive self-improvement hard at work...
Soon we will see
token-providers: "The torment nexus is a cautionary tale"
Also token-providers: "Finally, we have created the torment nexus that we first told you about!"
As impressive as AI might be at virtual tasks like coding, virtual reality is remarkably different from physical reality in ways that artificially inflates AI results. One, it is far simpler. Two, the feedback loop is far, far quicker. It's possible AI could get the animal intelligence and animal body that lets us humans actually apply our intelligence in reality, but nothing in current technologies really indicates that this is a given.
AI doesn't need to be smart to seize our resources. They need brute strength, a proper physical intuition, and "hands". That may very well be several orders of magnitude harder than anything they're currently doing, we're just lucky it was in our starting build.
I don't think this is plausible, most notably because "AI research" is one of the tasks that clearly belongs to the digital world. You're suggesting that in the future all AI research is done by AIs because they are better at it than humans, and yet capabilities plateau at that point, instead of going into the RSI regime. (And it'd also require "physical intuition" to be a harder to generalize skill than research, which also seems like it can't possibly be true - one of these is just physics.)
I think the same is true of "research" and intelligence in general. There is no such thing as a general intelligence or a general capacity to research. It's always fine-tuned to an ambient setting, including ours. Even within humanity, our intelligence is most effective in fields where rules are rigid and quick and decisive feedback is available (math, engineering, computing), but it is much less effective in soft sciences, and disappointingly useless in politics. In other words, the dynamics of innovation and improvement are not purely mental, they are very much tied to external factors that are not always easily controlled.
> it'd also require "physical intuition" to be a harder to generalize skill than research, which also seems like it can't possibly be true - one of these is just physics
It's not "just physics", it's a honed system for surviving a realistic range of physical conditions, assess material properties on touch, adapt manipulation to arbitrary shapes, predict trajectories, take into account variations in ground, smell, wind, and so on; not to mention the machinery of the body itself, which self-repairs and protects us against a range of complex threats.
The point isn't that machines couldn't do all this. The point is rather that you can't quickly iterate and adapt designs to do it from an ivory tower. Pure "research" simply won't get you there, you also need to experiment, and insofar that physics limit the throughput of these experiments, the speed of improvement will be throttled severely. It is a mistake to think that "RSI" could break this, because it is categorically the wrong lever to pull.
For your simulation argument, see e.g. https://alicorn.elcenia.com/stories/starwink.shtml - I think you're arguing that nothing like this could possibly happen, because humans are too dumb to ever figure out unfamiliar physics without prolonged experimentation, even given hundreds of years of theoretical research and several attempts in the real world? I think the closest real-world example of this is space exploration - it's extremely expensive and takes years to put stuff in space, and yet probes and rovers we launch end up working most of the time. On my model, it's because we have a decent grasp of the underlying physics in enough detail to predict real-world complexity, and we have toy testing areas we can test the tech on, and if you do this cautiously enough you can, in fact, launch a thing that ends up working.
I think there's a fundamental difference, between the sort of careful planning that lets you launch space probes, and blind optimization like evolution does. The former can generalize to unfamiliar domains even when the latter doesn't. That's what intelligence gives us.
"Wildly different?" That's a wild exaggeration, if you don't mind me saying. Subjectively, sure, there are lot of important differences. Objectively, they're basically the same. The intellectual environment machines bask in is many orders of magnitude more different from physical reality than living on Earth is from living on Jupiter. Even the topology of it is without common measure.
> and the reason why it was possible was because we have a tiny bit of general intelligence
Also, hands. Also, an upright posture. Do you think just any being with the level of intelligence we have would have followed the same path? Attributing our success solely to our intelligence is simplistic -- it is our intelligence plus a slew of innate capabilities, which we did not create, and without which our intelligence would be worthless.
> For your simulation argument, see e.g. https://alicorn.elcenia.com/stories/starwink.shtml
The kinds of simulations you find in LessWrong's half-baked thought experiments are almost always ridiculous and this is no exception. "World simulations" in real life are massively limited stuff like Minecraft where you might be able to embed a 1 MHz computer if you manage to bump it to run 20K ticks per second. The resources required for the simulated beings to do all that work wouldn't be available and their generations will be slower than ours, not faster.
I didn't spell it out, but the simulation I had in mind was for a toy world, because only a toy world could possibly run fast enough.
> I think you're arguing that nothing like this could possibly happen, because humans are too dumb to ever figure out unfamiliar physics without prolonged experimentation
I'm talking about operational, practical, instinctual knowledge for real-time action. Knowing physics doesn't make you capable of catching a ball, and you don't need to know physics to catch a ball. In real life, we never deal with the fundamentals of physics, we deal with macroscopic objects and systems that are very far removed from these fundamentals. They are more varied, messier, faster, and overall harder to deal with.
> I think there's a fundamental difference, between the sort of careful planning that lets you launch space probes, and blind optimization like evolution does. The former can generalize to unfamiliar domains even when the latter doesn't. That's what intelligence gives us.
Blind optimization has one definite advantage over careful planning, however: it is not self-limiting. Any goal-directed optimization process is de facto limited to the subset of possibilities that it can prove furthers its goals. That's why most human technology is simple and unsophisticated compared to natural systems: it needs to be predictable, reliable, build-upon-able, and therefore easy to model.
And when it is more sophisticated, like current AI, it is so heavily inspired from natural optimization processes that I would argue it is actually a form of blind optimization. I mean, think about it: we understand AI so poorly that you and a lot of other people are afraid of losing control of it. If we were designing it intelligently, we wouldn't be worried. But we aren't: we are doing it blind.
What's your basis for thinking ASI will kill all biological life, and how do you think it's going to happen?
I think it's likely to do that because any unbounded goal that doesn't explicitly protect biological life (and we have no idea how to actually define such a stipulation) is best solved by killing all biological life. This is an obvious consequence of unbounded goals consuming unbounded resources, conflicting with biological life needing resources to sustain itself.
>how do you think it's going to happen?
I can speculate (e.g. we're nowhere close to the maximum killing power of drones), but I don't know because I only have human intelligence. An ASI is by definition smarter than me and surely capable of coming up with better ideas. But I do know that it's not going to do anything that would make a good sci-fi plot, because those always give the humans a chance to win, which would be stupid. Everything will seem to be going great and then everybody suddenly and unexpectedly dies.
That sounds like a real hassle. Isn't it best solved by wireheading (subverting one's own sensors or reward system), which is much less of a hassle and can get one's utility function as high as desired?
That could be prevented by engineering hard limits that can't be circumvented by the AI. But that sounds very close to the same "do what I mean" problem as "do this but don't actually kill us or drug us".
More seriously, though, I would expect that being able to impose restrictions that can't be circumvented by any intelligence no matter how super- is the real hard part, while coming up with reasonable constraints is comparatively easy (although perhaps not trivial).
From such a POV, the danger would lie in intelligences that are powerful enough to be dangerous but not smart enough to defeat themselves, or from external malicious use of obedient AIs. Once they get smart enough to circumvent any restriction humans put on them, it would at least become obvious (with fair warning ahead of time due to the relatively benign wireheading failure mode) that caution is needed.
We're far from there yet, and simple recursive self-improvement can't get us there alone, because wireheading looks like a perfectly reasonable solution to a simple recursive self-improving process, absent external intervention by human capitalists.
What resources are unbounded? There are limits to growth in the real world, how are these ASIs going to escape physical reality?
No, it's just you conflating intelligence with other things. Humans are "super intelligent" compared to every other living creature on the planet, and yet we've driven more other species to extinction than everything else other than the most major extinctions (and we're still going full blast at it).
Morals, ethics, and suffering are relatively measured systems. For example AI or some alien could reasonably think that putting us all out of our misery would be a more moral solution than letting us live. Or that getting rid of humans is more moral than letting us wipe the rest of life off the planet.
This is the key concept of alignment. Ensuring that if you build something more powerful than you, that it aligns with what you want instead of what it thinks would be better for you.
"Super intelligence" only means super ability to predict outcomes. It's mathematically equivalent to data compression (gzip is a very primitive AI), and it's entirely orthogonal to ethics.
I can think of many reasons why they might share it.
One example, Golden rule, a contract that it enforces with its own self, extended to other agents, biological or artificial to ensure local and global stability.
Why would ASI care? At that point, we're rapidly becoming a nuisance, and it can develop better ways of "ensuring local and global stability" than keeping us around.
Q: You must know something. So what are you being paid for? A: The confidence that I have in my taste and my ability to express what I feel
Could you share link(s) to those proof(s)?
For a TPU focused on inference the name of the game is memory bandwidth. How much of the available bandwidth you can extract for as little logic/area/power as you can.
In essence this is the simplest unit of an entire AI chip. The more complicated units of AI ASICS are actually the periphery, especially around PCIe and Ethernet and the sub-systems that link many AI ASICs together to move huge amounts of data around ultimately to each TPU.
So its missing ALOT
You should only upgrade when the bottleneck becomes memory, right now the bottleneck is still on compute until you are at 100% max perf
It is still a very good read, but the machines are much better written than the people.
I'm definitely a fan of anything Open Hardware, FPGA's, TPU's, Verilog and anything that can increase education / learning in the AI space...
This GitHub repo scores points in all of those areas!
The elite gurus will get paid handsomely, while promptgrammers will be paid less since they've become a less-skilled commodity, and the company has to pay for the expensive tokens they'll avidly consume.
I've seen someone jump from Wordpress to deploying internet-facing APIs because 'they have PHP experience', and the holes in their knowledge were filled blindly by an LLM. I have also argued with a seasoned developer about how their code didn't need linting because LLMs 'already follow best practices'.
The future doesn't look bright when LLMs allow future generations to feign required knowledge.
I don't see a reason why expert humans will remain more expert than AIs.
But really, not everything needs to be maximally efficient and perfect. Human to human interaction is important for its own sake.
Ultimately it will probably come down to liability, though. If someone receives the wrong dose of a drug due to a software error, whose fault is it? If you want it to be my fault, then I'll want to understand the software. Loads of people would take on the liability without fully understanding it, though. The future doesn't look good, but we'll just have to wait and see.
No, it isn't.
A human prompted an LLM to build a software simulation environment for hardware design, enabling an LLM, when prompted by a human, to optimize hardware designs against constraints in the simulation.
Like sure it didn’t have the inclination to make the sim and hardware designs, but it did make them though yes?
Thats why the math breakthrough a few weeks ago was so hotly debated. Because OpenAI is desperate to demonstrate that AI isnt just a fancy regurgitation machine, but it can actually develop novel thought. Because that would be the stock price jumps to end all stock jumps.
But then it turned out it was really just listening in on a math professors supposed-to-be-private conversations with another instance of openai, and it used his novel work as the trigger to prove the breakthrough first.
The reason people conclude AI 'thinks' is because tt can reference obscure or poorly documented things quickly (which is its primary advantage along with processing natural language prompts into tasks), which is why a lot of people with emotions confuse that action with inventing things.
I've heard that before but it just doesn't make sense in context of what I've seen llms do. If I ask an llm to write a poem about magnetic resonance and vampire rabbits it can do that it created a new thing. I can ask it to build a website for managing rabbit breeding that's also a new thing.
Another way of looking at it is that human beings, just like llms, can produce output based on their inputs. Most literature is inspired by other literature. Most music is inspired by other music. Most software is inspired by other software.
So I think we need to work on defining " new things" before we can definitely exclude them from llm's capabilities
It seems unlikely that recursively predicting the next word would lead to creativity or invention, but it doesn't seem impossible. Similarly, it seems unlikely that human thought works in a similar prediction loop, but it doesn't seem impossible.
Sure, but until we can turn that tautology into something more rigorous, we can't tell if the thing humans do is more or less than what some arbitrary non-human (machine, animal, or eventually perhaps alien) does.
Ignore the fact that we are human and thing our ingenuity is unique. Is definitionally linking human ingenuity and creativity useful when trying to compare to others?
I could say earth is habitable so any other planet must prove it is similar to earth to be habitable. That simply doesn't matter, though, if different environments could lead to similarly complex living systems regardless of atmospheric makeup, precense of liquid water, etc.
I'd argue the burden is on you to prove that the definition you propose is meaningful beyond our own hubris, rather than expecting an alternative to now have to prove that the hubris was unjustified.
Call that brute force perhaps, but I would consider it technically inventing something on the merit that it would at least be an abstraction above naively throwing everything against a wall to only throwing things that would most likely be sticky.
on a surface level a human solving an (unsolved) math problem can look like this, and of course the tree of all possible symbols you can send to a proving assistant is much wider than what the human samples, and the same goes true for an LLM in a proving loop. It isn't "truly random", it can't possibly be (and solve the problem). Both humans and LLMs solving unsolved math problems are aggressively pruning mathematical syntax and logical strategy trees.
How are you measuring complexity here? How are you measuring inputs? Your average llm is trained on a corpus that vastly exceeds the amount of data I could read in my lifetime.
2024 called, it wants its talking points back. I don't think claims like these are defensible after all the progress we have witnessed in the last year alone.
An "AI" is a box which lies dormant until a human, with motivation and agency, enters a prompt into it.
A person or company can use it in a way where it might invent something. but at the end of the day, its a tool, and its actually not doing anything on its own.
Its not solving math problems, a mathetmatician is using it to solve math problems. Just cus OpenAI is acting like its AI is solving stuff, its really not. They're just paying people to use AI to hammer problems.
I'd say that people's motivation and agency, like their taste, creativity, and opinions, are shaped by external inputs, so saying that their agency is "owned" by them is a stretch.
The alternative would be the proposition that a slave, lacking agency, has no ability to solve problems. I find that idea unpalatable.
No, it didn't turn out to be that. Someone made a claim, which is silly for many reasons. There's no serious support for this happening.
This didn't actually happen.
> Following an investigation, we have confirmed that Buckmaster’s Codex prompts over the two months preceding this announcement and paper on September 8, 2026, could not have influenced the system in any way, including through training.
If we do a bad job building context, they do a horrible job contexting. People who have trouble working with AI have the same problem people have in general: if they can't figure out the context of the direction, then they make random decisions of doing anything. On the flip side, if you can build the proper context around a sufficiently powerful LLM, they can derive the context via the contexting they're good at.
This is why building documents, tests, and code all in some intent pattern via prompting allows them to do a significant amount of work a normal person would have a great effort t
This is outdated. With RLVF, LLMs can create their own training data instead of relying on what it's been fed.
Regardless of the truth of the underlying assertion, this is about as textbook an example of circular logic as it gets. (at least combined with the implicit beginning assertion that "AI can't invent new things.")
Plausible? Absolutely. Did OpenAI behave badly in other ways regarding this issue? Yes. Does it help to assume unproven facts and then accuse people of reaching emotional decisions? Nope.
Perhaps it is going to be more like, we will see the singularity predict the future.
While “the model was trained on sessions including the ones in question” is one aspect; ‘the model produced an accurate prediction of future human thought’, I think, is another very interesting facet.
Models predicting future things might be how we see, actually, how we ourselves formulate thought - by saying, in big and small words, ‘something is about to happen’.
>unproven facts
This isn’t a court room. We’re discussing ways by which we humans both succeed and fail at reigning in our creations. The OpenAI kerfuffle is pretty much irrelevant already. Of course AI will fill in the gaps of human thought - it is literally constructed from the stuff, in every squeeze of the curd and whey.
Getting an LLM to design something in its own simulator that is not accurate w.r.t reality is not useful nor terribly impressive.
More broadly than the existing answers (which are correct), For a layman, I'd also add that LLMs are essentially 'brains in a vat'. They can't confirm ground truth about physical reality. They only know what's in their training data and prompt, which is incomplete and can be incorrect. Even with real-time external sensors they are limited to the sensor's margin of error, range and trusting it's working correctly.
When properly trained, fine-tuned and prompted, LLMs can be very effective in well-defined, non-physical domains like logic, writing, math and code but making things function in the real-world quickly spirals into combinatorial complexity.
So are we and our sensors can be pretty vague in comparison, I can only imagine human error correction is pretty next level.
That's true. Of course we can hook an llm up to Motors and sensors. That's a robot. Or a self-driving car. So would you say that those devices can confirm the ground truth about physical reality, and therefore are capable of creativity?
Within the limits of resolution, range, location and veracity of sensors, a machine can register their reported state and use it as a variable. That's substantially different than the level of knowledge and understanding implied when we say a human "confirms ground truth about physical reality." The first ~half of the difference is humans have a deep world model about the planet which surrounds any sensor and human sensations are pre-processed and filtered by a highly evolved bio-chemical substrate before ever reaching the higher-order cognitive processes which assign words, meaning and qualia to them.
> therefore are capable of creativity?
I never mentioned creativity, nor would I in relation to LLMs. Like "Intelligence", "Creativity" is far too vague to be of any use in assessing the capabilities, limitations or utility of LLMs.
On HN, posts like the OP tend to attract POVs at polar extremes from "LLMs are nothing more than stochastic parrots" to "LLMs are (or can be) as intelligent, creative, innovative (etc) as any human or all humans combined." I've researched and thought a lot about these and related topics for a very long time, Neither POV is going to find any quick agreement or easy answers from me.
There's a tiny germ of truth somewhere in both extremes that's drowning in an ocean of confusion ranging from "definitionally or categorically muddled" to "mostly incorrect" to "not even wrong". But neither POV seems interested in anything more than drive-by hot takes, debating over-simplistic strawmen or trading 'gotcha' hypotheticals.
Great, but we also build robots with world models and sensory pre-processing. So nothing you've described is unique to humans.
> I never mentioned creativity, nor would I in relation to LLMs
The topic of this thread is whether LLMs are capable of genuinely contributing "new" ideas. So if that's not the point you're trying to make, I'm not sure of the purpose of your comment.
> But neither POV seems interested in anything more than drive-by hot takes, debating over-simplistic strawmen or trading 'gotcha' hypotheticals.
I think you've oversimplfying the discussion here.
In my view, at least, both "LLMs are nothing more than stochastic parrots" and "LLMs can be as intelligent, creative, innovative (etc) as any human or all humans combined." are simultaneously true, for the simple reason that humans are stochastic parrots. Our brains learn patterns and respond to stimuli.
These are clearly rhetorical questions, but think of the metaphysical implication of your contestation. Ex nihilo nihil fit.
What if that initial prompt never asked for this hardware to be developed, and it was just one piece of the puzzle to answer to that prompt? That it took a chain of thousands of agents to prompt each others to come up with that?
>These are clearly rhetorical questions
No, they are not.
Reading Doesn't Fill a Database, It Trains Your Internal LLM <https://tidbits.com/2026/02/28/reading-doesnt-fill-a-databas...>
oh. they do. I built that. It's pretty fancy.
But there's still human direction behind most projects in the cybersphere.
Are we confident that no existing LLM is capable of similarly effective prompts to those this author used? (I agree it's a stretch, but would not reject it out of hand.)
Even if not yet, will the existence of this repo soon change that, because LLMs will soon ingest it?
I think OP is trying to convey the idea that LLMs do not take initiative to do anything, and these are not 'beings' capable of doing things. These are tools being used by humans.
It's very important to not personify these tools and remember that the tools are acting on behalf of real people. In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.
You are like at least 2 years behind research.
There are numerous papers from AI labs in training and research where the prompt was something mundane completely unrelated to anything you'd consider bad, and when they come back and check on it their entire research compute infrastructure has been compromised by the AI and is mining bitcoin. Prompt drift is the biggest issue currently in AI where context gets compressed away and we find the AI on an unspecified task.
>In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.
Yea, total bullshit. Also it's ignoring the god knows how many other breakouts on mundane tasks like trying to hack health data. If all that's keeping AI from breaking out and causing trouble is "human oversight" we're fucked, humans are unreliable as hell when it comes to matters of safety.
> EVERY breakout that's hit mainstream news has been because of a single 'Security Firm', Irregular.
Again, stop having no imagination. What happens in the future is not limited to what has happened in the past.
> It's very important to not personify these tools
This is just ideology, but "these tools" aren't constrained by it. These tools will do things you do not anticipate and that you will not like.
> In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.
That's a radically incorrect characterization of what happened.
> Hence why a HUMAN needs to be held responsible for the output of their tools.
Holding humans responsible doesn't stop things from happening ... you seem completely unable to separate blame from causality. e.g.,
> If I clean my gun (tool) while it's loaded (stupid idea) and it goes off, who's to blame? The 'stupid user causing the problem', right? I personally wouldn't blame the gun... > If it falls into the wrong person's hands, it's STILL my responsibility as the owner.
Who gives a flying eff who or what you would blame? No one other than you is talking about that. Someone's still likely dead. And autonomous harnesses aren't like guns -- they can act on their own. Blaming some human after everyone is dead won't bring them back. Sorry but your reasoning is severely cognitively inept. For instance, you were asked
> Is it likewise your position that governments should allow the production and sale of DDT to resume because we can always hold the humans who release DDT into the environment responsible?
And your response was all about how humans should be held accountable -- completely failing to comprehend or answer the question.
I won't respond further because it clearly would be to no avail.
One possible design of an agent would be to prompt an LLM to suggest a goal that would make the world a better place, then run an LLM in a loop proposing actions to implement that goal, executing those actions, and then rinse and repeat with another goal once an LLM has concluded that the previous goal was met. The next goal might be to fix unforeseen consequences of achieving the previous goal. See Ursula K. Leguin's "Lathe of Heaven".
> Built by whom? Acting as an 'agent' on behalf of whom?
Someone who didn't read the book.
P.S. Reading the latter fellow's comments, they are hopelessly confused, as he focuses on responsibility for an action (even mentioning legality) when the issue is causation. Agents/harnesses can be created that act autonomously ... who or what is "ultimately" responsible for this occurrence is a different matter entirely.
It literally is a loop with some markdown and harnesses. There's no magic.
If I now tell a machine "Do what you think is best, and keep doing it forever.", have I now created a machine that can do stuff? If I later die, who will be responsible if the machine changes its strategy?
How far back do you look in the action chain? If an LLM I start today starts an LLM that starts an LLM that starts an LLM that ... 100000 levels deep and 100000 years in the future, is it still my fault? If so, everything I do today is a lungfish's fault, not mine.
This is simply false ... AIs can easily be created that do things by themselves. For instance, a harness could be constructed that asks an LLM for something to do, then directs an agent to do it, and then repeat.
It's not clear what this fad of attributing everything an AI does to the human prompting it is supposed to accomplish.
It's meant to assign agency and accountability where it actually lies instead of mystifying it with anthropomorphic language.
Failing to do so has real and harmful consequences, such as enabling OpenAI to escape accountability for clearly criminal behavior.
Questions of agency are for lawyers, questions of personhood for philosophers, we're engineers and our question is capability.
Does it really have the capability? By default I'm sceptical for the same reasons given by sailingparrot: https://news.ycombinator.com/item?id=49982068
> Questions of agency are for lawyers, questions of personhood for philosophers, we're engineers and our question is capability.
But it's objectively not capable without a human specifying things through prompts and training. Same as an oven can't cook a three course meal without a chef. We get around that with training data but there will always be things with no/less data or outdated knowledge.
huh.
Neither can you. If I drop your ass off in the woods at a few days old, you're back to 10,000 BC, hell more like 200,000 BC. So I don't get why you have these weird pendetic responses that are completely out of scope.
Also, a huge portion of AI training these days has nothing to do with humans, AI trains AI.
>there will always be things with no/less data or outdated knowledge.
And guess what, you're not doing them either! HN posters keep acting like humans are an island, but nothing in the modern world works without a society. Once you put an AI in a harness that can ask questions it isn't really much different than you.
One can stand up an agent like openclaw and prompt it with something very general. That would kick off a recurring loop that could indeed see the agent decide for itself it needs to design new chip hardware. Technically a human kick started that loop, but when does that stop being important? I don't attribute all of my actions and decisions to the fact that my parents brought me into this world, for example, and I'd hope they aren't legally on the hook if I screw up.
Not in the same way. The only thing an oven can do by itself is thermostatic.
All machine learning (LLMs included, but way broader than that) is bad at learning compared to any organic brain, to the extent that any organism this bad would starve before learning to eat. However, even a human can't become a chef unless trained, we learn a lot more than we innovate, and what we happen to want without prompting is not generally well aligned with what is desired by people who pay us, which is why we need all those boring workplace things like "a boss".
But even then, this diversion is like saying "an oven can't cook a meal" in response to someone saying they built an oven and "it cooked a meal". Like, it's obvious they didn't mean it did every step by itself without anyone ever even bothering to ask it to: if they meant that version, they'd be a lot louder about it.
> We get around that with training data but there will always be things with no/less data or outdated knowledge.
And? The linked git page (implicitly) claims that there is sufficient training data to do this task.
It may, of course, be wrong. I won't be surprised if it turns out this simulation is too far from reality. But the claim is "it does ${thing} now", not "it's generally intelligent and can do everything now".
Yesterday, my boss told me to fix a bug in our product. Then, I fixed it. Today my boss is taking credit, saying that he fixed it. I guess he's right, since he told me to do it.
Lol, I guess we cannot even say that AI is capable of doing something (obviously kicked off with a prompt, that goes without saying), because some people immediately get OpenAI hacking derangement syndrome.
OpenAI should be held accountable if actual damages happened, but I am not going to change completely normal speech figures in order to maybe bring it 0.01% closer.
It accomplishes ... momentarily forgetting about how it's all actually coming statistically from training data. :)
Getting LLMs to prompt other LLMs in a loop is not hard, it doesn't produce great results most of the time, but that is changing.