zachbee

They're not wrong that sampling a complex, higher-dimensional probability distribution is hard to do efficiently. I'm not sure how useful it is to do it more efficiently, though.

Also, the fact that they're using ultra-cold superconductors makes me wonder how much noise helps and how much it hurts. If your system is all about leveraging noise well, but you can only use super special well-behaved noise, then "bad noise" could easily ruin the quality of your generated solutions.

It's cool to see something so wacky out there, though!

The reason HBM "isn't deterministic" is because the memory controller sometimes stalls reads when doing a refresh operation. It's possible to pause refreshes [1] or schedule refreshes ahead of time to get HBM and DRAM to act deterministic.

To be fair, that's a huge hassle, and I have no clue what level of support commercial HBM controllers have for refresh pausing or manual refresh scheduling. It could very well be impossible to do in practice -- and regardless, their performance numbers speak for themselves :)

[1] https://memlab.ece.gatech.edu/papers/TACO_2014_1.pdf

The article says they ran GPT-2! Which isn't particularly large, but replicating a large language model with a spiking neural network seems like novel work at least.

Neuromorphic computing is cool, but not new tech. However, using a neuromorphic spiking architecture to run LLMs seems new. Unfortunately, there doesn't seem to be a paper associated with this work, so there's no deeper information on what exactly they're doing.

Neuromorphic computing basically uses individual "neurons", represented with either analog or digital circuits, which communicate using asynchronous pulses called "spikes". Unlike the human brain, neuromorphic chips are 2D, but we can replicate a good amount of neural dynamics in silicon.

It's unclear how they managed to use this to run LLMs, though. Getting GPT-2 running with SNNs is a legitimate achievement, because SNNs have traditionally lagged significantly behind conventional deep learning architectures.

https://web.stanford.edu/group/brainsinsilicon/documents/ANe... https://web.stanford.edu/group/brainsinsilicon/documents/Ben...

I'm a professional chip designer. And as somebody who's had to build this sort of internal tooling myself, I think this sort of product is desperately needed!

It makes sense to start with Verilator, because it's fast, easy, and open source, but it seems like it'll fall short on a lot of metrics. It seems like the big challenge is going to be making sure your tool can do everything that chip designers actually need to do. I remember spending almost a week bringing up gate-level simulation, which was a huge pain in the ass because our design used an ILM-based hierarchy. Handling things like X-propagation, timing constraints, mixed-language designs, and non-synthesizable models for mixed-signal IP is going to be really tricky, especially because they work differently in every simulator.

Is there any reason why you're not starting by targeting FPGA developers instead of semiconductor developers? It seems like the Verilator flow will do a much better job of fulfilling their needs, which are generally simpler, than fulfilling the needs of an ASIC team. Obviously in the long run, semiconductor developers are the bigger market, but you need far more features to be able to sell to ASIC teams.

They get impressive performance, but it's not really the first "LLM-native processor". They built the thing a while ago before LLMs were that hot, and rebranded it as an LPU to pivot into Generative AI: https://www.eetimes.com/groq-demos-fast-llms-on-4-year-old-s...

You need an entire rack or two worth of Groq systems to get this performance, because the only memory the system has is on-chip SRAM (220MB). If you're trying to run on a small number of systems, you need local DRAM or HBM, which Groq doesn't have.

What's the difference between your product and Gridspace's? I get the sense that your offering is more developer-focused, but I'm curious if there are any technical differences.