Hacker News Reader

I wonder if we will see LLMs on FPGAs or FPGA like devices.

Crypto went from graphics cards to ASICs, we may see something similar with LLMs given the hype.

I always think of FGPAs for things you want to update without having to repurchase the hardware. If the FPGA inside a gadget can be reprogrammed over the Internet then I think it's more suited than an ASIC for LLMs.

It could go even further, in theory. The kind of ops that the current crop of LLMs needs is very simple, and at the same time there's no hard requirement for precision (which is why 4-bit quantization works so well). This means that unconventional approaches such as analog computing are potentially in the play again - it's easy to do addition and multiplication in an analog circuit, if you don't care about the answer being precise, and in theory one could pack a lot more of those circuits in the same space.

There's already at least one company with a product: Mythic AMP https://mythic.ai/products/m1076-analog-matrix-processor/

And there's several research papers about other architectures. For example: https://arxiv.org/abs/2205.05853

Reply on news.ycombinator.com