E09: Analog Computing: The Once and Future King
stateofthefuture.substack.com
stateofthefuture.substack.com
"costly digitization" you say? Let's look at the examples.
AD633 [0] is an analog 4 quadrant multiplier. It costs $10 for a single chip (which can only multiply two numbers). It's datasheet claims "1% total error", that corresponds to roughly 7 bit of digital performance (this is _after_ each chip being individually laser trimmed for best performance). It's small signal bandwidth is 1MHz, so that's about 1 million multiplies/second _in best case_ (for larger signal it's going to be slower). It draws 4mA @ 15V - that's 60mW.
Compare to digital multiplier: STM32F030F4 is a 32-bit microcontroller that runs at 48MHz, which costs $0.80. It can do a lot of stuff, including _single cycle_ multiplier - that means multiplying 32-bit numbers, and doing up to 48 million operations/second (practically, it's going to be less.. say 12 million ops/second). And for that, it takes 22mA @ 3.3V - that's 72mW. And that's a _value line_ MCU, there are slightly more expensive versions with much more speed and less per-operation power consumption.
So I don't think analog is coming back. It costs much more for significantly less precision and lower speed, and sees no significant benefits from scaling up. Moreover, there are fundamental physical laws which limit analog circuits improvements (like thermal noise).
[0]https://www.analog.com/media/en/technical-documentation/data...
> Deep learning and recent transformer models are really just matrix multiplication machines. Analog integrated circuits are well-suited for efficiently implementing matrix multiplication operations. [...] Keeping computations in the analog domain avoids costly digitization and leverages the parallelism and density possible on an IC substrate.
I call BS on this. You are never going to have enough precision in analog world to do what top current ML models do. And even if you can create model dumbed down enough to run on analog, it's cost (and power usage) is going to be astronomical.