Hacker News Reader

Comment by skew-aberration | Hacker News Reader

I'm not sure that more efficient compute can win out, since it also has to beat out the competition on capital investment. There's an upper bound of 'just get 3 sets of GPUs and run them on solar in three different timezones' where the cost of electricity is negligible. Efficient digital architectures run about 1/5 or 1/10th the speed and require much more wafer area. Analog technologies seem unlikely to have the fidelity required for training, though inference sounds plausible. It's also not clear that analog technologies would necessarily use less power.

View on HN

Transistors are most power efficient when they are either completely open (no resistance) or completely closed (no current).

If they are in some analog half-open state they have both resistance and current, which means waste heat.

Also transistors in modern chips are already so small that you can literally count the number of electrons flowing through them. At the smallest scale, nature becomes discrete. So if there are something like 128 electrons, you only get 7 bits of 'analog' to play with anyway.

Recently I started thinking about an analog optical device capable of matrix multiplication. After some online research, the Mach-Zehnder interferometer seems to have been adapted by a German company into a full addon optical based matrix multiplication system.

It is a very interesting concept and the company Q.Ant has delivered working products and software. Apparently it can greatly speed up certain training applications and future designs will solve some of the current problems.

The energy and performance advantages seem to be worth the R&D. If some form of optical memory to replace DRAM in the chain could be designed, that would solve several conversion costs between optical to digital.

Either way it is pretty cool to do math this way, basically just using light.

The assumption that optical or analog will be more efficient for a given accuracy though is not grounded. Any non-linear optical phenomenon requires multiple photons interacting in matter, or interacting sequentially with the same matter, and this requires high intensities of (almost always coherent) light in order to make the interaction happen frequently enough.

e.g for a Mach-Zehnder interferometer measuring the phase shifts requires many photons. A single photon arrives or doesn't arrive at the detector. Even if you could perfectly ensure it hits detector at the same time as the reference photon and subsequently detect with 100% quantum efficiency, that still only gives you 1 bit of phase information, not an analog value.

Digital can be done more efficiently with adiabatic or asynchronous circuits. The limiting factor then is speed and chip area rather than joules-per-and-gate.

Reply on news.ycombinator.com