I definitely see how my language would convey that whereas I mean to refer to digital computers, not all hardware, when I mentioned the Von Neumann bottleneck. There are different arguments against digital hardware in general, even though FPGA's can be about an order of magnitude more efficient than microprocessors. Modern analog goes much further, although we're still not very good at 'programming' it, which is why I think AGI is a ways out.
Also, perhaps I'm misreading their paper[1], but that DARPA project is still using digital logic. In fact, they specifically say that they decided against using analog computation due to the increased density of digitial (one of the main problems with analog computing, as I mentioned). Regardless, it's pretty clear that the project doesn't have the goal of replacing general purpose computing, so I'm curious why you consider this evidence that analog computing will replace digital computing?
Finally, in what way are FPGAs much closer to analog computing than microprocessors? The advantages of FPGAs (configurability) are orthogonal to analog vs digital.
The topic is AI, where efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. Those are best described as specialized FPAA's and they certainly do rely on digital for programming, and special (although still solid state circuits) for pseudo-analog properties. However, the most interesting parts are yet to come, and are being developed by partners/founders of the program: http://knowm.org/about/
The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. This will obviously need to interface with/through digital circuits to be of any use.
However, I still take issue with your overall point.
> Efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse.
I think it's at least plausible that we'll see gains from fundamentally different architectures (namely combining memory and computation) but where's the evidence that we need to switch from digital to analog? Where's the evidence that analog is even better for an architecture like that? Again, the DARPA project you linked actually gave evidence of the opposite.
> The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems.
You keep talking as if there's some obvious reason that analog is more efficient/faster/better for the types of architectures you're talking about. I'm not seeing that reason.
Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor).
It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains).
That's not necessary true; again, see the DARPA project you listed.
> Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations.
The analog hardware doesn't necessarily behave like the desired components either (for example, due to manufacturing errors). With digital you get to choose your accuracy and with much greater control.
> Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor).
Unless you don't care about noise at all, your design will be much more complicated than a single transistor. Either way, it's actually much easier to do this sort of "approximate computing" in the digital domain.
> It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains).
A bird has feathers yet our most efficient airplanes don't use them; there's no reason at all to believe that just because the brain is analog, our design should be as well.