Brain dead trend-extrapolation is a pretty good prediction tool, but I suppose my assumption that modern NNs aren't more efficient was sort of an arbitrary assumption that makes the exercise useless. Is there any work on this?
I'm not sure what evidence the performance of these systems show us, as bees are pretty smart for their size. These system are about as optimized for lip-reading and translation as bees are for their various tasks, so it wouldn't be surprise me if it turned out that they are similarly powerful per synapse to NNs.
http://timdettmers.com/2015/07/27/brain-vs-deep-learning-sin...
A key assumption I don't share with Detmer is that we must match the brain's raw computational power to have a chance at human-level AI. From an evolutionary point of view, most of the brain's computations surely must be superfluous to intelligence and necessary instead to a vast host of other biological functions.
I'm not sure how you get that. My point is that artificial NNs may well be more efficient than biological ones because they are punching so far above their weight based on what you would expect from a naive comparison of parameters, in which case it's not 17 iterations but a lot less. Further, modern NNs don't typically max out the memory of a single GPU (since we want to do minibatches of at least n=10 for the training phase and researchers care a lot less about the forward passes or deployment), and groups like Google Brain or Deepmind have shown the ability to use ~1000 GPUs, so that's 4 orders of magnitude right there (doable with asychronous training like synthetic gradients).
> I'm not sure what evidence the performance of these systems show us, as bees are pretty smart for their size.
What do bees do that is remotely as demanding as being able to translate between English, French, German, Japanese etc at very high quality?
>What do bees do that is remotely as demanding as being able to translate between English, French, German, Japanese etc at very high quality?
This seems more impressive than anything any one NN can do, but less impressive than what all NNs can do: https://en.wikipedia.org/wiki/Bee_learning_and_communication