Cortical Column Networks
thesmartrobot.github.io
thesmartrobot.github.io
I think your statement could be better informed. The article you link to for example doesn't deny the common structure of cortical columns, but posits that they don't have a known function. That's kinda the point of Numenta's research. To suggest a function.
And it's not that the idea of cortical columns (or more specifically minicolumns) was found to be wrong, we've gone from a simplistic explanation to a more complex one via both more careful measurement, new techniques, and cross species comparisons. I think this review is pretty good on these points:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2820381/
There's also this idea that the neocortex is composed of 6 layers. That idea isn't totally wrong but it's required updating as we gain more information.
Even the neuron hypothesis itself has had to undergo significant revision as we better understand the computational power of sub-neuron units and companion cells like astrocytes.
So if you want to criticize cortical columns for being an over simplification, that's fine, but 1. we all know that and 2. simplifications are particularly useful when you need to reduce such systems to code as the OP has done.
The idea of cortical columns isn’t like the 6-layer-cortex. The cortex just isn’t organized into little column-shaped modules. It’s true that if you record two neurons right on top of each other, they’ll have a similar receptive field/preferred direction/…. BUT there is no consistent horizontal organization of neurons into columns. There’s whisker barrels, which reflect the way rat and mouse whiskers work, and there’s ocular dominance columns, which are a totally different thing at a different spatial scale. But there’s no global organizing principle that should be called “cortical columns”. It’s a lovely idea that would be great if it were true—-it would be so much easier to understand the brain if there were a common unit of organization larger than a neuron but smaller than a brain! But it’s not so.
Neuroscience research is huge and moving fast. Researchers have trouble keeping up with the bleeding edge in their sub-sub-field, let alone neuroscience as a whole. Conferences are one major way that most of us find out about new work.
Pre-covid, conferences were pretty exclusive and unwelcoming to non-researchers. However, now that conferences are all online due to covid, one thing I would recommend is going through the videos of old conferences to find something you are interested in. Then you can pause the video and look up any words or concepts which are unfamiliar. It may take a while to get through a talk, but you'll learn a lot this way. For example, Neuromatch is the big "mostly computational neuroscience" online conference: https://www.youtube.com/channel/UCcBKrxkfNv04R9PXLovjf5w/vid... Another formerly-in-person conference is Cosyne: https://www.youtube.com/channel/UCzOTbZTHTubFNjANAR33AAg/vid...
Heck, registration for Neuromatch is cheap so you can attend this year's conference if you want to: https://conference.neuromatch.io/
These exist consistently across the cortex. We've mapped out their shape and connections. This is what one looks like https://www.youtube.com/watch?v=jmdPOaTpZWA (this even links to the paper you cited but calls them 'cortical columns').
Let's call the larger column structures with significant horizontal coverage posited by Mountcastle 'hypercolumns.' As you and others have noted these seem to be limited or non-existent.
As I've said thanks to updates to our technology we've refined the idea of 'cortical columns' into a class which doesn't have useful explanatory power ('hypercolumns') and one which does ('minicolumns'.)
The vertical integration of processing across layers of cortex by a collection of neurons is IMO of significant interest and import. You may disagree, but please don't pretend that these structures don't exist.
Meanwhile, cortical column fans (including neuroscientists!) engage in a motte-and-bailey argument, where the bailey is “the cortex is divided into columns that represent an intermediate level of modularity” and the motte is “neurons on top of each other usually have similar firing properties”.
Listening to an interview w/ Jeff Hawkins for "A Thousand Brains", it reminded me of Lingua Ex Machina [2009].
"A neuroscientist and a linguist show how evolution could have given rise to structured language."
https://mitpress.mit.edu/books/lingua-ex-machina
https://www.amazon.com/Lingua-Machina-Reconciling-Darwin-Cho...
I tried to dig a bit, to see if there's a connection between Jeff Hawkins and William Calvin's works. No joy. I'm probably just grasping at straws.
Am just a curious layperson. It's been ages since I read On Intelligence and Lingua Ex Machina. At the time, just the notion of Darwinian processes happening within the brain rocked my world.
From "Jeff Hawkins Is Finally Ready to Explain His Brain Research" https://news.ycombinator.com/item?id=18214707 https://westurner.github.io/hnlog/#comment-18218504
Doesn't sound like these CCNs necessarily reuse a lot of information across object categories.
> New research from a Google team proposes replacing the self-attention sublayers with simple linear transformations that “mix” input tokens to significantly speed up the transformer encoder with limited accuracy cost. Even more surprisingly, the team discovers that replacing the self-attention sublayer with a standard, unparameterized Fourier Transform achieves 92 percent of the accuracy of BERT on the GLUE benchmark, with training times that are seven times faster on GPUs and twice as fast on TPUs."
Would Transformers (with self-attention) make what things better? Maybe QFT? There are quantum chemical interactions in the brain. Are they necessary or relevant for what fidelity of emulation of a non-discrete brain?