Research on the cerebellum yields rewards

First, it represents a fundamental shift in how the cerebellum learns. For a long time, it was thought that the cerebellum learned liked a perceptron neural network, i.e. that an error signal was computed and used to change the strength of synapses within the cerebellum to result in the correct output signal to guide motor control (and, we know now, cognitive control). In other words, the cerebellum was a supervised learning machine [1]. But how these error signals were actually computed using the neuronal circuitry was never made clear; most arguments centered around the microcircuitry of the inferior olive. In a perceptron, an error signal is the difference between the correct output and the predicted output. But how is the "correct output" supplied?

As far as I know the first to propose the cerebellum learned by reinforcement learning was the famed cerebellum researcher Richard Thompson [2]. Unfortunately, the idea was only vaguely sketched out the field didn't take this very seriously and continued on with the general belief that the cerebellum learned by supervised learning.

To me and my collaborator, Tadashi Yamazaki, it seemed a more natural signal that the nervous system could capably supply would be a graded reward signal. Moreover, this meant we could interpret the structure of the cerebellum within the theoretical frameworks of reinforcement learning such as the actor-critic framework [3,4]. This is the second reason these finding are a big deal: the paradigm shift to the cerebellum being a reinforcement learning machine, if correct, will be a boon for building better models of it. In the last few years there has been some impressive work done with reinforcement learning in artificial neural networks that could be applied to models of the cerebellum, especially within context of the brain at large.

[1] Doya, Kenji "What are the computations of the cerebellum, the basal ganglia and the cerebral cortex" (1999) [2] Thompson, Richard "The nature of reinforcement in cerebellar learning" (1998). [3] Lennon, William "Towards more biologically plausible computational models of the cerebellum with emphasis on the molecular layer interneurons" (2015) [4] Yamazaki and Lennon "Revisiting a theory of cerebellar cortex" (2019)