Neural Abstract Reasoner

Claims 78.8% on the Abstract Reasoning Corpus [1] (ARC) in the abstract, but clarifies that this is only on 100 out of the 400 evaluation tasks, all on 10 x 10 grids. The 10x10 grids are hard-coded in the network architecture so it's not possible for the system to perform any better without re-design and re-training.

Trains on a dataset augmented to 1500 tasks by permuting colours and "exploiting that the tasks are invariant to rotation and symmetry". So it's cheating, but that just goes to show that the ARC dataset is not the full-proof benchmark of intelligence that its author meant it to be. It's possible to solve at least a quarter of the tasks by cheating and without demonstrating what the author considers the hallmarks of intelligence- the ability for abstraction and reasoning [2].

____________

[1] https://github.com/fchollet/ARC

[2] https://arxiv.org/abs/1911.01547