ARC-AGI

On the Measure of Intelligence, François Chollet [1] Fellow HNer seems to have liked a lot of ML paper, this is not breaking the trend. This is a great meta paper questioning the goal of the field itself, and proposing ways to formally evaluate intelligence in a computational sense. Chollet is even ambitious enough to propose a proof of concept benchmark! [2] I also like some out of the box methods people tried to get closer to a solution, like this one combining cellular automata and ML [3] [1] https://arxiv.org/abs/1911.01547 [2] https://github.com/fchollet/ARC [3] https://www.kaggle.com/arsenynerinovsky/cellular-automata-as...

Claims 78.8% on the Abstract Reasoning Corpus [1] (ARC) in the abstract, but clarifies that this is only on 100 out of the 400 evaluation tasks, all on 10 x 10 grids. The 10x10 grids are hard-coded in the network architecture so it's not possible for the system to perform any better without re-design and re-training. Trains on a dataset augmented to 1500 tasks by permuting colours and "exploiting that the tasks are invariant to rotation and symmetry". So it's cheating, but that just goes to show that the ARC dataset is not the full-proof benchmark of intelligence that its author meant it to be. It's possible to solve at least a quarter of the tasks by cheating and without demonstrating what the author considers the hallmarks of intelligence- the ability for abstraction and reasoning [2]. ____________ [1] https://github.com/fchollet/ARC [2