Thank you for this excellent post series. It reminds me a lot of the pre-LLM days, though I was mostly using LSTMs back then. When the original GPT paper came out, I thought the future would be using LLMs to generate tons of synthetic labeled data and then training specialized LSTM or transformer models per-task.

Had a couple of questions:

1) You note that ARC-AGI is a meta-learning task, have you tried any meta-learning algorithms such as MAML?

2) Do you think this approach could extend to ARC-AGI 3? Or do you think the interactive environments require a higher level of complexity than what can be achieved with a small model?