Amazing experiment. Indeed, with the benefit of hindsight, we can say that breakout is a game that is probably better suited to deterministic code (it's not like Go where an AlphaZero style training is essential), and Astra is probably translating this internal representation of code into the shape of linear algebra and model weights.
I should note that the original authors did not allow Astra to write any significant code to construct the weights. I think this is unfair. If you allow strided assignments to numpy arrays, you should probably also allow the use of functions like np.linspace, np.repeat, and np.tile.
Oh and Astra wrote this (https://github.com/RikiyaT/lm_understand-logs/blob/4d09ba6d6...):
W1[63:120:6, 5] = 10
W1[123:180:6, 5] = 20
W1[183:240:6, 5] = 30
W1[243:300:6, 5] = 40
W1[303:360:6, 5] = 50
W1[363:420:6, 5] = 60
W1[423:480:6, 5] = 70
W1[483:540:6, 5] = 70
W1[543:600:6, 5] = 90
I think the second 70 there is probably a typo; it looks like it should be 80 instead.