Josef – A robot who learns how to draw

So you've got a (deterministic) L-system, which defines a family of paths (a family because you get to choose when to stop applying your production rules). It looks like it's an L-system with a single production rule of the form F -> something, with the "base case" being that F means one step forwards.

And then you have a neural network which "tries to predict next action of the drawing grammar". Its input is the last 5 "basic operations", meaning {forward, left, right, leave-red-box, save, restore} or something (can't be quite that because it's got 7x5 inputs and there are only 6 of those; do we have an extra pseudo-op for "nothing" to use near the start of the path?). Its output is its guess about the next operation (as a set of activations for 7 outputs, presumably the obvious 6 plus "end"). I guess we could interpret those outputs, suitably transformed, as probabilities for the next output symbol (and interpret training the NN as likelihood maximization).

And then ... I don't know what. Do you expand the L-system to some fixed depth, and then run through it training the NN along the way, and then let the NN run freely and draw the output, or what?

(That feels like there wouldn't be anywhere near enough training data, unless the depth you expand to is terribly large. Perhaps not even then -- training won't tell you anything about what to do when a sequence of basic ops arises that can't come out of the L-system, but you'll get that happening sometimes when you free-run the NN. But it's not clear what "should" happen in that case, so maybe anything you do is OK then.)