I'ts like an electric circuit where the residual stream provides the concept of a feedback analogous to the trace in traced monoidal categories https://en.wikipedia.org/wiki/Traced_monoidal_category.

Attention is a recurrence relationship that gets gradually pruned.