>>"RNN weights and the recurrent weights Tf in an LSTM tend to look more like (I + low_rank)" - Do you have reference for this? But thanks for suggestion, I'll have a look into this.

>>"I gather that with your F-LSTM you do:" - Your understanding of F-LSTM looks correct

>>"output = input + T2" Not quite clear where to put non-linearities. But this looks similar to residual connections. Which, in my experience, is almost always a good idea.