If you've seen tensor trains, might be interesting to reduce the rank even further by splitting an input into many sequential small matrices (instead of just two.)