RWKV does: https://github.com/BlinkDL/RWKV-LM It uses „the Pile“: https://pile.eleuther.ai/ And I’ve seen some more in the last weeks.