I don't know, but there was a paper posted here yesterday: LongRoPE: Extending LLM Context Window Beyond 2M Tokens[0]

[0]: https://news.ycombinator.com/item?id=39465357