We have many great things in research and development phases, so stay tuned. I’m hopeful we can share more in the coming weeks and month!
I hope y'all consider longer context models as well.
Also, are ya'll looking alternative architectures like Mamba? Being "first" with a large Mamba model would cement your architectural choices/framework support like llama did for Meta.