To be clear, this is a 7B model. It's just trained on 1.7 trillion tokens. At first I was confused why they were making such a big deal of of a massive 1.7T model outperforming a 7B model.
By the way, GPT-4 has been predicted to be using a 1.7T model, although OpenAI has not confirmed this to my knowledge.