Hacker News Reader

We don't actually know how big the Dataset is, right? It could be the same dataset used for Llama 2, but trained for more Epochs.

The dataset is 7 times bigger than the dataset used for Llama 2 as reported by Meta.

Has Meta disclosed how much parts of the dataset were repeated? I've only seen the "number of tokens trained" number.