What does “pretrain” mean in this context? It sounds like normal training
The "main" training step using huge amounts of inputs is called pre-training. The idea is that after that pre-training, you might fine tune the model for your specific use case.