The task LLMs are trained on is "predict the next word", which elegantly is included for free in your training set of text. Typically no annotation is provided, since that would involve a ton of human labor doing the annotations.
Using models to generate/score/rank/modify data to be more useful as training data is a very interesting angle.