Yes, but it also makes a huge difference whether we are asking the model to interpolate or extrapolate.
Generally speaking, models perform much better on the former task, and have big problems with the latter.
Generally speaking, models perform much better on the former task, and have big problems with the latter.
If an LLM has the exact answer needed it could simply be returned without needing to be rephrased or predicted at all.
If the exact answer is not found, or if the LLM attempts to paraphrase the answer through prediction, isn't it already extrapolating? That doesn't even get to the point where it is attempting to combine multiple pieces of training data or fill in blanks that it hasn't seen.