Yes but fine tuning for RL is not expected to be hard. You're essentially limited by how much human feedback is available, so it's very different from training the foundational model on random bulk data.
They kept the details closely guarded and only hinted at how they did RLHF and transitioned the architecture to self supervised learning.