Not well. Xiaomi published a detailed report of their RL process for their recent MiMo v2.6 model (https://news.ycombinator.com/item?id=49792730), which I read through recently. It is an interesting insight into how constructing RL environments work -- for some things, like code generation or code exploitation, there are naturally-perfect RL environments (Github issues + the closing PRs for the former, CVE databases + getting smaller language models to generate exploits triggering them for the latter) -- but for some other things, like code /quality/, Xiaomi at least (and presumably Anthropic et al.) simply do not have any objective criteria.

So they generate a code "rubric" and give a model some number of gold stars for how "well" another language model thinks it satisfies specific enumerated aspects (for code, this is gonna be things like using language features idiomatically, compactness, etc; for writing, this is gonna be things like what you'd get graded on in writing assignments in the fifth grade). Incredibly stupid. I'm shocked it works at /all/. But the lack of any truly objective criteria of "quality" wrt. code/prose explains why it doesn't work all too well, IMO.