zimablue22

Yeah. This work is over claiming the novelty quite a bit.

Any of the Best of N papers that exploded in popularity after GRPO.

Likelihood is not fundamental to the spirit of GRPO, any exploratory mechanism would work.

That sequential LLMs have a step-wise probability is convenient but not critical to this approach (where rejection sampling is widely used in diffusion models).

You're right about the original GRPO proposal, but there are simplified variants that do just use best of K sampling.

GRPO (or GRPO like approaches) for diffusion/flow matching similarly can be likelihood free.

That's fair.

On the other hand, Minibatch OT (optimal transport) was one of the more fundamental advancements early on in flow matching & rectified flow models.

This best of K approach effectively discards matches that would otherwise be preserved by optimal transport assignment.

I still think the authors here are too broad regarding their claimed novelty.

This is just GRPO (proposed by DeepSeek), which similarly samples many plausible generations, selects the best of K, and trains that sample.

Minibatch OT in flow matching also has a very similar mechanism, where samples from a noise distribution are matched to the closest data point.

There is a lot of prior work here that the authors neglect to discuss, which portrays this work as more novel than it actually is.