Minibatch OT in flow matching also has a very similar mechanism, where samples from a noise distribution are matched to the closest data point.
There is a lot of prior work here that the authors neglect to discuss, which portrays this work as more novel than it actually is.
A comparison with minibatch optimal transport is in appendix A.2 of the paper.
GRPO (or GRPO like approaches) for diffusion/flow matching similarly can be likelihood free.
Do you have a particular paper in mind that does likelihood-free best-of-K but just calls it GRPO?
Likelihood is not fundamental to the spirit of GRPO, any exploratory mechanism would work.
That sequential LLMs have a step-wise probability is convenient but not critical to this approach (where rejection sampling is widely used in diffusion models).
Exploration beyond Pretraining. The mode collapse XMs address during pretraining also often shows up in post-training, where RL fine-tuning is known to sharpen models onto a narrow set of behaviors [76]. Recent fixes such as pass@krewards [77] and best-of-N-aware fine-tuning [78] can be seen through our lens as Forward XM, with a verifier standing in for ground truth data. These fixes act only during post-training, though; pretraining with exploration may yield base models that capture more modes in the first place, leaving RL more to select among.
On the other hand, Minibatch OT (optimal transport) was one of the more fundamental advancements early on in flow matching & rectified flow models.
This best of K approach effectively discards matches that would otherwise be preserved by optimal transport assignment.
I still think the authors here are too broad regarding their claimed novelty.