Hacker News Reader

I think that this is what you will be looking for

https://huggingface.co/blog/constrained-beam-search

This technique has been quietly available within HF for awhile, and has a lot of real world use-cases.

For any models where you get access to the code (counting HF in this), definitely a lot of tooling available - and this tool looks really nice hadn't seen it before! Hope similar levels of control can become available for the bigger, less accessible systems, even though people don't want to expose token log-likelihoods there should be ways.

I've also been reading your paper on these topics, based on previous comments on HN. So thanks for that! Always interested in work about constrained generation.

I'd really like to see some version of the techniques from this palindrome generation https://www.ijcai.org/Proceedings/15/Papers/353.pdf revisited in modern contexts. Or some of the other Sony CSL work on this topic (e.g. https://www.francoispachet.fr/wp-content/uploads/2021/01/pap...) which I will admit I (still) do not understand yet.

Reply on news.ycombinator.com