https://paperswithcode.com/paper/most-language-models-can-be...
I want to update that just because GPT-4 appears to be far better at following constraints, doesn't mean that it's anywhere near perfect at following them. It's better now at my easy example of "ban the letter e" but if you ask for several constraints, or mixing lexical and phonetic constraints, it gets pretty awful pretty quickly. Filter assisted decoding can make any LLM (no matter how awful they are) follow constraints perfectly.
I can't wait to get someone whose better at coding than me to implement these techniques in the major LLM frontends (oogabooga, llamma.ccp, etc) since my attempt at it was quite poopy research code: https://github.com/hellisotherpeople/constrained-text-genera... - especially with the idea of this but with all LLM types (e.g. bert), using a declarative syntax like langchain, and also supporting constrained beam search for sequence level constraints.
BTW - just to motivate the nay-sayers on why filtering LLM vocabularies may be a cool (and dangerous) technique even in a world where LLMs became "perfect" at following lexical, semantic, or phonetic constraints in the prompt. Imagine asking ChatGPT to "generate some social security numbers". Now imagine doing that again when it's only allowed to generate number or hyphens. This is one of the many reasons why full vocabulary probability distributions are not exposed by folks like OpenAI