Guiding Text Generation with Constrained Beam Search in 🤗 Transformers

Summary

Constrained beam search introduces direct control over generated text in Hugging Face Transformers. Standard beam search operates token-by-token without knowing the optimal step to force specific words or phrases, making it difficult to enforce mandatory vocabulary or choose between alternative expressions. To resolve this, the generate interface accepts constraints through arguments such as force_words_ids and a list of Constraint subclasses like PhrasalConstraint. This mechanism enables disjunctive constraints, where generation must include at least one phrase from a provided set, alongside strictly required sequences. Consequently, practitioners can inject prior knowledge or formatting requirements directly at generation time rather than filtering candidate outputs afterward.

Takeaways

1. 1

   Constrained beam search allows injecting prior requirements directly during generation instead of generating and filtering multiple outputs.

2. 2

   Disjunctive constraints allow users to supply a list of alternative words or forms, requiring the model to include at least one in the final text.

3. 3

   Custom constraints can be constructed by subclassing the Constraint abstract interface class and passing instances to model.generate().