Convert Transformers to ONNX with Hugging Face Optimum

Summary

Exporting Hugging Face Transformers models to the ONNX format can be accomplished through three different abstraction levels. The low-level approach utilizes torch.onnx.export, requiring manual specification of dummy inputs, input names, output names, opset versions, and dynamic axes configurations. At the intermediate level, the transformers.onnx package simplifies the conversion process by relying on FeaturesManager and prebuilt configuration objects to handle dynamic axis definitions automatically. The high-level method uses Hugging Face Optimum classes such as ORTModelForSequenceClassification by setting the from_transformers flag to True inside from_pretrained. This Optimum export leverages transformers.onnx internally and produces a model ready for immediate inference execution or integration into pipelines.

Context

Developers need methods to export Hugging Face Transformers models, such as distilbert-base-uncased-finetuned-sst-2-english for text classification, into ONNX graphs across different levels of API abstraction.

Approach / What changed

The post demonstrates three conversion workflows: low-level export using torch.onnx.export with explicit dynamic axes and dummy inputs, mid-level export using transformers.onnx with FeaturesManager configuration objects, and high-level export using Optimum ORTModelForSequenceClassification with from_transformers=True.

Takeaways

• The low-level torch.onnx.export API requires explicit configuration of dummy inputs, input_names, output_names, dynamic_axes mappings, and opset_version.
• The transformers.onnx package eliminates manual dynamic axis mapping by using FeaturesManager and configuration objects to export models.
• Optimum ORTModelForXxx classes convert models automatically via from_transformers=True in from_pretrained, enabling immediate prediction and pipeline usage.

Related reading