---
title: "Faster TensorFlow models in Hugging Face Transformers"
description: "Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference."
---

# Faster TensorFlow models in Hugging Face Transformers

huggingface.co · Julien Plu · Jan 26, 2021

**Type:** Tutorial

## Summary

Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference.

## Context

Hugging Face aimed to make TensorFlow models in the Transformers library faster, more robust, and easier to deploy in production environments using TensorFlow Serving across CPU, GPU, and TPU hardware.

## Approach / What changed

Transformers updated core architectures including BERT, RoBERTa, ELECTRA, and MPNet, enhanced SavedModel export capabilities with flexible sequence lengths and unified outputs, and enabled deployment through Docker-based TensorFlow Serving over HTTP and gRPC.

## Takeaways

- In Transformers v4.2.0 benchmarks on a V100 GPU with sequence length 128, BERT inference with TensorFlow Serving is up to roughly 10 percent faster than Google's implementation and twice as fast as Transformers v4.1.1.
- Transformers v4.2.0 SavedModel export allows sequence lengths to vary freely between runs, makes all model inputs available for inference, and groups hidden states or attentions into single output tensors.
- TensorFlow models exported as SavedModels can be deployed via Docker containers running TensorFlow Serving and queried using either REST endpoints on port 8501 or gRPC on port 8500.

**Tags:** [Docker](https://yomu.fyi/topic/docker), [gRPC](https://yomu.fyi/topic/grpc), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [REST APIs](https://yomu.fyi/topic/rest-api)

- Source: [huggingface.co](https://huggingface.co/blog/tf-serving)
- Source URL: https://huggingface.co/blog/tf-serving
- Ingested by Yomu: 2026-08-27T15:05:50.930Z

[Read original post](https://huggingface.co/blog/tf-serving)