Machine Learning Experts - Lewis Tunstall

Summary

In this interview with Britney Muller, Hugging Face machine learning engineer Lewis Tunstall discusses his background, his book Natural Language Processing with Transformers, and tooling optimizations for deploying transformer models. Tunstall notes that large model parameter sizes often lead to high latency, making off-the-shelf transformers impractical for responsive applications like chatbots. To resolve this issue, he developed single-line export tooling within the transformers library to convert models into the ONNX format. This conversion enables engineers to run PyTorch models in TensorFlow or on specialized hardware, achieving faster latency and higher throughput. Tunstall also recounts co-authoring the O'Reilly book with Leandro von Werra and Thom Wolf, collaborating on the Hugging Face Course, and training a GPT-2 model named code parrot on Python code.

Context

Off-the-shelf transformer models possess large parameter counts that cause high latency and sluggish responses, presenting significant deployment challenges for production applications such as chatbots.

Approach / What changed

Tunstall implemented a single-line export feature in the transformers library that converts PyTorch models into the ONNX format, allowing deployment to TensorFlow or dedicated hardware to optimize latency and throughput.

Takeaways

• Transformer parameter sizes cause latency issues in production applications like chatbots when using unoptimized models.
• The Hugging Face transformers library added one-line export functionality to convert PyTorch models to the ONNX format for deployment on dedicated hardware.
• Tunstall co-authored the book Natural Language Processing with Transformers after identifying an information gap in learning materials for the architecture.

Related reading

• huggingface.co · Jun 28, 2022

  Announcing Evaluation on the Hub

  Hugging Face introduced Evaluation on the Hub, a no-code tool designed to streamline the evaluation of machine learning models across diverse datasets. Traditional evaluation approaches often suffer from reproducibility issues, implementation inconsistencies, and cumbersome workflows across varied metrics and datasets. Powered by AutoTrain and Hugging Face Spaces, the service allows users to configure tasks, map dataset columns, select evaluation metrics, and run tests directly from dataset pages. Completed evaluations automatically open pull requests on the respective model cards to encode standardized verification metadata and update public dataset leaderboards. This system establishes consistent benchmarking pipelines across tasks like image classification, text summarization, and named entity recognition without requiring local code execution.

  Lewis Tunstall, Abhishek, Tristan Thrush, Sasha Luccioni, Leandro von Werra, Nazneen Rajani, Aleksandra Piktus, Omar Sanseviero, Douwe Kiela

• huggingface.co · Mar 23, 2022

  Machine Learning Experts - Margaret Mitchell

  Machine learning models often internalize harmful dataset skews and fail to evaluate real-world contexts accurately. During work on vision-to-language generation, visual recognition systems learned to misinterpret destructive disasters as positive scenes because training images predominantly featured sunsets and fireworks. Addressing these structural flaws requires moving beyond benchmark optimization toward critical data analysis, dataset genealogies, and standardized ethical AI protocols. ML organizations must establish shared vocabularies for power differentials and dismantle competitive cultural norms that marginalize underrepresented contributors. Furthermore, lowering technical barriers—such as enabling non-engineers to inspect data without SQL—allows diverse stakeholders to evaluate models and prevent widening societal power divides.