# Machine Learning Experts - Lewis Tunstall

huggingface.co · Britney Muller · Apr 13, 2022

**Type:** Explainer

## Summary

In this interview with Britney Muller, Hugging Face machine learning engineer Lewis Tunstall discusses his background, his book Natural Language Processing with Transformers, and tooling optimizations for deploying transformer models. Tunstall notes that large model parameter sizes often lead to high latency, making off-the-shelf transformers impractical for responsive applications like chatbots. To resolve this issue, he developed single-line export tooling within the transformers library to convert models into the ONNX format. This conversion enables engineers to run PyTorch models in TensorFlow or on specialized hardware, achieving faster latency and higher throughput. Tunstall also recounts co-authoring the O'Reilly book with Leandro von Werra and Thom Wolf, collaborating on the Hugging Face Course, and training a GPT-2 model named code parrot on Python code.

## Context

Off-the-shelf transformer models possess large parameter counts that cause high latency and sluggish responses, presenting significant deployment challenges for production applications such as chatbots.

## Approach / What changed

Tunstall implemented a single-line export feature in the transformers library that converts PyTorch models into the ONNX format, allowing deployment to TensorFlow or dedicated hardware to optimize latency and throughput.

## Takeaways

- Transformer parameter sizes cause latency issues in production applications like chatbots when using unoptimized models.
- The Hugging Face transformers library added one-line export functionality to convert PyTorch models to the ONNX format for deployment on dedicated hardware.
- Tunstall co-authored the book Natural Language Processing with Transformers after identifying an information gap in learning materials for the architecture.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/lewis-tunstall-interview)