# Getting Started with Hugging Face Transformers for IPUs with Optimum

huggingface.co · Tim Santos, Julien Simon · Nov 30, 2021

**Type:** Tutorial

## Summary

Transformer prediction speed can hinder latency-sensitive workloads like conversational systems and search, while optimizing hardware performance requires significant effort. Hugging Face introduced Optimum, an open-source library that reduces Transformer model latency across diverse hardware targets. Through a partnership with Graphcore, BERT is introduced as the initial IPU-optimized model for parallel AI hardware. Developers configure the Poplar SDK environment and PopTorch inside Graphcloud, install optimum-graphcore, and execute the run\_qa.py script with an IPU configuration file. Fine-tuning bert-base-uncased on SQuAD v1.1 using the IPUTrainer class yields an evaluation F1 score of 88.2757 and an exact match score of 80.6623 across 10,784 evaluation samples.

## Context

Large Transformer model prediction speeds make them impractical for latency-sensitive applications like search or conversational systems, and optimizing their real-world hardware performance requires specialized skills and effort beyond many organizations' reach.

## Approach / What changed

Graphcore and Hugging Face integrated IPU acceleration into the open-source Optimum library, enabling developers to run BERT on Graphcore IPU-POD16 systems via Poplar SDK, PopTorch, and the optimum-graphcore package using the IPUTrainer class.

## Takeaways

- Optimum serves as the interface between Hugging Face Transformers and Graphcore IPUs, enabling training and fine-tuning with minimal code modifications via PopTorch and IPUTrainer.
- The Optimum Graphcore question-answering fine-tuning script run\_qa.py requires a fast tokenizer backed by the Hugging Face Tokenizers library and an ipu\_config.json configuration file.
- Fine-tuning bert-base-uncased on SQuAD v1.1 for 3 epochs on the IPU setup completed in 368.40 seconds, achieving an evaluation F1 score of 88.2757 and exact match of 80.6623.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/graphcore-getting-started)