Getting Started with Hugging Face Transformers for IPUs with Optimum

Summary

Transformer prediction speed can hinder latency-sensitive workloads like conversational systems and search, while optimizing hardware performance requires significant effort. Hugging Face introduced Optimum, an open-source library that reduces Transformer model latency across diverse hardware targets. Through a partnership with Graphcore, BERT is introduced as the initial IPU-optimized model for parallel AI hardware. Developers configure the Poplar SDK environment and PopTorch inside Graphcloud, install optimum-graphcore, and execute the run_qa.py script with an IPU configuration file. Fine-tuning bert-base-uncased on SQuAD v1.1 using the IPUTrainer class yields an evaluation F1 score of 88.2757 and an exact match score of 80.6623 across 10,784 evaluation samples.

Takeaways

1. 1

   Optimum serves as the interface between Hugging Face Transformers and Graphcore IPUs, enabling training and fine-tuning with minimal code modifications via PopTorch and IPUTrainer.

2. 2

   The Optimum Graphcore question-answering fine-tuning script run_qa.py requires a fast tokenizer backed by the Hugging Face Tokenizers library and an ipu_config.json configuration file.

3. 3

   Fine-tuning bert-base-uncased on SQuAD v1.1 for 3 epochs on the IPU setup completed in 368.40 seconds, achieving an evaluation F1 score of 88.2757 and exact match of 80.6623.