Graphcore and Hugging Face Launch New Lineup of IPU-Ready Transformers

Summary

Graphcore and Hugging Face expanded the range of machine learning modalities and tasks available in Hugging Face Optimum. Developers can now access ten transformer models optimized for Graphcore IPUs across natural language processing, speech, and computer vision. The available architectures include BERT, ViT, GPT-2, RoBERTa, DeBERTa, BART, LXMERT, T5, HuBERT, and Wav2Vec2, complete with IPU configuration files and ready-to-use pre-trained or fine-tuned weights. The integration supports the Bow IPU processor, which uses 3D Wafer-on-Wafer stacking to achieve up to 350 teraFLOPS of AI compute. Optimum also integrates with the Poplar SDK 2.5, enabling compatibility with frameworks such as PyTorch, TensorFlow, Docker, and Kubernetes.

Context

Real-world transformer performance optimization requires substantial time, effort, and specialized skills that are beyond the reach of many companies.

Approach / What changed

Hugging Face and Graphcore integrated ten transformer model architectures across vision, speech, and NLP into the open-source Hugging Face Optimum library, providing IPU configuration files, pre-trained weights, and seamless compatibility with Graphcore's Bow IPU and Poplar SDK 2.5.

Takeaways

• The expanded Hugging Face Optimum library supports ten IPU-optimized transformer models across computer vision, NLP, and speech domains, including ViT, GPT-2, RoBERTa, DeBERTa, BART, LXMERT, T5, HuBERT, and Wav2Vec2.
• Graphcore's Bow IPU uses Wafer-on-Wafer 3D stacking to deliver up to 350 teraFLOPS of AI compute, offering up to 40% higher performance and 16% better power efficiency than its previous generation without code changes.
• Poplar SDK 2.5 integrates with PyTorch, PyTorch Lightning, TensorFlow, Docker, and Kubernetes, enabling developers to port models to IPUs from other compute platforms.

Related reading

• huggingface.co · Sep 14, 2021

  Hugging Face and Graphcore partner for IPU-optimized Transformers

  Hugging Face launched its Hardware Partner Program at the 2021 AI Hardware Summit to offer device-optimized models and software integrations. As a founding member of the program, Graphcore collaborated with Hugging Face to optimize Transformer models for its Intelligence Processing Unit (IPU). Graphcore IPUs use a massively parallel MIMD architecture with on-die ultra-high bandwidth memory designed for AI workloads featuring fine-grained parallelism, low precision arithmetic, and sparsity. Through Hugging Face's open-source Optimum toolkit, developers will receive plug-and-play access to certified, hardware-optimized models across vision, speech, translation, and text generation. Graphcore's Poplar SDK integrates with PyTorch, TensorFlow, Docker, and Kubernetes, simplifying the process of porting workloads to IPU-POD datacenter compute systems.

  Sally Doherty

• huggingface.co · Nov 30, 2021

  Getting Started with Hugging Face Transformers for IPUs with Optimum

  Transformer prediction speed can hinder latency-sensitive workloads like conversational systems and search, while optimizing hardware performance requires significant effort. Hugging Face introduced Optimum, an open-source library that reduces Transformer model latency across diverse hardware targets. Through a partnership with Graphcore, BERT is introduced as the initial IPU-optimized model for parallel AI hardware. Developers configure the Poplar SDK environment and PopTorch inside Graphcloud, install optimum-graphcore, and execute the run_qa.py script with an IPU configuration file. Fine-tuning bert-base-uncased on SQuAD v1.1 using the IPUTrainer class yields an evaluation F1 score of 88.2757 and an exact match score of 80.6623 across 10,784 evaluation samples.