Hugging Face and Graphcore partner for IPU-optimized Transformers

Summary

Hugging Face launched its Hardware Partner Program at the 2021 AI Hardware Summit to offer device-optimized models and software integrations. As a founding member of the program, Graphcore collaborated with Hugging Face to optimize Transformer models for its Intelligence Processing Unit (IPU). Graphcore IPUs use a massively parallel MIMD architecture with on-die ultra-high bandwidth memory designed for AI workloads featuring fine-grained parallelism, low precision arithmetic, and sparsity. Through Hugging Face's open-source Optimum toolkit, developers will receive plug-and-play access to certified, hardware-optimized models across vision, speech, translation, and text generation. Graphcore's Poplar SDK integrates with PyTorch, TensorFlow, Docker, and Kubernetes, simplifying the process of porting workloads to IPU-POD datacenter compute systems.

Context

Developers seeking to harness hardware acceleration for Transformer models face challenges with coding complexity and learning new software workflows when targeting specialized AI hardware instead of GPUs.

Approach / What changed

Graphcore joined Hugging Face's Hardware Partner Program to release certified, IPU-optimized Transformer models via the open-source Optimum library, supported by the Poplar SDK's integrations with PyTorch, TensorFlow, Docker, and Kubernetes.

Takeaways

• Graphcore IPUs use a massively parallel MIMD architecture with ultra-high bandwidth on-die memory, contrasting with traditional SIMD/SIMT GPU architectures.
• Hugging Face introduced Optimum, an open-source library and toolkit providing developers with plug-and-play access to hardware-optimized models.
• Graphcore's Poplar SDK integrates directly with standard frameworks such as PyTorch and TensorFlow alongside orchestration tools like Docker and Kubernetes.

Related reading

• huggingface.co · May 26, 2022

  Graphcore and Hugging Face Launch New Lineup of IPU-Ready Transformers

  Graphcore and Hugging Face expanded the range of machine learning modalities and tasks available in Hugging Face Optimum. Developers can now access ten transformer models optimized for Graphcore IPUs across natural language processing, speech, and computer vision. The available architectures include BERT, ViT, GPT-2, RoBERTa, DeBERTa, BART, LXMERT, T5, HuBERT, and Wav2Vec2, complete with IPU configuration files and ready-to-use pre-trained or fine-tuned weights. The integration supports the Bow IPU processor, which uses 3D Wafer-on-Wafer stacking to achieve up to 350 teraFLOPS of AI compute. Optimum also integrates with the Poplar SDK 2.5, enabling compatibility with frameworks such as PyTorch, TensorFlow, Docker, and Kubernetes.

  Sally Doherty

• huggingface.co · Nov 30, 2021

  Getting Started with Hugging Face Transformers for IPUs with Optimum

  Transformer prediction speed can hinder latency-sensitive workloads like conversational systems and search, while optimizing hardware performance requires significant effort. Hugging Face introduced Optimum, an open-source library that reduces Transformer model latency across diverse hardware targets. Through a partnership with Graphcore, BERT is introduced as the initial IPU-optimized model for parallel AI hardware. Developers configure the Poplar SDK environment and PopTorch inside Graphcloud, install optimum-graphcore, and execute the run_qa.py script with an IPU configuration file. Fine-tuning bert-base-uncased on SQuAD v1.1 using the IPUTrainer class yields an evaluation F1 score of 88.2757 and an exact match score of 80.6623 across 10,784 evaluation samples.