# Graphcore and Hugging Face Launch New Lineup of IPU-Ready Transformers

huggingface.co · Sally Doherty · May 26, 2022

**Type:** Announcement

## Summary

Graphcore and Hugging Face expanded the range of machine learning modalities and tasks available in Hugging Face Optimum. Developers can now access ten transformer models optimized for Graphcore IPUs across natural language processing, speech, and computer vision. The available architectures include BERT, ViT, GPT-2, RoBERTa, DeBERTa, BART, LXMERT, T5, HuBERT, and Wav2Vec2, complete with IPU configuration files and ready-to-use pre-trained or fine-tuned weights. The integration supports the Bow IPU processor, which uses 3D Wafer-on-Wafer stacking to achieve up to 350 teraFLOPS of AI compute. Optimum also integrates with the Poplar SDK 2.5, enabling compatibility with frameworks such as PyTorch, TensorFlow, Docker, and Kubernetes.

## Context

Real-world transformer performance optimization requires substantial time, effort, and specialized skills that are beyond the reach of many companies.

## Approach / What changed

Hugging Face and Graphcore integrated ten transformer model architectures across vision, speech, and NLP into the open-source Hugging Face Optimum library, providing IPU configuration files, pre-trained weights, and seamless compatibility with Graphcore's Bow IPU and Poplar SDK 2.5.

## Takeaways

- The expanded Hugging Face Optimum library supports ten IPU-optimized transformer models across computer vision, NLP, and speech domains, including ViT, GPT-2, RoBERTa, DeBERTa, BART, LXMERT, T5, HuBERT, and Wav2Vec2.
- Graphcore's Bow IPU uses Wafer-on-Wafer 3D stacking to deliver up to 350 teraFLOPS of AI compute, offering up to 40% higher performance and 16% better power efficiency than its previous generation without code changes.
- Poplar SDK 2.5 integrates with PyTorch, PyTorch Lightning, TensorFlow, Docker, and Kubernetes, enabling developers to port models to IPUs from other compute platforms.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://huggingface.co/blog/graphcore-update)