# Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training

huggingface.co · Susan Lansing · Apr 12, 2022

**Type:** Announcement

## Summary

Training transformer models across computer vision, speech, and natural language processing tasks at scale often demands heavy compute resources, incurring significant time and financial expense. To address these bottlenecks, Habana Labs and Hugging Face partnered to integrate the SynapseAI software suite into the Hugging Face Optimum open-source library. This integration allows machine learning practitioners to accelerate transformer training workflows on Habana Gaudi processors using minimal code adjustments. Habana Gaudi hardware, featured in Amazon EC2 DL1 instances and Supermicro X12 servers, incorporates ten 100 Gigabit Ethernet ports per processor to scale from single units to thousands of chips. The combined hardware and software architecture supports TensorFlow and PyTorch while delivering price and performance metrics up to 40% lower than comparable training alternatives.

## Context

Transformer models deliver state-of-the-art performance in natural language processing, computer vision, and speech, but training them at scale requires massive computing power, leading to long, complex, and expensive workflows.

## Approach / What changed

Habana Labs and Hugging Face integrated the SynapseAI software suite with the Hugging Face Optimum open-source library, enabling engineers to execute transformer training on Habana Gaudi hardware with minimal code modifications.

## Takeaways

- Habana Gaudi processors deliver price and performance metrics up to 40% lower than comparable deep learning training solutions.
- Each Gaudi processor includes ten integrated 100 Gigabit Ethernet ports, facilitating scaling from single processors to thousands of chips.
- The integration connects Habana SynapseAI with Hugging Face Optimum, supporting PyTorch and TensorFlow frameworks with minimal code changes.

**Tags:** [AWS](https://yomu.fyi/topic/aws), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://huggingface.co/blog/habana)