Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training

Summary

Training transformer models across computer vision, speech, and natural language processing tasks at scale often demands heavy compute resources, incurring significant time and financial expense. To address these bottlenecks, Habana Labs and Hugging Face partnered to integrate the SynapseAI software suite into the Hugging Face Optimum open-source library. This integration allows machine learning practitioners to accelerate transformer training workflows on Habana Gaudi processors using minimal code adjustments. Habana Gaudi hardware, featured in Amazon EC2 DL1 instances and Supermicro X12 servers, incorporates ten 100 Gigabit Ethernet ports per processor to scale from single units to thousands of chips. The combined hardware and software architecture supports TensorFlow and PyTorch while delivering price and performance metrics up to 40% lower than comparable training alternatives.

Context

Transformer models deliver state-of-the-art performance in natural language processing, computer vision, and speech, but training them at scale requires massive computing power, leading to long, complex, and expensive workflows.

Approach / What changed

Habana Labs and Hugging Face integrated the SynapseAI software suite with the Hugging Face Optimum open-source library, enabling engineers to execute transformer training on Habana Gaudi hardware with minimal code modifications.

Takeaways

• Habana Gaudi processors deliver price and performance metrics up to 40% lower than comparable deep learning training solutions.
• Each Gaudi processor includes ten integrated 100 Gigabit Ethernet ports, facilitating scaling from single processors to thousands of chips.
• The integration connects Habana SynapseAI with Hugging Face Optimum, supporting PyTorch and TensorFlow frameworks with minimal code changes.

Related reading

• huggingface.co · Jun 15, 2022

  Intel and Hugging Face Partner to Democratize Machine Learning Hardware Acceleration

  Intel has joined Hugging Face's Hardware Partner Program to accelerate Transformer training, fine-tuning, and inference on Intel platforms. Large Transformer models introduce latency bottlenecks in production workloads like search and chatbots, where hardware-level optimization typically requires tedious trial and error. To streamline model optimization, the collaboration introduces Optimum Intel, an open-source library integrating the Intel Neural Compressor for automated quantization, pruning, and distillation. A demonstration applies post-training dynamic quantization to a fine-tuned DistilBERT classification model using a CPU-only PyTorch setup. The quantized model converted 38 Linear and 2 Embedding operators to 8-bit integers, reducing evaluation duration by 1.34x while keeping the accuracy drop within a 5% threshold.

  Julien Simon

• huggingface.co · Sep 14, 2021

  Hugging Face and Graphcore partner for IPU-optimized Transformers

  Hugging Face launched its Hardware Partner Program at the 2021 AI Hardware Summit to offer device-optimized models and software integrations. As a founding member of the program, Graphcore collaborated with Hugging Face to optimize Transformer models for its Intelligence Processing Unit (IPU). Graphcore IPUs use a massively parallel MIMD architecture with on-die ultra-high bandwidth memory designed for AI workloads featuring fine-grained parallelism, low precision arithmetic, and sparsity. Through Hugging Face's open-source Optimum toolkit, developers will receive plug-and-play access to certified, hardware-optimized models across vision, speech, translation, and text generation. Graphcore's Poplar SDK integrates with PyTorch, TensorFlow, Docker, and Kubernetes, simplifying the process of porting workloads to IPU-POD datacenter compute systems.