Getting Started with Transformers on Habana Gaudi

Summary

Habana Labs and Hugging Face partnered to accelerate Transformer model training using Habana Gaudi accelerators, which offer up to 40% better price performance than recent GPU-based Amazon EC2 instances. Setting up the environment requires launching an Amazon EC2 dl1.24xlarge instance using the Habana Deep Learning Base AMI on Ubuntu 20.04. Inside an interactive PyTorch Docker container configured for Habana runtimes, users install the Optimum Habana package from source along with necessary dependencies. Fine-tuning the bert-large-uncased-whole-word-masking model on the GLUE MRPC task completes in 2 minutes and 12 seconds with an F1 score of 0.9181. Utilizing EC2 Spot Instances reduces the hourly compute cost by 70%, from $13.11 down to $3.93.

Context

Habana Labs and Hugging Face formed a partnership to accelerate Transformer model training, leveraging Habana Gaudi processors that provide up to 40% better price performance over GPU-based Amazon EC2 instances.

Approach / What changed

The author provisions an Amazon EC2 dl1.24xlarge Spot Instance with 50GB EBS storage using the Habana Deep Learning Base AMI, runs a prebuilt Habana PyTorch container, installs Optimum Habana from source, and executes the GLUE MRPC text classification script using bert-large-uncased-whole-word-masking.

Takeaways

• Amazon EC2 DL1 instances contain 8 Habana Gaudi processors and are available exclusively in the dl1.24xlarge instance size.
• Using EC2 Spot Instances for the dl1.24xlarge instance reduced running costs by 70%, lowering the hourly rate from $13.11 to $3.93.
• Fine-tuning bert-large-uncased-whole-word-masking on the GLUE MRPC benchmark with Optimum Habana took 2 minutes and 12 seconds and yielded an F1 score of 0.9181.

Related reading