The Partnership: Amazon SageMaker and Hugging Face

Summary

Hugging Face and Amazon formed a strategic partnership establishing Amazon Web Services as Hugging Face's preferred cloud provider to simplify training and deploying Transformer models. To facilitate this collaboration, the teams introduced Hugging Face Deep Learning Containers optimized for PyTorch and TensorFlow in Amazon SageMaker. The integration provides an extension to the SageMaker Python SDK, enabling data science teams to configure distributed training jobs across single-node and multi-node GPU clusters. Workflows integrate directly with SageMaker Automatic Model Tuning, SageMaker Studio, and Amazon S3 for managed experiment tracking and artifact storage. The open-source containers allow trained models to be exported for inference via SageMaker or published directly to the Hugging Face Model Hub while utilizing EC2 Spot instances to reduce compute expenses.

Context

Hugging Face and Amazon established a partnership to simplify the adoption of state-of-the-art machine learning models and accelerate natural language processing feature delivery. Data science teams needed streamlined workflows to configure, train, tune, and deploy Transformers-based NLP models on scalable cloud infrastructure without spending days on environment setup.

Approach / What changed

Hugging Face introduced open-source Deep Learning Containers (DLCs) licensed under Apache 2.0 and integrated them directly into Amazon SageMaker. The integration includes a dedicated extension for the SageMaker Python SDK, built-in support for PyTorch and TensorFlow, compatibility with SageMaker Automatic Model Tuning, and native connectivity to Amazon S3, EC2 Spot instances, and SageMaker distributed training libraries.

Takeaways

• Hugging Face Deep Learning Containers are open-source under Apache 2.0 and provide pre-optimized environments for PyTorch and TensorFlow across single-GPU, multi-GPU, and multi-node clusters.
• The SageMaker Python SDK extension integrates with SageMaker Automatic Model Tuning and SageMaker Studio to automate hyperparameter optimization and experiment tracking.
• SageMaker Training provisions compute only for the duration of the job with per-second billing and supports Amazon EC2 Spot instances for up to 90% cost reduction.

Related reading

• huggingface.co · Apr 8, 2021

  Distributed Training: Train BART/T5 for Summarization using 🤗 Transformers and Amazon SageMaker

  Hugging Face and Amazon SageMaker introduced optimized Deep Learning Containers alongside a SageMaker Python SDK estimator to streamline distributed transformer model training. This workflow demonstrates fine-tuning the 400-million-parameter facebook/bart-large-cnn model for dialogue summarization on the 16,000-conversation samsum dataset. Using SageMaker Data Parallelism enabled in the distribution configuration, training executes across two ml.p3dn.24xlarge compute instances equipped with sixteen total GPUs and a cumulative batch size of 64. The HuggingFace estimator pulls the run_summarization.py script directly from GitHub, enables mixed precision with fp16, and tracks evaluation performance through ROUGE scores. Following completion, the unzipped model artifacts and a generated model card are pushed to the Hugging Face Hub using the huggingface_hub library for hosted inference.

  Philipp Schmid

• huggingface.co · Jul 8, 2021

  Deploy Hugging Face models easily with Amazon SageMaker

  Hugging Face and Amazon introduced an inference solution integrating Hugging Face Transformers with Amazon SageMaker. The SageMaker Hugging Face Inference Toolkit and specialized Deep Learning Containers enable deployment of trained models or publicly available Hub models to managed production endpoints. Users configure endpoints with minimal code using the SageMaker Python SDK, standard pipelines, or custom inference scripts that override default behaviors. The architecture supports models stored in Amazon S3 as well as direct references via HF_MODEL_ID and HF_TASK environment variables. Deployments benefit from native AWS infrastructure features, including built-in monitoring, Identity and Access Management permission controls, and Virtual Private Cloud connectivity.