# Deploy Hugging Face models easily with Amazon SageMaker

huggingface.co · Philipp Schmid · Jul 8, 2021

**Type:** Announcement

## Summary

Hugging Face and Amazon introduced an inference solution integrating Hugging Face Transformers with Amazon SageMaker. The SageMaker Hugging Face Inference Toolkit and specialized Deep Learning Containers enable deployment of trained models or publicly available Hub models to managed production endpoints. Users configure endpoints with minimal code using the SageMaker Python SDK, standard pipelines, or custom inference scripts that override default behaviors. The architecture supports models stored in Amazon S3 as well as direct references via HF\_MODEL\_ID and HF\_TASK environment variables. Deployments benefit from native AWS infrastructure features, including built-in monitoring, Identity and Access Management permission controls, and Virtual Private Cloud connectivity.

## Context

Deploying transformer-based deep learning models to scalable, secure production endpoints often involves complex environment configuration, serving stack maintenance, and custom pre- and post-processing code.

## Approach / What changed

Hugging Face created Inference Deep Learning Containers and a SageMaker Inference Toolkit that leverage transformers pipelines for zero-code deployments from Amazon S3 or the Hugging Face Model Hub, while also supporting custom inference code.

## Takeaways

- The SageMaker Hugging Face Inference Toolkit uses transformers pipelines to enable zero-code inference deployments without custom pre- or post-processing code.
- Models can be deployed directly from the Hugging Face Model Hub by supplying HF\_MODEL\_ID and HF\_TASK environment variables.
- Deployments can be instantiated using the SageMaker Python SDK, boto3, AWS CLI, Terraform, or CloudFormation templates.

**Tags:** [AWS](https://yomu.fyi/topic/aws), [Deployment](https://yomu.fyi/topic/deployment), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/deploy-hugging-face-models-easily-with-amazon-sagemaker)