Deploy Hugging Face models easily with Amazon SageMaker

Summary

Hugging Face and Amazon introduced an inference solution integrating Hugging Face Transformers with Amazon SageMaker. The SageMaker Hugging Face Inference Toolkit and specialized Deep Learning Containers enable deployment of trained models or publicly available Hub models to managed production endpoints. Users configure endpoints with minimal code using the SageMaker Python SDK, standard pipelines, or custom inference scripts that override default behaviors. The architecture supports models stored in Amazon S3 as well as direct references via HF_MODEL_ID and HF_TASK environment variables. Deployments benefit from native AWS infrastructure features, including built-in monitoring, Identity and Access Management permission controls, and Virtual Private Cloud connectivity.

Takeaways

1. 1

   The SageMaker Hugging Face Inference Toolkit uses transformers pipelines to enable zero-code inference deployments without custom pre- or post-processing code.

2. 2

   Models can be deployed directly from the Hugging Face Model Hub by supplying HF_MODEL_ID and HF_TASK environment variables.

3. 3

   Deployments can be instantiated using the SageMaker Python SDK, boto3, AWS CLI, Terraform, or CloudFormation templates.