Welcome Stable-baselines3 to the Hugging Face Hub 🤗

Summary

Hugging Face announced an official integration with Stable-Baselines3, a popular PyTorch library for training and testing Deep Reinforcement Learning agents across diverse environments like Gym, Atari, MuJoco, and Procgen. The integration enables researchers and developers to host their saved reinforcement learning checkpoints on the Hugging Face Hub and download pre-trained community models. Interacting with the Hub requires installing the huggingface_hub and huggingface_sb3 packages, which supply helper methods for authentication, downloading, and uploading. Practitioners can retrieve checkpoint zip files using the load_from_hub function by providing the target repository identifier and filename before loading them into Stable-Baselines3 algorithms. Furthermore, users authenticated via CLI or notebook login can train policies such as PPO and publish their saved zip files to the Hub using push_to_hub.

Context

Hugging Face aimed to contribute to the Deep Reinforcement Learning ecosystem by enabling researchers and enthusiasts to host and share models trained with Stable-Baselines3.

Approach / What changed

Hugging Face created the huggingface_sb3 library alongside huggingface_hub to provide load_from_hub and push_to_hub functions, allowing users to pull saved checkpoint zip files directly into Stable-Baselines3 algorithms and push trained model files to Hub repositories.

Takeaways

• The huggingface_sb3 library provides load_from_hub to fetch model zip files from the Hugging Face Hub using a repository ID and filename.
• Users must authenticate via notebook_login or huggingface-cli before pushing saved reinforcement learning models to the Hub with push_to_hub.
• Hugging Face plans future integrations including RL-baselines3-zoo, pre-trained RL-trained-agents collections, and Decision Transformers.

Related reading

• huggingface.co · Jul 13, 2021

  Welcome spaCy to the Hugging Face Hub

  Hugging Face has integrated spaCy pipelines into the Hugging Face Hub to simplify the distribution, testing, and deployment of natural language processing models across the community. More than 60 canonical models from the spaCy 3.1 release are hosted in the spaCy organization, alongside discoverable community pipelines. Users can package and upload models directly through the command line using the spacy-huggingface-hub extension, which auto-generates required metadata and model cards. Hosted pipelines offer direct pip installation from live wheel URLs, interactive browser exploration, and production-ready HTTP endpoints through the Inference API. The platform provides immediate interactive widget support for named entity recognition components, with upcoming support planned for text classification and part-of-speech tagging pipelines.

  Omar Sanseviero, Ines Montani

• huggingface.co · Mar 28, 2022

  Introducing Decision Transformers on Hugging Face 🤗

  Hugging Face has integrated the Decision Transformer architecture into its transformers library and Hub, expanding support for offline reinforcement learning. Unlike standard reinforcement learning methods that maximize returns by learning value functions or policies through active environment interaction, Decision Transformers model trajectories as conditional sequence problems. The model feeds past states, actions, and target returns-to-go over a context window into a GPT-2 autoregressive backbone to generate future actions. Evaluating pre-trained continuous control checkpoints requires standardizing observations with training set statistics and iteratively updating the target return based on received rewards. This sequence modeling paradigm enables practitioners to adjust agent behaviors and skill levels directly by specifying different target returns at runtime.