Introducing Snowball Fight ☃️, our first ML-Agents environment

Summary

Hugging Face has released Snowball Fight 1vs1, its first custom Deep Reinforcement Learning environment created using the Unity Machine Learning Agents Toolkit. The game enables players to shoot snowballs against a trained agent and is hosted on Hugging Face Spaces. This release marks the beginning of an effort to build an ecosystem for Deep Reinforcement Learning researchers and enthusiasts using Unity ML-Agents. The initiative focuses on building open-source environments, hosting models and training environments on the Hugging Face Hub, and deploying interactive demos via Spaces. Future plans include publishing technical tutorials, releasing additional environments, and developing a Snowball Fight 2vs2 version that utilizes the MA-POCA algorithm to train cooperative team behaviors.

Context

Deep Reinforcement Learning researchers and enthusiasts using Unity ML-Agents need dedicated ecosystem support to build, share, and host custom environments and interactive demos.

Approach / What changed

Hugging Face integrated Unity ML-Agents with its platform to allow hosting environments and models on the Hugging Face Hub, deploying demos on Hugging Face Spaces, and creating custom open-source training environments starting with Snowball Fight 1vs1.

Takeaways

• The Unity Machine Learning Agents Toolkit is an open-source library used to build games and simulations in Unity that serve as training environments for intelligent agents.
• Hugging Face provides infrastructure to host custom Unity ML-Agents environments, save trained models to the Hub, and run playable demos on Spaces.
• An upcoming Snowball Fight 2vs2 environment will use the MA-POCA Deep Reinforcement Learning algorithm to train cooperative team behaviors.

Related reading

• huggingface.co · Jan 21, 2022

  Welcome Stable-baselines3 to the Hugging Face Hub 🤗

  Hugging Face announced an official integration with Stable-Baselines3, a popular PyTorch library for training and testing Deep Reinforcement Learning agents across diverse environments like Gym, Atari, MuJoco, and Procgen. The integration enables researchers and developers to host their saved reinforcement learning checkpoints on the Hugging Face Hub and download pre-trained community models. Interacting with the Hub requires installing the huggingface_hub and huggingface_sb3 packages, which supply helper methods for authentication, downloading, and uploading. Practitioners can retrieve checkpoint zip files using the load_from_hub function by providing the target repository identifier and filename before loading them into Stable-Baselines3 algorithms. Furthermore, users authenticated via CLI or notebook login can train policies such as PPO and publish their saved zip files to the Hub using push_to_hub.

  Thomas Simonini

• huggingface.co · Mar 28, 2022

  Introducing Decision Transformers on Hugging Face 🤗

  Hugging Face has integrated the Decision Transformer architecture into its transformers library and Hub, expanding support for offline reinforcement learning. Unlike standard reinforcement learning methods that maximize returns by learning value functions or policies through active environment interaction, Decision Transformers model trajectories as conditional sequence problems. The model feeds past states, actions, and target returns-to-go over a context window into a GPT-2 autoregressive backbone to generate future actions. Evaluating pre-trained continuous control checkpoints requires standardizing observations with training set statistics and iteratively updating the target return based on received rewards. This sequence modeling paradigm enables practitioners to adjust agent behaviors and skill levels directly by specifying different target returns at runtime.