Heroku AI Expands Model Offering with OpenAI’s gpt-oss-120b

Summary

Heroku has made OpenAI’s open-weight gpt-oss-120b available through Heroku Managed Inference and Agents, giving developers an OpenAI-compatible chat completions API for compatible SDKs and frameworks. The model uses a Mixture-of-Experts architecture with 117 billion total parameters, activating 5.1 billion per token and enabling operation on a single 80GB GPU; its Apache 2.0 license permits inspection and fine-tuning. It is designed for agentic workflows involving instruction following, function calling, web searches, and Python execution, while reported benchmarks place it near or above several open-weight models and sometimes OpenAI o4-mini. The add-on provides deployment, secure application access, agent actions, monitoring, and scaling without infrastructure management, with pricing of $0.15 per million input tokens and $0.60 per million output tokens.

Takeaways

1. 1

   gpt-oss-120b has 117 billion total parameters but activates 5.1 billion per token through its Mixture-of-Experts architecture, allowing it to run on a single 80GB GPU.

2. 2

   The model’s reported benchmarks include 90.0% on MMLU, 97.9% on AIME 2025 with tools, and a 2463 Elo rating on Codeforces without tools.

3. 3

   Heroku lists pricing at $0.15 per million input tokens and $0.60 per million output tokens; the model is available through the Elements Marketplace add-on.