---
title: "How to price serverless GPUs"
description: "Modal’s engineering blog presents a cost model comparing serverless and reserved GPU pricing for AI workloads, centered on the peak-to-average demand ratio. Serverless GPUs charge only for usage, while reserved GPUs require long-term contracts for fixed capacity, often achieving under 30% utilization. The model shows that when the peak-to-average ratio exceeds the reservation discount rate (typically 2-5x), serverless GPUs become cheaper despite higher base rates, with real workloads like inference and training having ratios of 5-10x, making serverless an obvious cost win. An interactive tool allows teams to estimate costs using preset workloads or custom traffic patterns, defaulting to current B200 GPU rates on Modal and hyperscaler reservations. The post discusses model limitations, including assumptions of perfect prediction and instant allocation, and notes that serverless also reduces engineering costs through faster development and reduced dev/prod skew, while mixed strategies can combine baseline reservations with serverless bursting for high peak-to-average bursts."
---

# How to price serverless GPUs

[Modal](https://yomu.fyi/company/modal) · Charles Frye · Jul 6, 2026

**Type:** Explainer

## Summary

Modal’s engineering blog presents a cost model comparing serverless and reserved GPU pricing for AI workloads, centered on the peak-to-average demand ratio. Serverless GPUs charge only for usage, while reserved GPUs require long-term contracts for fixed capacity, often achieving under 30% utilization. The model shows that when the peak-to-average ratio exceeds the reservation discount rate (typically 2-5x), serverless GPUs become cheaper despite higher base rates, with real workloads like inference and training having ratios of 5-10x, making serverless an obvious cost win. An interactive tool allows teams to estimate costs using preset workloads or custom traffic patterns, defaulting to current B200 GPU rates on Modal and hyperscaler reservations. The post discusses model limitations, including assumptions of perfect prediction and instant allocation, and notes that serverless also reduces engineering costs through faster development and reduced dev/prod skew, while mixed strategies can combine baseline reservations with serverless bursting for high peak-to-average bursts.

## Context

The post addresses the tension between finance teams seeking to control GPU costs and engineering teams needing flexible GPU access for AI workloads. It presents a cost model comparing serverless GPUs (pay-per-use) with reserved GPUs (long-term contracts), using the peak-to-average demand ratio as the key metric. The model is accompanied by an interactive tool for teams to estimate costs based on their traffic patterns.

## Approach / What changed

The post provides a mathematical cost model comparing total cost of serverless GPUs (rate times demand over time) versus reserved GPUs (rate times peak demand times reservation length). It introduces the peak-to-average ratio as the deciding factor: serverless is cheaper when this ratio exceeds the reservation discount rate. An interactive widget allows users to select preset workloads or draw custom traffic patterns to compute costs.

## Takeaways

- The peak-to-average demand ratio is the critical parameter: if it exceeds the reservation discount rate, serverless GPUs are cheaper.
- Real AI workloads like inference and training have peak-to-average ratios of 5-10x, while reservation discounts are typically 2-5x, making serverless cost-effective.
- The model assumes perfect prediction and instant allocation; in practice, prediction errors lead to SLO violations or waste, and serverless autoscaling can be supplemented with warm pools.

## FAQ

### How do serverless GPU costs compare to reserved GPU costs?

Serverless GPUs charge only for usage, while reserved GPUs require paying for peak capacity throughout the contract. The comparison depends on the peak-to-average ratio: if it exceeds the reservation discount rate, serverless is cheaper. For typical AI workloads with ratios of 5-10x and discounts of 2-5x, serverless wins on cost.

### What is the peak-to-average ratio and why does it matter?

The peak-to-average ratio is the maximum GPU demand divided by the average demand. High ratios mean you need many GPUs for short bursts but few on average. With reservations, you must pay for the peak constantly, leading to low utilization and waste. Serverless GPUs only charge for actual usage, making them cheaper for high peak-to-average workloads.

### What are the limitations of the cost model?

The model assumes perfect demand prediction and instant GPU allocation. In reality, prediction errors can cause SLO violations or waste, and serverless autoscaling may need warm pools for speed. The model also assumes all-or-none strategies, but mixed approaches using reservations for baseline and serverless for bursts are common.

**Tags:** [Cost Optimization](https://yomu.fyi/topic/cost-optimization), [GPU](https://yomu.fyi/topic/gpu), [Pricing](https://yomu.fyi/topic/pricing), [Serverless](https://yomu.fyi/topic/serverless)

- Source: [Modal](https://modal.com/blog/how-to-price-serverless)
- Source URL: https://modal.com/blog/how-to-price-serverless
- Ingested by Yomu: 2026-09-20T07:47:10.276Z

[Read original post](https://modal.com/blog/how-to-price-serverless)