---
title: "From post-training to production: How TinyFish runs its web-native model on Databricks AI Runtime"
description: "TinyFish builds infrastructure for enterprises that need agents to work across the live web and is developing its own specialized web-native model. The company uses Databricks AI Runtime to manage GPU compute for post-training with supervised fine-tuning and moving toward Direct Preference Optimization, MLflow for evaluation, and Model Serving for production deployment. TinyFish evaluates its model by measuring full agent task completion rather than individual responses, achieving a 77.6% success rate on its internal go/no-go benchmark where 1,691 of 2,178 tasks were completed successfully. Over a 15-day period, the web-native model handled 294,781 agent runs across 6.4 million tool steps at an estimated serving cost of $0.088 per run or $0.0040 per tool step. By consolidating training, evaluation, serving, and operational data on Databricks, TinyFish reduces infrastructure overhead and maintains a continuous development cycle from agent behavior to deployment."
---

# From post-training to production: How TinyFish runs its web-native model on Databricks AI Runtime

[Databricks](https://yomu.fyi/company/databricks) · Sudheesh Nair, Zach Richardson, Julia Powell, Tejas Sundaresan · Oct 12, 2026

**Type:** Problem & solution

## Summary

TinyFish builds infrastructure for enterprises that need agents to work across the live web and is developing its own specialized web-native model. The company uses Databricks AI Runtime to manage GPU compute for post-training with supervised fine-tuning and moving toward Direct Preference Optimization, MLflow for evaluation, and Model Serving for production deployment. TinyFish evaluates its model by measuring full agent task completion rather than individual responses, achieving a 77.6% success rate on its internal go/no-go benchmark where 1,691 of 2,178 tasks were completed successfully. Over a 15-day period, the web-native model handled 294,781 agent runs across 6.4 million tool steps at an estimated serving cost of $0.088 per run or $0.0040 per tool step. By consolidating training, evaluation, serving, and operational data on Databricks, TinyFish reduces infrastructure overhead and maintains a continuous development cycle from agent behavior to deployment.

## Context

TinyFish provides APIs for large-scale authenticated web agent workflows and is developing a specialized web-native model. The company needed to manage GPU compute for post-training, evaluate full agent task success, serve models in production, and track operational data without maintaining separate systems.

## Approach / What changed

TinyFish consolidated its model development lifecycle on Databricks, using AI Runtime for managed GPU compute during post-training (supervised fine-tuning and DPO), MLflow for evaluation against full agent task success, and Databricks Model Serving for production deployment. Unity Gateway provides centralized cost tracking and governance for external foundation models.

## Takeaways

- TinyFish evaluates its web-native model by measuring full agent task completion rather than individual model responses, using a go/no-go benchmark that achieved a 77.6% success rate (1,691 of 2,178 tasks).
- Over a 15-day period, TinyFish’s web-native model served 294,781 agent runs across 6.4 million tool steps at an estimated cost of 0.088 per run ( 0.0040 per tool step).
- TinyFish uses Databricks AI Runtime for managed GPU infrastructure, MLflow for evaluation, and Model Serving for production, keeping training, evaluation, serving, and operational data in one environment to reduce infrastructure overhead.

## FAQ

### How does TinyFish evaluate its web-native model?

TinyFish evaluates the full agent's task completion rather than individual model responses. It uses an internal go/no-go benchmark where a separate LLM judge reviews the original request, agent action history, and final result. A task passes only when the requested action is completed or the correct answer is returned with all requirements satisfied.

### What is the serving cost of TinyFish's web-native model?

Over a 15-day period, the estimated model-serving cost was $0.088 per run or $0.0040 per tool step. These figures reflect serving costs for the web-native model only, not the full agent stack.

### What post-training techniques does TinyFish use?

TinyFish started with supervised fine-tuning and is moving toward Direct Preference Optimization (DPO). These workloads run on Databricks-managed GPUs through AI Runtime.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [Databricks](https://yomu.fyi/topic/databricks), [GPU](https://yomu.fyi/topic/gpu), [LLMs](https://yomu.fyi/topic/llm), [MLflow](https://yomu.fyi/topic/mlflow)

- Source: [Databricks](https://www.databricks.com/blog/post-training-production-how-tinyfish-runs-its-web-native-model-databricks-ai-runtime)
- Source URL: https://www.databricks.com/blog/post-training-production-how-tinyfish-runs-its-web-native-model-databricks-ai-runtime
- Ingested by Yomu: 2026-10-12T15:00:13.304Z

[Read original post](https://www.databricks.com/blog/post-training-production-how-tinyfish-runs-its-web-native-model-databricks-ai-runtime)