---
title: "The Open Source AI Stack"
description: "The post presents an open model AI stack for developers moving from closed-source models, focusing on ownership, control, and economics. It organizes the stack into independent layers: model, inference, gateways and routers, harness, and tools such as Skills and MCP. Models generate responses from prompts; large models generally handle ambiguity, multi-step reasoning, and long context better, while small models can be faster and cheaper for tightly specified work. The separation lets developers choose providers and models independently and experiment with newly released models without rebuilding their workflow, while the post uses Kimi K3 and GLM 5.3 Flash to illustrate differences in active parameters, capability, speed, and cost."
---

# The Open Source AI Stack

[Together AI](https://yomu.fyi/company/together-ai) · Sep 9, 2026

**Type:** Explainer

## Summary

The post presents an open model AI stack for developers moving from closed-source models, focusing on ownership, control, and economics. It organizes the stack into independent layers: model, inference, gateways and routers, harness, and tools such as Skills and MCP. Models generate responses from prompts; large models generally handle ambiguity, multi-step reasoning, and long context better, while small models can be faster and cheaper for tightly specified work. The separation lets developers choose providers and models independently and experiment with newly released models without rebuilding their workflow, while the post uses Kimi K3 and GLM 5.3 Flash to illustrate differences in active parameters, capability, speed, and cost.

## Context

The post addresses developers and organizations considering open source models for agentic software development. It argues that using open models does not require training models, purchasing large GPU infrastructure, or becoming an expert in machine learning.

## Approach / What changed

The explicit approach is to separate the stack into five independent layers: models, inference infrastructure and providers, gateways and routers, the harness that manages conversations and tools, and tools such as Skills and MCP. This separation supports independent technical choices and lets developers switch models without rebuilding their workflow.

## Takeaways

- Large models are suited to ambiguous, complex work such as authentication refactoring, framework upgrades, pull-request review, and diagnosing slow SQL databases.
- Small models can perform well on narrow, explicit tasks, while offering lower latency and cost; GLM 5.3 Flash is described as having 320B total parameters and 18B active parameters.
- Independent stack layers let developers change models or providers as new models appear, without replacing the surrounding harness and workflow.

## FAQ

### When should a developer choose a small model instead of a large one?

Small models fit clearly defined tasks that do not require architectural inference, extensive codebase understanding, or multiple interpretations. They are faster and cheaper, making repeated day-to-day coding work more practical.

### What are the layers of the open model AI stack?

The layers are model, inference, gateways and routers, harness, and tools. The harness manages the conversation and codebase access, while Skills and MCP provide task-specific knowledge or context.

### Why keep the stack layers independent?

Independence allows technical choices at each layer to match a development workflow. It also makes experimenting with newly released models possible without rebuilding the workflow.

**Tags:** [Inference](https://yomu.fyi/topic/inference), [LLMs](https://yomu.fyi/topic/llm), [Model Routing](https://yomu.fyi/topic/model-routing), [Open Source](https://yomu.fyi/topic/open-source)

- Source: [Together AI](https://www.together.ai/blog/the-open-source-ai-stack)
- Source URL: https://www.together.ai/blog/the-open-source-ai-stack
- Ingested by Yomu: 2026-09-19T10:13:47.383Z

[Read original post](https://www.together.ai/blog/the-open-source-ai-stack)