---
title: "The production platform for open-weight AI inference"
description: "The update presents a production inference platform for open-weight, licensed closed-weight, and fine-tuned models, aimed at giving teams control over performance, cost, quality, and functionality without building an inference stack from scratch. It supports uploading weights or adapters from Hugging Face, S3, or local machines, uses tested deployment profiles for hardware, quantization, tensor parallelism, speculative decoding, and serving engine choices, and lets operators tune placement and autoscaling signals. Endpoints can keep a stable interface while multiple deployments receive traffic through canary, blue-green, rolling, A/B, or shadow testing, with threshold-based automatic rollback. The rebuilt caching and distribution layer proactively warms shared weights; internal testing reports roughly 4× faster warm starts across several frontier models, while a closed beta adds full-weight and LoRA reinforcement learning and supervised fine-tuning with deployable checkpoints."
---

# The production platform for open-weight AI inference

[Together AI](https://yomu.fyi/company/together-ai) · Jul 23, 2026

**Type:** Announcement

## Summary

The update presents a production inference platform for open-weight, licensed closed-weight, and fine-tuned models, aimed at giving teams control over performance, cost, quality, and functionality without building an inference stack from scratch. It supports uploading weights or adapters from Hugging Face, S3, or local machines, uses tested deployment profiles for hardware, quantization, tensor parallelism, speculative decoding, and serving engine choices, and lets operators tune placement and autoscaling signals. Endpoints can keep a stable interface while multiple deployments receive traffic through canary, blue-green, rolling, A/B, or shadow testing, with threshold-based automatic rollback. The rebuilt caching and distribution layer proactively warms shared weights; internal testing reports roughly 4× faster warm starts across several frontier models, while a closed beta adds full-weight and LoRA reinforcement learning and supervised fine-tuning with deployable checkpoints.

## Context

Open-weight models are presented as a way for teams to control performance, quality, functionality, and proprietary intellectual property. The platform is designed to carry models from experimentation into production without requiring a platform change or a rebuilt deployment.

## Approach / What changed

The platform combines Together deployment profiles with configurable hardware and optimization choices, model uploads from Hugging Face, S3, or local machines, shared and proactively warmed model caching, region controls, and autoscaling based on workload-specific signals. It keeps endpoints stable across multiple deployments and provides canary, blue-green, rolling, A/B, and shadow testing with threshold-based rollback. A closed custom-training beta offers full-weight and LoRA reinforcement learning and supervised fine-tuning, with checkpoints deployable to production.

## Takeaways

- Deployment profiles package tested choices for hardware, quantization, tensor parallelism, speculative decoding, and serving engines, while allowing more manual control when workloads require it.
- Autoscaling can use inflight requests, GPU utilization, time to first token, latency, decoding speed, or throughput instead of relying on one generic metric.
- Internal testing found roughly 4× faster warm starts across several frontier models after rebuilding the model caching and distribution layer.

## FAQ

### How can teams roll out a new model without changing the endpoint?

An endpoint can serve multiple deployments behind one stable interface. Traffic can be shifted using canary, blue-green, rolling, A/B, or shadow testing, with automatic rollback when configured thresholds are exceeded.

### Which signals can control autoscaling?

The platform supports scaling based on inflight requests, GPU utilization, time to first token, latency, decoding speed, or throughput. Teams can choose a signal that reflects the demand pattern of their workload.

### What reduces startup time for large model deployments?

The model caching and distribution layer shares weights across the fleet and proactively warms them before deployment needs them. Internal testing reports roughly 4× faster warm starts across several frontier models.

**Tags:** [Autoscaling](https://yomu.fyi/topic/autoscaling), [Deployment](https://yomu.fyi/topic/deployment), [Model Serving](https://yomu.fyi/topic/model-serving)

- Source: [Together AI](https://www.together.ai/blog/the-production-platform-for-open-weight-ai-inference)
- Source URL: https://www.together.ai/blog/the-production-platform-for-open-weight-ai-inference
- Ingested by Yomu: 2026-09-19T10:13:41.834Z

[Read original post](https://www.together.ai/blog/the-production-platform-for-open-weight-ai-inference)