---
title: "Rust AI in Practice: Building LLM Applications With Rig"
description: "Rig is an open-source Rust library that provides a common interface for LLM providers including OpenAI, Anthropic, and Gemini, avoiding application rewrites when switching services. It structures applications around a provider client, completion model, agent, preamble, and registered tools, with async APIs handling provider network requests. In the Rat Code demo, built with Ratatui, separate tools read and write files and run shell commands, while Rig’s streaming API updates the terminal as responses arrive. The library also supports RAG through vector store abstractions, local inference via Ollama, llama.cpp, or Candle, and cassette-based integration tests that replay recorded HTTP interactions in CI; live model quality requires separate scheduled testing."
---

# Rust AI in Practice: Building LLM Applications With Rig

[Jetbrains](https://yomu.fyi/company/jetbrains) · Irina Mihajlovic · Sep 9, 2026

**Type:** Explainer

## Summary

Rig is an open-source Rust library that provides a common interface for LLM providers including OpenAI, Anthropic, and Gemini, avoiding application rewrites when switching services. It structures applications around a provider client, completion model, agent, preamble, and registered tools, with async APIs handling provider network requests. In the Rat Code demo, built with Ratatui, separate tools read and write files and run shell commands, while Rig’s streaming API updates the terminal as responses arrive. The library also supports RAG through vector store abstractions, local inference via Ollama, llama.cpp, or Candle, and cassette-based integration tests that replay recorded HTTP interactions in CI; live model quality requires separate scheduled testing.

## Context

Different LLM providers expose separate APIs that can behave differently despite compatibility claims. LLM applications also need tools, external context, local inference, and testing approaches that account for nondeterministic model responses and changing provider behavior.

## Approach / What changed

Rig addresses provider differences with a shared Rust interface and organizes applications around clients, models, agents, preambles, and tools. The Rat Code example uses Rig and Ratatui for a terminal coding agent, while vector-store integrations, local runtimes, streaming, and recorded HTTP cassettes support broader application and testing needs.

## Takeaways

- Rig’s Tool trait defines a tool’s name, input, output, error types, JSON Schema arguments, and execution function; registered tools can let agents call Rust functionality such as file operations, database queries, or shell commands.
- The rig-candle integration runs supported models directly in Rust, can embed model weights into an application, and supports inference through WebAssembly without a hosted API or separate local inference server.
- Rig’s cassette tests record live provider HTTP traffic as YAML and replay it through a mock server in CI; these tests verify integration behavior, not whether changing live models still produce acceptable output.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [LLMs](https://yomu.fyi/topic/llm), [Retrieval-Augmented Generation](https://yomu.fyi/topic/retrieval-augmented-generation), [Rust](https://yomu.fyi/topic/rust), [Testing](https://yomu.fyi/topic/testing)

- Source: [Jetbrains](https://blog.jetbrains.com/rust/2026/09/09/rust-ai-in-practice)
- Source URL: https://blog.jetbrains.com/rust/2026/09/09/rust-ai-in-practice
- Ingested by Yomu: 2026-09-10T00:05:18.019Z

[Read original post](https://blog.jetbrains.com/rust/2026/09/09/rust-ai-in-practice)