Rust AI in Practice: Building LLM Applications With Rig

Summary

Rig is an open-source Rust library that provides a common interface for LLM providers including OpenAI, Anthropic, and Gemini, avoiding application rewrites when switching services. It structures applications around a provider client, completion model, agent, preamble, and registered tools, with async APIs handling provider network requests. In the Rat Code demo, built with Ratatui, separate tools read and write files and run shell commands, while Rig’s streaming API updates the terminal as responses arrive. The library also supports RAG through vector store abstractions, local inference via Ollama, llama.cpp, or Candle, and cassette-based integration tests that replay recorded HTTP interactions in CI; live model quality requires separate scheduled testing.

Takeaways

1. 1

   Rig’s Tool trait defines a tool’s name, input, output, error types, JSON Schema arguments, and execution function; registered tools can let agents call Rust functionality such as file operations, database queries, or shell commands.

2. 2

   The rig-candle integration runs supported models directly in Rust, can embed model weights into an application, and supports inference through WebAssembly without a hosted API or separate local inference server.

3. 3

   Rig’s cassette tests record live provider HTTP traffic as YAML and replay it through a mock server in CI; these tests verify integration behavior, not whether changing live models still produce acceptable output.