Bringing custom knowledge to agents with AutoRAG

Summary

Enterprise LLMs can hallucinate when they lack company-specific vocabulary and documentation, so the guide builds a bank agent grounded in internal data. It configures Red Hat OpenShift AI 3.5 Technology Preview features, enables OGX, AutoRAG, and the MCP catalog, and deploys PostgreSQL, Milvus, and MinIO for metadata, vector embeddings, and artifacts. AutoRAG processes documents, evaluates pipeline configurations against a benchmark dataset, and identifies a preferred setup; the example uses Qwen3-8B and embeddinggemma-300m models. A MariaDB MCP server adds access to customer records, with a patch supplying writable logging storage and a warning that unrestricted ALLOWED_HOSTS is for demo/development only. The resulting Streamlit agent answers a customer-information query and a Gold-card-limit question using RAG context and database tools.

Takeaways

1. 1

   AutoRAG evaluates document processing and retrieval pipeline configurations against an evaluation dataset to identify the best setup for a business context.

2. 2

   The example stores OGX metadata in PostgreSQL, RAG embeddings in Milvus, and documents and artifacts in MinIO.

3. 3

   The final agent combines the selected RAG configuration with a MariaDB MCP server to answer customer and Gold-card-limit questions using database tools.