---
title: "Building AI with MongoDB and Patronus AI"
description: "Large language models can be unreliable, with serious consequences in regulated industries. Patronus AI, an industry-first automated evaluation platform, enables engineers to score and benchmark LLM performance on real-world scenarios, generate adversarial test cases, and monitor hallucinations and PII. Research using FinanceBench found that state-of-the-art LLMs hallucinated or incorrectly answered up to 81% of financial analysts’ questions, even with RAG and external context. Patronus partnered with MongoDB Atlas to offer a RAG evaluation API and published a 10-minute guide demonstrating evaluation of a RAG system built with LlamaIndex and Atlas Vector Search on an SEC 10-K filing. Developers can improve RAG performance by adjusting indexes, chunking sizes, prompts, or fine-tuning embedding models, and should continuously test to maintain improvements."
---

# Building AI with MongoDB and Patronus AI

[MongoDB](https://yomu.fyi/company/mongodb) · Oct 15, 2026

**Type:** Explainer

## Summary

Large language models can be unreliable, with serious consequences in regulated industries. Patronus AI, an industry-first automated evaluation platform, enables engineers to score and benchmark LLM performance on real-world scenarios, generate adversarial test cases, and monitor hallucinations and PII. Research using FinanceBench found that state-of-the-art LLMs hallucinated or incorrectly answered up to 81% of financial analysts’ questions, even with RAG and external context. Patronus partnered with MongoDB Atlas to offer a RAG evaluation API and published a 10-minute guide demonstrating evaluation of a RAG system built with LlamaIndex and Atlas Vector Search on an SEC 10-K filing. Developers can improve RAG performance by adjusting indexes, chunking sizes, prompts, or fine-tuning embedding models, and should continuously test to maintain improvements.

## Context

The post discusses the unreliability of large language models, particularly in regulated industries, and introduces Patronus AI’s automated evaluation platform. It highlights research showing high hallucination rates in financial QA even with RAG, and describes a partnership with MongoDB Atlas to provide RAG evaluation services. A 10-minute guide demonstrates evaluating a RAG system using LlamaIndex, Atlas Vector Search, and Patronus on an SEC 10-K filing.

## Approach / What changed

Patronus AI provides an automated evaluation platform that scores LLM performance, generates adversarial test cases, and monitors hallucinations and PII. The company partnered with MongoDB Atlas to offer a RAG evaluation API. A 10-minute guide demonstrates a workflow using LlamaIndex to ingest and chunk a PDF, Atlas Vector Search to store and query embeddings, and Patronus to score responses, focusing on hallucination and answer relevance for an SEC 10-K filing.

## Takeaways

- Patronus AI’s research using FinanceBench found that state-of-the-art LLMs hallucinated or incorrectly answered up to 81% of financial analysts’ questions, even with RAG and external context.
- Patronus AI offers an automated evaluation platform that scores LLM performance, generates adversarial test cases, and detects hallucinations and PII.
- Developers can improve RAG system performance by adjusting indexes, modifying document chunking sizes, re-engineering prompts, or fine-tuning the embedding model, and should continuously test to maintain improvements.

## FAQ

### What error rate did Patronus AI find in financial QA?

Research using FinanceBench found that state-of-the-art LLMs hallucinated or incorrectly answered up to 81% of financial analysts' questions, even with retrieval augmented generation and context from an external vector store.

### How does Patronus AI evaluate RAG systems built on MongoDB Atlas?

Patronus AI offers a RAG evaluation API that scores model responses for hallucination and answer relevance. A 10-minute guide demonstrates a workflow using LlamaIndex for data ingestion, Atlas Vector Search for storage and querying, and Patronus for scoring.

### What steps can developers take to improve RAG system performance?

Developers can explore different indexes, modify document chunking sizes, re-engineer prompts, or fine-tune the embedding model. Continuous testing with the Patronus API is recommended to maintain improvements.

**Tags:** [AI](https://yomu.fyi/topic/ai), [LLMs](https://yomu.fyi/topic/llm), [MongoDB Atlas](https://yomu.fyi/topic/mongodb-atlas), [Patronus AI](https://yomu.fyi/topic/patronus-ai), [RAG](https://yomu.fyi/topic/rag)

- Source: [MongoDB](https://www.mongodb.com/company/blog/how-patronus-automates-llm-evaluation-boost-confidence-genai)
- Source URL: https://www.mongodb.com/company/blog/how-patronus-automates-llm-evaluation-boost-confidence-genai
- Ingested by Yomu: 2026-10-16T00:00:21.124Z

[Read original post](https://www.mongodb.com/company/blog/how-patronus-automates-llm-evaluation-boost-confidence-genai)