Intercom vs Zendesk: Two AI agents put to the test

Summary

The post compares Intercom’s Fin AI Agent with Zendesk’s AI agent, focusing on answer quality, complexity, and resolution performance. In a test using 48 fictional bed-and-breakfast help articles and 200 questions, both systems answered the same dataset, while four AI models judged responses and an Elo rating system selected winners. It reports that Fin produced better answers in 80% of cases, answered 96% of multi-source questions versus Zendesk’s 78%, and delivered responses averaging 120 words versus 50. The analysis also found no statistical difference in hallucination levels and estimated that Fin was 66% more likely to provide a complete resolution when both agents answered. The post notes limitations including simulated data, AI judging, and differing product features, then describes Fin’s integrations and 99¢-per-resolution pricing.

Takeaways

1. 1

   Fin reportedly gave better answers in 80% of tested cases, with no statistical difference in hallucination levels between the two agents.

2. 2

   For questions requiring information from multiple sources, Fin answered 96% compared with Zendesk’s 78%, and it answered more than twice as many hard questions.

3. 3

   Fin’s average response was 120 words versus Zendesk’s 50, with more scannable formatting; the post estimated a 66% higher likelihood of complete resolution when both agents answered.