Explaining Every Rupee: How We Built Reliable LLM Support Bots for Delivery Partners

Summary

Swiggy faced close to 9 lakh monthly payout- and incentive-related tickets, where delivery executives needed accurate explanations of earnings, deductions, missed incentives, and credited amounts. The system separates financial truth from language generation: deterministic preprocessing and internal APIs resolve incentive eligibility, payout breakdowns, deductions, and conditions before an LLM explains the result. Two multilingual bots—the Daily Incentive Bot and Payout Bot—use a LangGraph StateGraph with LLM, tool, and validator nodes, Redis checkpointing for multi-turn context, and post-processing translation into the executive’s preferred language. Azure Translator preserve-list controls help keep business-critical terminology consistent, while structured escalation sends out-of-scope cases and context to human agents. Moving retrieval from Databricks SQL queries to an aggregated internal API reduced latency from 8–10 seconds to milliseconds; the bots now resolve about 90% end to end, cut resolution from roughly 20 minutes to seconds, and improved CSAT by 40%.

Takeaways

1. 1

   The validator compares claimed incentive amounts in the generated response with raw tool output and sends mismatches back for regeneration.

2. 2

   Replacing Databricks SQL retrieval and heavy joins with an aggregated internal API reduced data-fetch latency from 8–10 seconds to milliseconds.

3. 3

   The bots resolve about 90% of incentive- and payout-related queries end to end, reduce resolution time from roughly 20 minutes to a few seconds, and improve CSAT by 40%.