Agent platform (Part 1): How we help Grab build and run AI agents at scale

Summary

Grab’s AI support bot grew from an internal GPT-powered Level-0 service into the foundation for LLM-Kit, an internal framework now used by more than 500 services, alongside over 50 registered remote MCP servers and a company-wide LLM gateway handling billions of tokens monthly. The bot used a single-agent reasoning loop and a tool plane spanning Glean, Kibana, GitLab, Slack, and HTTP plugins, but scaling exposed missing evaluations, fragmented observability, provider-switching friction, and costly production plumbing. LLM-Kit addresses those problems with generated FastAPI services, LangGraph agent templates, OpenTelemetry tracing, configuration and secret-management integrations, evaluation routes, remote MCP tool discovery, and gRPC contracts, clients, health checks, and channel selection. The post concludes that framework concerns became platform concerns at 500 agents, leading to a gateway, remote MCP framework, and evals platform described as the series continues.

Takeaways

1. 1

   The original bot combined a single-agent reasoning loop with tools for Glean retrieval, Kibana logs, GitLab files and runbooks, Slack searches, and HTTP plugins.

2. 2

   LLM-Kit favors a framework over a heavy centralized platform, standardizing production plumbing while preserving teams’ ability to iterate quickly.

3. 3

   At more than 500 agents, the remaining challenges shifted from building individual services to governing shared models, reusable tools, and evaluation results.