Agent platform (Part 1): How we help Grab build and run AI agents at scale

Summary

Grab scaled its internal AI tooling by transforming the infrastructure behind a single technical support bot into a standardized agent framework named LLM-Kit. The original Go-based bot revealed critical bottlenecks, including the lack of automated evaluation, painful provider migrations, fragmented observability, and excessive production scaffolding overhead. To address these frictions, engineers developed a FastAPI template pre-wired with LangGraph ReAct agents, remote Model Context Protocol servers, and built-in ROUGE, BLEU, and LLM-as-judge evaluation suites. The framework centralizes model access through an OpenAI-compatible GrabGPT Gateway and unifies observability across services, outbound calls, and agent steps using OpenTelemetry. This extracted scaffolding reduced day-one production setup time from two weeks to roughly an hour while supporting hundreds of production agents.

Takeaways

• Early agent deployments stalled because production overhead—such as OpenID Connect authentication, Vault secret management, and vector database integration—took weeks, compared to an afternoon for reasoning loop logic.
• LLM-Kit scaffolds services with pre-configured LangGraph StateGraph agents featuring a 30-second per-step timeout and day-one evaluation endpoints supporting ROUGE, BLEU, and LLM-as-judge metrics.
• Routing model requests through the OpenAI-compatible GrabGPT Gateway enables centralized provider switching, fallback routing, and cost attribution without requiring application code changes.

Related reading

• Grab · May 7, 2026

  Enhancing Flink deployment with shadow testing

  Production deployments of Apache Flink applications at Grab previously risked up to ten minutes of downtime when new versions failed under live traffic or could not resume from existing checkpoints. To eliminate these deployment failures, Grab's Coban data streaming team introduced an optional shadow testing stage directly into their production Kubernetes deployment pipeline. The shadow pipeline runs new application releases in parallel with the active application for a default observation period of one hour before prompting promotion to main. Dynamic runtime configuration based on an injected environment variable redirects source consumer groups and isolates output to dedicated shadow sinks, metrics, and logs.

  Tee Long Lang

• Grab · Jul 10, 2026

  Scaling Grab's Data Lake: Our journey to Apache Iceberg adoption

  Grab transitioned its petabyte-scale data lake from Hive Parquet to Apache Iceberg to resolve metadata bottlenecks, small file fragmentation, and lack of native ACID support. Implementing Iceberg with Z-ordering reduced navigation query runtimes from 70 seconds to 6 seconds and cut daily S3 API costs by up to 95% on key operational datasets. To seamlessly manage concurrent table formats without forcing engineers to specify format prefixes, Grab developed and open-sourced UnifiedSparkCatalog. This custom catalog transparently detects table types via the Hive Metastore, routes queries to format-specific handlers, and retains backwards compatibility for legacy Hive operations.

  Rahul Penti

• Grab · Jun 19, 2026

  Palana (Part 1): Why Grab built a secure platform for autonomous AI Agents

  Autonomous AI agents introduce significant operational and security risks when granted network access, persistent state, and credentials. To address these concerns without impeding developer productivity, Grab created Palana, an in-house Kubernetes-native execution substrate. The platform isolates each agent workload within its own namespace, pairing it with dedicated storage, network policies, and role-based access control. Network egress is funneled through an Envoy and Open Policy Agent proxy layer that audits requests and injects credentials from HashiCorp Vault using placeholder tokens, keeping raw secrets outside the agent runtime. This design allows Grab to securely host hundreds of long-running workflows, remote coding environments, and automation bots.

  Kevin Littlejohn

• Grab · Aug 12, 2026

  Grab Bench: Evaluating AI on Grab-shaped production work

  AI models evaluated on internal tasks often produce subtly plausible errors, such as altering metric definitions in SQL queries or missing hidden stateful invariants in code patches. To address this, Grab built Grab Bench, a configurable evaluation harness that executes model providers through specialized task plugins. The platform evaluates synthetic or redacted datasets that preserve production constraints across domains like query generation, tool use, passenger-profile reasoning, and agentic coding. By recording granular row-level outputs with deterministic scorers and LLM judges, the harness exposes failure modes rather than relying solely on aggregate leaderboards.

  Christian Coffrant