Building Jarvis Pro: Route first, answer later

Summary

Grab designed Jarvis Pro as an AI assistant for account managers turning complex account data into actionable merchant conversations. Early prototypes generated polished but operationally unsound advice, such as pushing promotions when order drops were caused by outlet pauses and fulfillment issues. To address this, the team implemented a "route first, answer later" design that uses a prompted LLM classification step constrained by a routing guide to dictate context, tool paths, answer structure, and guardrails before generation. The pipeline also isolates narrow memory decisions, reconciles metric freshness across data sources, and caps tool calls to maintain usability. In offline evaluations across 351 realistic prompts, routing matched safe routes 99.4% of the time while answer quality scores in a portfolio subset improved from 78.5 to 91.0.

Context

Early prototypes of Jarvis Pro produced fluent, plausible answers that were operationally incorrect, such as recommending promotions when merchant order drops were actually driven by operational issues like paused outlets and slipping fulfillment.

Approach / What changed

The team adopted a 'route first, answer later' architecture where a prompted LLM classification step determines task types, allowed context, memory rules, guardrails, and tool paths before retrieval and answer generation, supported by narrow memory checks and source metric reconciliation.

Takeaways

• In offline launch-readiness evaluations on 351 realistic prompts, routing matched the expected safe route for 99.4% of cases.
• A 150-case portfolio and brand answer-quality evaluation subset improved its average rubric score from 78.5 to 91.0 after implementing layered routing and guardrails.
• Data discrepancies between dashboards and warehouse tables were addressed by building regression checks that normalized source values and compared daily rows across approved metric paths.

Related reading

• Grab · Jul 3, 2026

  Migrating Counter Service storage: Design choices and learnings

  Grab migrated its Counter Service storage backend from a legacy wide-column database to Aerospike to support real-time anti-fraud windowed aggregations across tens of thousands of queries per second. To decouple storage from the Rust reader service, engineers introduced a storage facade using enum dispatch, avoiding the per-query heap allocations of boxed trait objects. The reader used configuration-driven operating modes to support shadow reads and deterministic traffic splitting without requiring code updates. On the write path, the schema was redesigned to collapse multiple bucket records into a single sorted map per counter, using atomic server-side operations to increment counters and prune expired entries. This data model redesign reduced total record counts and in-memory primary index usage by over an order of magnitude while enabling a zero-downtime transition.

  Jia Long Loh

• Grab · May 22, 2026

  The Hugo evolution: Engineering Grab's unified, one-click data ingestion platform with Apache Flink

  Grab's self-service data platform, Hugo, faced significant onboarding friction as streaming pipelines expanded across fragmented systems like Kafka Connect, custom Go applications, and Spark. Engineering teams struggled with cross-platform configuration translations and brittle, manual schema mappings that stretched onboarding over several days. To resolve these bottlenecks, Grab modernized the ingestion architecture by introducing a centralized automation layer powered by Apache Flink and Flink CDC. The updated platform dynamically retrieves Protobuf schemas from Confluent Schema Registry and ingests MySQL binlogs directly into queryable Hive tables without intermediate Kafka hops. This shift dropped pipeline onboarding times to roughly six minutes for Kafka and three minutes for MySQL CDC, driving more pipeline adoptions in one year than in the previous five.

  Shuguang Xiang

• Grab · Jun 22, 2026

  Scaling out Distroless adoption With AI

  Grab is transitioning its microservices to Distroless base images to eliminate unnecessary binaries and reduce vulnerability risks, but the migration risks runtime failures from missing shared objects and system utilities. To safely validate container execution in continuous integration without staging dependencies, the team relied on medium tests that run containerized services alongside internal dependencies managed by Testcontainers. Because hundreds of services lacked this test harness, Grab implemented an agentic workflow using Claude Code and Model Context Protocol integrations to inspect repositories, generate test boilerplate, and resolve configuration errors. Once test baselines are established, an automated patch-test-compare pipeline updates Dockerfiles, constructs multi-stage builds for necessary dynamic libraries, and creates draft merge requests for human approval.

  Jia Yee Chong

• Grab · Aug 12, 2026

  Grab Bench: Evaluating AI on Grab-shaped production work

  AI models evaluated on internal tasks often produce subtly plausible errors, such as altering metric definitions in SQL queries or missing hidden stateful invariants in code patches. To address this, Grab built Grab Bench, a configurable evaluation harness that executes model providers through specialized task plugins. The platform evaluates synthetic or redacted datasets that preserve production constraints across domains like query generation, tool use, passenger-profile reasoning, and agentic coding. By recording granular row-level outputs with deterministic scorers and LLM judges, the harness exposes failure modes rather than relying solely on aggregate leaderboards.

  Christian Coffrant