Data Mesh at Grab (Part III): Operationalizing data reliability with automated DPIs

Summary

Grab’s data mesh uses the Signals Marketplace to publish, discover, and reuse trusted data products, while certification establishes ownership, documentation, lineage, and quality controls but does not prevent runtime failures. Data Production Issues (DPIs) turn data-quality signals into an operational workflow that begins when Kinabalu detects unhealthy contract tests and closes when the underlying condition recovers. Triage promotes meaningful contract breaches, deduplicates repeated signals, groups related issues, and records affected assets, intervals, contracts, tests, and dependencies. For diagnosis, the Data Health API exposes a portable taxonomy of UPSTREAM_ERROR, PLATFORM_ERROR, JOB_ERROR, and DATA_ERROR, allowing ownership to follow the likely root cause while platforms retain detailed internal diagnostics. Combined with Hugo’s diagnosis and remediation patterns, the approach is presented as reducing resolution time and feeding operational reliability back into certification.

Takeaways

1. 1

   A contract test’s overall health can remain unhealthy after individual runs pass; for example, a failed historical partition may need to be backfilled and verified before recovery is complete.

2. 2

   Kinabalu can link a downstream DPI to an active upstream DPI and inherit its owner, keeping related failures with the team best positioned to resolve the root cause.

3. 3

   The Data Health API separates portable orchestration needs from platform-specific diagnostics, allowing heterogeneous platforms to evolve their internal error catalogs and tools independently.