PII masking for privacy-grade machine learning

Summary

Data engineers at Grab require real-world streaming data to tune machine learning models, but user personal data cannot be exposed in non-production environments. To solve this, the data streaming team implemented an automated masking pipeline between production and staging Kafka clusters. Developers explicitly tag Personally Identifiable Information (PII) types in Protocol Buffers schemas, which a CI script validates against a keyword list before packaging the schemas into Scala JARs. An in-house Apache Flink application running in production consumes the unmasked streams, applies dynamic and consistent transformations like keyed HMACs or seeded random values, and produces sanitised records to staging. Untagged new fields in schema updates are automatically dropped by the deserialiser until the masking application is redeployed, preventing accidental data leaks.

Context

Data engineers need access to realistic production streaming data to train and tune machine learning models, but privacy regulations and security policies forbid granting them direct access to unmasked user PII in production.

Approach / What changed

Developers tag PII fields within Protobuf schemas, which are verified via CI keyword-scanning scripts and compiled into Scala JARs for an Apache Flink application that consumes production Kafka streams, dynamically applies consistent HMAC or seeded masking, and outputs sanitised data to a staging Kafka cluster.

Takeaways

• CI pipelines validate Protobuf schema changes against an extensive PII keyword list, failing merge requests unless matching fields are tagged with specific PII types or added to a verified whitelist.
• Data masking maintains consistency for ML model training by using keyed HMACs for strings and unmasked values as seeds for numerical and date randomisation.
• If a schema introduces a new PII-tagged field before the Flink masking application is recompiled and redeployed, the deserialiser drops the field, preventing unmasked data from reaching staging.

Related reading

• Grab · Nov 17, 2023

  Road localisation in GrabMaps

  Grab needed to localise nearly 30 million road segments across more than 10,000 area boundaries in Southeast Asia to attach hyperlocal attributes like language, driving side, and vehicle access rules. Testing direct geometric inclusion between complex road polylines and boundary polygons proved computationally prohibitive for daily map generation pipelines. To optimize this process, Grab introduced geohashes as rectangular proxies, precomputing geohash coverage across borders and road segments before joining the datasets in parallel. To resolve misclassification errors near borders without sacrificing performance, the team evaluated geohash coverage percentages and refined boundary-touching geohashes with polygon clipping. A laptop benchmark on a subset of data reduced runtime from 38 minutes with geometric intersection to 78 seconds using the geohash approximation.

  Roxana Crisan

• Grab · Apr 28, 2022

  Graph Networks - Striking fraud syndicates in the dark

  Grab's Integrity team faced escalating threats from organized fraud syndicates operating device farms and GPS spoofing tools that mimic normal user behavior. To overcome the limitations of traditional rule-based systems, Grab built a Graph-based Prediction Platform that links entities across shared attributes to expose hidden network structures. The architecture includes a scalable graph database platform holding over one billion nodes, powering real-time graph visualization for analysts alongside a configurable network feature extraction system. In addition, the team deployed Graph Neural Networks to capture structural correlations across transaction networks. These systems collectively help detect coordinated promo abuse, payment fraud, and complex money laundering schemes.

  Muqi Li

• Grab · Apr 17, 2023

  Message Center - Redesigning the messaging experience on the Grab superapp

  Grab redesigned its messaging infrastructure from GrabChat to Message Center to overcome two-party chat limitations and support complex superapp requirements like group conversations and varied user roles. The architecture separates core processing logic from message delivery by splitting the system into a backend processor and an independently scalable postman service. Communication relies on an in-house TCP gateway named Hermes that proxies client payloads via gRPC, alongside Apache Kafka streams and Amazon SQS delay queues. Custom client-server acknowledgements and a DynamoDB event store ensure reliable message delivery even across dropped mobile TCP connections and offline reconnects.

  Jonathan Lee

• Grab · Nov 8, 2023

  Graph modelling guidelines

  Graph modelling uses graph theory to represent real-world entities, relationships, and properties through nodes and edges, unlocking insights across interconnected datasets. The process begins with defining domain concepts, mapping entities and relationships, and assigning relevant descriptive properties to both nodes and edges. Practitioners choose between graph structures such as property graphs or Resource Description Framework models before optionally developing a structural schema. Data is then imported or generated to populate storage systems such as Neo4j, Amazon Neptune, Azure Cosmos DB, or in-memory data structures. Finally, the graph is visualised and analysed using algorithmic methods like shortest path calculation, centrality measurement, and community detection.

  Wenxiang Lu