LLM-powered data classification for data entities at scale

Summary

Grab needed to classify sensitive data at the table and column level across petabytes of database tables and streaming schemas. Manual schema-tiering campaigns had resulted in half of all schemas receiving overly strict Tier 1 access controls, while an initial automated service using regex patterns and third-party machine learning produced high false-positive rates and lacked customizability. To address this, the Caspian data engineering and governance teams enhanced their internal orchestration service, Gemini, by integrating GPT-3.5 via Azure OpenAI. Gemini aggregates classification requests into mini-batches, handles API rate limits, and uses prompt engineering—including few-shot examples, curated tag libraries, and explicit JSON DTO schemas—to reliably tag columns for data owner verification.

Context

Manual sensitivity tagging caused half of all schemas to be locked down under strict Tier 1 access controls due to a few sensitive tables, while regex and third-party ML classifiers suffered from high false-positive rates and limited customizability.

Approach / What changed

Grab enhanced its internal orchestration service, Gemini, to batch scan requests, enforce API rate limits, and send table and column metadata to GPT-3.5 using structured prompts, curated tag definitions, and few-shot JSON formatting.

Takeaways

• Schema-level sensitivity tagging overly restricted access because a single Tier 1 table caused an entire schema of hundreds of tables to inherit the strictest access controls.
• Gemini relies on message queues to aggregate scanning requests into mini-batches and applies workflow-level rate limiting to adhere to Azure OpenAI limits of 240K tokens per minute.
• Prompt engineering techniques such as providing JSON DTO schemas, few-shot examples, and explicit default fallback tags (<None>) ensure the LLM returns structured, machine-parseable classifications.

Related reading

• Grab · Aug 15, 2022

  How we store and process millions of orders daily

  The Grab Order Platform processes millions of food and mart transactions daily, requiring high throughput, fault tolerance, and reduced cloud costs across transactional and analytical workloads. To meet these demands, the engineering team decoupled their database architecture by using Amazon DynamoDB for critical OLTP queries and MySQL RDS for historical OLAP queries. DynamoDB handles online order lifecycles with strong consistency, utilizing sparse Global Secondary Indexes for ongoing orders and TTL configurations to limit storage growth. Updates propagate asynchronously to MySQL RDS through a Kafka ingestion pipeline backed by Amazon SQS retries and timestamp-based version checks. This dual-database approach isolated core transaction availability from analytical queries and delivered significant cloud cost savings.

  Xi Chen

• Grab · Aug 5, 2024

  How we improved translation experience with cost efficiency

  Grab faced translation quality and cost challenges in its booking chat system when tourism resumed after COVID restrictions eased. Device language settings proved unreliable for determining message language, prompting the team to use Lingua alongside booking context heuristics and third-party fallbacks for real-time detection. To replace expensive and inaccurate general translation APIs, Grab distilled a large open-source Hugging Face model into lightweight, language-specific models trained on ten million synthetic chat examples. A post-translation validation step verifies non-translatable entities such as numbers and emojis before falling back to external services if mutations occur. Caching layers were also introduced across translation paths to curb redundant on-the-fly execution.

  Jie Zhang

• Grab · Dec 26, 2023

  Kafka on Kubernetes: Reloaded for fault tolerance

  Grab's real-time data streaming platform, Coban, operates Kafka on AWS Elastic Kubernetes Service using Strimzi, allocating an entire EC2 worker node with NVMe instance store volumes to each broker. An initial architecture suffered from client connection errors, broken Network Load Balancer target groups, and zombie Persistent Volume Claims when worker nodes terminated. To achieve automated fault tolerance, the team integrated the AWS Node Termination Handler in Queue Processor mode with Auto Scaling lifecycle hooks, ensuring Kafka receives a SIGTERM to migrate partition leadership gracefully before shutdown. They also introduced the Kubernetes Cluster Autoscaler to dynamically provision replacement nodes during maintenance events and used the AWS Load Balancer Controller with TargetGroupBinding custom resources to dynamically update load balancer targets using IP mode.

  Fabrice Harbulot

• Grab · Aug 15, 2023

  Streamlining Grab's Segmentation Platform with faster creation and lower latency

  Grab's Segmentation Platform previously stored user-to-segment mappings across individual rows in ScyllaDB, causing write bottlenecks during segment creation and read latencies too high for downstream consumers. To resolve these performance limitations, the team transitioned to storing segments as Roaring Bitmaps saved as single blobs in object storage. This compression strategy splits 32-bit integer user IDs into chunks across array, bitmap, and run containers based on data density, reducing a one-million-member segment to under one megabyte. An accompanying client SDK manages segment retrieval, decoding, update notifications, and least-recently-used in-memory caching. Consequently, consumers such as Grab's communications platform achieved peak throughput of 15,000 queries per second with sub-millisecond p99 read latencies.

  Jake Ng