# Metasense V2: Enhancing, improving and productionisation of LLM powered data governance

[Grab](https://yomu.fyi/company/grab) · Nick Buhrer · Nov 14, 2024

## Summary

Grab scaled Metasense to automate metadata generation, column-level classification, and sensitivity tiering across its entire data lake. Post-rollout analysis revealed that high tag volumes, wide tables, and mixed content—such as nested JSON and customer communications—strained model capacity and led to missed Personally Identifiable Information (PII). To overcome these limitations, the team split the classification process into distinct PII and non-PII tasks, reduced prompt word counts, and partitioned tables with over 150 columns into smaller units. Integrating LangChain and LangSmith modernized the architecture by enabling direct prompt experimentation, custom metric tracking, and version-controlled deployments. The updated pipeline achieved low misclassification rates alongside automated alert thresholds to trigger model improvement protocols if errors increase.

## Takeaways

- To resolve capacity constraints caused by having 13 non-PII tags among 21 total tags, the model was split into two separate parts: one dedicating 8 tags strictly to PII and another handling non-PII.
- Prompt engineering optimizations reduced prompt length from 1,254 words to 737 words, while wide tables exceeding 150 columns were divided into smaller tables to maintain LLM focus per column.
- The backend integrated LangChain for component chaining and LangSmith to manage evaluation datasets, track latency and error rates, and deploy prompt template updates via configuration identifiers.

**Tags:** [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Privacy](https://yomu.fyi/topic/privacy)

[Read original post](https://engineering.grab.com/metasense-v2)