Unlocking the Archives: Turning Unstructured Documents into a Searchable Database for Groundwater Discovery

Summary

MapAid and the Sudan Association for Archiving Knowledge (SUDAAK) needed to make nearly 700 scanned geological reports searchable and recover well data for MapAid’s WellMapr groundwater models. Databricks built a Databricks-only pipeline that renders pages as images, uses multimodal AI Functions for page classification and schema-constrained JSON, and applies targeted sampling to cut classification processing by more than 70%. Water-relevant documents receive page-by-page OCR through the Foundation Model API, with well and borehole identifiers linking information across pages before structured records capture locations, depths, static water levels, and yields. In its first full run, 654 documents and 5,570 pages were classified in under three hours; 95% received excellent or good evaluations, about half contained water data, and 299 well or borehole records were extracted.

Takeaways

1. 1

   Sampling shorter documents fully and longer documents from informative sections reduced AI processing volume by more than 70% while preserving classification quality.

2. 2

   Well and borehole identifiers serve as anchor entities during OCR, allowing locations, depths, water levels, and yields distributed across multiple pages to be linked into structured records.

3. 3

   An AI evaluator rates classifications for accuracy, completeness, and consistency; documents below a confidence threshold are routed for manual review, and 95% of first-run classifications were rated excellent or good.