Loading…
Topic
Document AI
10 posts about Document AI. Every summary links to the original.
- 10 posts
- 3 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Power real-time applications and agents with Document Intelligence
Databricks announces General Availability of Document Intelligence REST APIs for real-time document parsing, extraction, and classification, with the ai_decide API for routing in beta. These synchronous APIs eliminate the manual bottleneck of reading documents by transforming unstructured contracts, invoices, and insurance claims into structured insights in real time. The ai_parse_document API converts tables and figures into structured text with citation bounding boxes, while ai_extract pulls specified fields with optional citations and confidence scores for verification. The ai_classify API assigns labels using a business taxonomy with optional rationales, and ai_decide uses content to route or flag documents for review. Developers can integrate these APIs into Databricks Apps for human-in-the-loop workflows, such as a contract intake flow where a lawyer uploads an agreement and sees key terms prefilled, or into custom agents on Agent Bricks.
What is Intelligent Document Processing?
Intelligent document processing (IDP) uses AI to extract, classify, validate, and structure information from PDFs, images, emails, forms, and other structured, semi-structured, or unstructured documents. Its pipeline combines document ingestion and preprocessing with OCR, document classification, NLP, layout analysis, data validation, human review for uncertain results, and delivery into downstream systems such as ERP, CRM, or accounting software. The post explains how embeddings support classification, how continuous learning adapts models to new layouts and corrections, and how metrics such as accuracy, throughput, confidence scores, and model drift support optimization. IDP can reduce manual effort, improve accuracy, lower costs, scale document-heavy workflows, and supply structured data for analytics, automation, AI agents, and retrieval-augmented generation, while high document variability and edge cases still make human-in-the-loop review relevant.
What is Document AI?
Document AI uses machine learning, natural language processing (NLP) and optical character recognition (OCR) to extract, classify and understand information from structured, semi-structured and unstructured documents. Unlike OCR alone, it interprets layout and context, turning files such as invoices, contracts and emails into structured, actionable data through ingestion, OCR, layout parsing, entity extraction, classification, validation and, when needed, human review. Modern systems add large language models for summarization, document Q&A and zero-shot extraction, but hallucination risk makes validation and human oversight essential, particularly in regulated settings. The guide also describes Databricks Document Intelligence, which processes and stores documents alongside organizational data under Unity Catalog, using AI Functions, Variant and Lakeflow Jobs to create governed, queryable workflows without moving data between systems.
Transforming solar and wind maintenance reports with Genie and AI agents
Plenitude and Databricks built an agent-based system that turns solar and wind plant maintenance PDFs into structured data for cross-plant analysis. Event-driven ingestion uses Databricks Jobs and the ai_parse_document AI Function to extract text, tables, figures, and metadata, then stores page- and object-level JSON records in Delta Lake with coordinates, version history, and links to source reports. A Genie space uses Unity Catalog metadata, knowledge-store instructions, and SQL generation to answer natural-language questions, produce visualizations, and export results, while Agent Bricks can orchestrate multi-step workflows and downstream actions. The design also applies automatic liquid clustering to dynamic queries and row-level security to restrict results by country. The resulting data layer supports historical trends, plant comparisons, recurring-fault analysis, and a foundation for predictive maintenance, although the source frames predictive use as a future improvement.
Unlocking the Archives: Turning Unstructured Documents into a Searchable Database for Groundwater Discovery
MapAid and the Sudan Association for Archiving Knowledge (SUDAAK) needed to make nearly 700 scanned geological reports searchable and recover well data for MapAid’s WellMapr groundwater models. Databricks built a Databricks-only pipeline that renders pages as images, uses multimodal AI Functions for page classification and schema-constrained JSON, and applies targeted sampling to cut classification processing by more than 70%. Water-relevant documents receive page-by-page OCR through the Foundation Model API, with well and borehole identifiers linking information across pages before structured records capture locations, depths, static water levels, and yields. In its first full run, 654 documents and 5,570 pages were classified in under three hours; 95% received excellent or good evaluations, about half contained water data, and 299 well or borehole records were extracted.
How to transform document activation workflows with Genie and Agent Bricks
Enterprise document workflows often require people to read PDFs, copy fields into spreadsheets, and re-enter data into business systems, creating delays, error risk, and weak auditability. The proposed alternative combines AI/BI Genie for natural-language queries over Delta tables, Agent Bricks for production agents, and Unity Catalog for permissions, lineage, and traceability. A five-phase workflow extracts fields into Lakeflow Bronze, Silver, and Gold layers, then supports structured queries, clause-level RAG answers, multi-agent routing, and actions through MCP servers. Examples include syncing validated rights data to ERP or CRM systems, updating billing entitlements, and triggering workflows from detected obligations. The approach targets media, advertising, ad tech, and telecommunications, where customers see improvements including faster month-end close, recovered revenue, reduced leakage, and lower operational risk.
Why your agents can't read enterprise documents — and how to fix it
Enterprise agents often reason well over clean text but struggle with scanned PDFs, inconsistent layouts, nested tables, images, and handwriting, making document processing an accuracy ceiling for business workflows. The post announces Document Intelligence, a pipeline of chainable AI Functions: ai_parse_document converts scans into layout-enriched structured text, while ai_classify and ai_extract support routing and entity extraction. On OfficeQA treasury-bond documents, preprocessing with ai_parse_document produced a 16% average performance gain across tested agent frameworks without changing their reasoning harness. Across invoice, contract, medical-note, and financial-filing extraction tasks, it reportedly delivered the highest accuracy at 5–7x lower cost than comparable pipelines, with serverless batch infrastructure scaling the same SQL call from 100 to 100,000. Databricks integrates ingestion, orchestration, governance, and agent building through Lakeflow Connect, Lakeflow Jobs or Spark Declarative Pipelines, Unity Catalog, and Agent Bricks, aiming to replace fragmented services with reusable workflows.
Building with Databricks Document Intelligence and Lakeflow
The post presents a production-oriented approach to Intelligent Document Processing (IDP) that brings PDFs, images, and office documents into Databricks’ governed data lifecycle. Lakeflow Connect ingests files from sources such as SharePoint and Google Drive into Unity Catalog Volumes and tables, applying access control, lineage, and auditing while supporting incremental reads and writes. Databricks’ Document Intelligence then uses ai_parse_document to preserve structure in VARIANT output, with ai_extract, ai_classify, and ai_prep_search available for enrichment, routing, and retrieval preparation. Lakeflow Jobs productionizes the workflow with orchestration, control flow, retries, triggers, serverless scaling, and observability across ingestion, parsing, and serving tasks. The post also recommends joining extracted data to master data, using service principals, and monitoring drift and extraction accuracy, presenting the combined platform as a way to create trusted, queryable datasets from unstructured content.
Financial Benchmarks
Ramp describes how it benchmarks large language models (LLMs) used in financial products against day-to-day tasks such as invoice extraction, financial-statement OCR, policy review, accounting autocoding, compliance detection, and fund routing. These benchmarks combine task-specific metrics with cost, latency, reasoning effort, human decisions, historical context, and ground-truth datasets where available. For contextual invoice OCR, perfect extraction requires every field to match the user's final bill, while financial-statement OCR uses a 1% relative-error threshold against transcriptions from over 500 real P&L documents. Results show application-specific trade-offs: Gemini 3 Flash is described as a cost-efficient leader for several visual and financial tasks, while Claude models lead some high-accuracy or low-miss-rate settings, and behavior can differ within one provider. The framework emphasizes Pareto trade-offs and continuous testing, with plans to expand from benchmarking current performance to hill climbing on capabilities.
How we built a custom vision LLM to improve document processing at Grab
Grab developed a specialized Vision LLM for extracting text and key information from Southeast Asian identity cards, driver’s licenses, and registration certificates, where varied languages and layouts challenged traditional OCR and existing models. It selected Qwen2-VL 2B for its efficient size, SEA language tokenizer coverage, and dynamic native-resolution image handling, while synthetic Common Crawl text images and Documint supplied reviewed training labels for detection, orientation, OCR, and KIE. LoRA fine-tuning improved Latin-script documents but struggled with Thai, Vietnamese, and dense unstructured layouts, leading to two-stage full-parameter training with synthetic SEA text images followed by task-specific document fine-tuning. The fully fine-tuned 2B model raised Thai accuracy by 70 percentage points and Vietnamese accuracy by 40 points over baseline, while a custom roughly 1B-parameter model combining Qwen2-VL 2B’s vision encoder with Qwen2.5 0.5B’s decoder stayed within a 3-point accuracy gap across most document types and achieved lower latency.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.