Transforming solar and wind maintenance reports with Genie and AI agents

Summary

Plenitude and Databricks built an agent-based system that turns solar and wind plant maintenance PDFs into structured data for cross-plant analysis. Event-driven ingestion uses Databricks Jobs and the ai_parse_document AI Function to extract text, tables, figures, and metadata, then stores page- and object-level JSON records in Delta Lake with coordinates, version history, and links to source reports. A Genie space uses Unity Catalog metadata, knowledge-store instructions, and SQL generation to answer natural-language questions, produce visualizations, and export results, while Agent Bricks can orchestrate multi-step workflows and downstream actions. The design also applies automatic liquid clustering to dynamic queries and row-level security to restrict results by country. The resulting data layer supports historical trends, plant comparisons, recurring-fault analysis, and a foundation for predictive maintenance, although the source frames predictive use as a future improvement.

Context

Maintenance and operations suppliers typically deliver plant reports as PDFs containing information across free text, tables, and images. Teams had to read reports individually, making fault analysis, trend detection, recurring-issue identification, cross-plant comparison, and structured output generation slow and inconsistent as the number of assets grew.

Approach / What changed

The system uses event-driven PDF ingestion, ai_parse_document extraction, JSON normalization, and Delta Lake storage with page-level metadata, coordinates, version history, and source links. A Genie space queries this curated layer using Unity Catalog metadata, knowledge-store instructions, and generated SQL. Agent Bricks can orchestrate multi-step workflows, while automatic liquid clustering and country-based row-level security address performance and access control.

Takeaways

• ai_parse_document extracts text, tables, figures, and metadata, while normalized Delta Lake records preserve page coordinates and direct links to the original reports.
• Genie instructions provide operational guardrails such as using month-level columns, excluding YTD fields, and validating that all required months are present before aggregation.
• Unity Catalog-backed row-level security filters Genie queries by each user's authorized countries, allowing the shared dataset to serve Genie, agents, and BI tools with user-specific visibility.

Related reading