AppleExplainer
The P-Completeness of Inverted Index Traversal: On the Complexity of Evaluating Boolean Query DAGs
Summary
Modern artificial intelligence agents increasingly rely on search infrastructure to run complex, neuro-symbolic reasoning workflows that compile into deeply nested, non-monotonic Boolean queries. Standard inverted index evaluation methods struggle with these query structures, facing either exponential query complexity under Document-at-a-Time models or universe-scale space penalties under Term-at-a-Time models during logical negation. To address these theoretical limitations, the author formalizes a DAG-based retrieval language named L_R and proves its query evaluation problem is strictly P-Complete. The paper introduces ComputePN, a deterministic and sparsity-aware algorithm that employs a Positive-Negative dual representation alongside native DAG memoization. This mechanism bounds evaluation time to O(|Q| · |U_active|), effectively evaluating P-Complete queries natively over the index while bypassing combinatorial tree expansion and universal scan overhead.
Context
Modern AI workflows often generate complex, deeply nested Boolean queries over text fields. Evaluating these queries natively over inverted indices causes Document-at-a-Time iterators to suffer an exponential worst-case blowup of O(2^|Q|) when unrolling re-convergent logic, while Term-at-a-Time models face an Omega(|U|) space penalty during negation.
Approach / What changed
The author formalizes a DAG-based retrieval language, L_R, proves its evaluation is P-Complete, and introduces ComputePN. ComputePN is a deterministic, sparsity-aware evaluation algorithm that utilizes native DAG memoization and decouples logical negation from full universe materialization via a Positive-Negative dual representation.
Takeaways
The query evaluation problem for the DAG-based retrieval language L_R is strictly P-Complete.
Document-at-a-Time evaluation suffers an O(2^|Q|) exponential complexity blowup on re-convergent logic, whereas Term-at-a-Time incurs an Omega(|U|) space penalty on negation.
ComputePN uses a Positive-Negative dual representation and DAG memoization to bound evaluation time to O(|Q| · |U_active|), avoiding both tree-expansion bottlenecks and universal scans.
Source and freshness
- Source
- machinelearning.apple.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.