Crowdsourced taxonomy verification: A feedback-driven framework for refining knowledge graph relationships via online search interactions

Summary

Dynamic catalogs in food delivery and e-commerce make knowledge-graph (KG) maintenance difficult: LLM- or scraper-generated relationships can be semantically wrong, manual review does not scale, and bad edges can degrade query expansion, CTR, and trust. The proposed feedback-driven verification engine treats unverified edges as hypotheses and uses the search interface as a validation environment. It separates verified and candidate edges, injects a limited number of candidates into lower-risk interface positions through an exploration-versus-exploitation policy, and records context-anchored user interactions with weighted positive and negative signals. An offline engine aggregates exposure and engagement into a confidence ratio, promoting candidates above a verification threshold and pruning those below a failure threshold. The framework is presented as a continuous human-in-the-loop method for validating parent-child and sibling relationships at scale; planned extensions add confidence tiers and defenses against spam, bots, and Sybil attacks.

Takeaways

1. 1

   Candidate edges remain hypotheses until live search interactions provide evidence; they are kept separate from verified graph structure and excluded from the main search algorithm until validated.

2. 2

   The behavior tracker anchors each signal to the user’s search term, so an interaction with a candidate relationship is interpreted in its query context rather than as an unqualified vote.

3. 3

   The proposed next stage uses progressive trust states requiring both support mass and corroboration, with automatic demotion when metrics fall below a tier floor; abuse controls exclude flagged traffic and cap cohort contributions.