How to Make Apache Hudi Copy-On-Write Upserts Less Wasteful
Learn how Apache Hudi copy-on-write upserts can use row-level file and page locations to update changed data while avoiding unnecessary decompression and rewriting.
Loading post…
ClickhouseExplainer
AI Functions in ClickHouse: Upgrade your SQL to the AI age
Summary
ClickHouse’s beta AI Functions let SQL queries call an LLM or embedding provider directly, keeping classification, extraction, generation, translation, redaction, filtering, embedding, and similarity work alongside stored data. The feature addresses workflows that export rows to separate LLM, orchestration, or vector-database systems and then reload results, with the stated goal of running the full RAG cycle in ClickHouse. Functions make remote HTTP calls per input, or batch embedding inputs, and return native ClickHouse values; in open-source deployments, named collections store provider credentials and endpoint settings. Examples use Hacker News data for topic classification and natural-language filtering, while the guidance emphasizes cheap predicates, quotas, LIMIT tests, prompt-injection defenses, non-determinism, cost, latency, and provider access to data in cleartext after TLS termination.
Context
Data such as logs, reviews, descriptions, and support tickets already resides in ClickHouse, but traditional LLM workflows move it into separate classification, embedding, vector-database, orchestration, and API systems before loading results back. The post describes that process as slow, error-prone, and operationally complex.
Approach / What changed
AI Functions expose LLM and embedding operations as ClickHouse SQL functions. They call configured providers over HTTP, parse responses into native ClickHouse values, and can use named collections for provider credentials and endpoint configuration in open-source deployments. The examples apply these functions directly to Hacker News data.
Takeaways
AI Functions arrived across ClickHouse releases: aiGenerate(), aiClassify(), aiExtract(), and aiTranslate() in 26.4; aiEmbed() in 26.6; and aiFilter(), aiRedact(), and aiSimilarity() in 26.8.
aiFilter() returns UInt8, allowing a natural-language condition to be used directly in WHERE, PREWHERE, or JOIN ... ON clauses after cheaper predicates have narrowed the input.
API-call and token quotas are shared within an execution context; retries count toward call limits, distributed queries can apply limits per shard, and providers that omit usage data do not trigger token limits.
Source and freshness
- Source
- clickhouse.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
AssemblyScript: the TypeScript-shaped path into WebAssembly
AssemblyScript is a TypeScript-like way to target WebAssembly. This explainer shows how source becomes a sandboxed module, crosses explicit host boundaries, and fits performance-sensitive services.
How to Build an ASL Gloss Workflow That Keeps Context
Learn a practical workflow for translating English discourse into ASL gloss while preserving spatial references, question-answer structure, and consistent concept-to-gloss mappings.
Apache Pinot: From Event Stream to Resilient Query
Apache Pinot is a distributed real-time OLAP datastore for low-latency analytics. This architecture view shows how streams, processing, partitioning, indexed storage, and replica-aware serving fit together.
Before You Ship Nova, Set Four Gates
A practical checklist for putting Amazon Nova into an application: choose nova-lite or nova-pro by workload, expose tools, and verify context, modality, region, and cost constraints.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.