---
title: "Open Source"
description: "568 posts about Open Source, summarised, each linking to the original."
---

# Open Source
> 568 posts about Open Source, summarised, each linking to the original.

## Articles

### [Reinforcement learning is an infrastructure problem](https://yomu.fyi/post/reinforcement-learning-is-an-infrastructure-problem.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: Jun 1, 2026
- Type: Announcement

Reinforcement learning for post-training large language models has become infrastructure-constrained as teams move to multi-node configurations for larger models like Qwen 3 30B and Kimi K2.6. The RL loop divides into three independent infrastructure challenges: training with reliable forward and backward passes, high-performance inference rollouts, and thousands of concurrent sandbox environments for policy execution. Weight synchronization between trainer and rollout engine becomes the critical bottleneck in multi-node setups, with RDMA offering up to 100x speedup over TCP and significant cost savings per step. Teams commonly get stuck maintaining glue code, queueing for cluster time, and under-utilizing GPUs when sandbox buffers are not correctly sized. Modal addresses these with RDMA-connected clusters via a single rdma=True keyword, fast container boot, and the new open-source Modal Training Gym library that abstracts common scaffolding into under 100 lines of code.


### [Open House 2026 Day 2: 10 years of open source and what the ecosystem is building next](https://yomu.fyi/post/open-house-2026-day-2-10-years-of-open-source-and-what-the-ecosystem-i.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 28, 2026
- Type: Announcement

Open House 2026 Day 2 recaps ClickHouse’s tenth year as an open-source project and the ecosystem forming around its data platform. The event reported more than 2,600 contributors, nearly 48,000 GitHub stars, over 200 integrations, and the launch of the House Mates partner program. Announcements included Fivetran’s generally available ClickHouse Cloud destination, a Google Cloud Dataflow Pub/Sub template, Sigma’s public-beta connector and ETL Cache Layer, and a ClickHouse dbt Fusion adapter available in alpha through the CLI. The recap also describes MCP and agent-oriented observability work, including a ClickStack MCP server that translates investigations into optimized ClickHouse queries. Internal testing reported up to 25% fewer tool calls, 2.5 times greater consistency, and nearly 20% better evaluation results.


### [Open House observability announcements: MCP server, AI Notebooks, and ClickStack Cloud](https://yomu.fyi/post/open-house-observability-announcements-mcp-server-ai-notebooks-and-cli.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At Open House, ClickHouse announced three observability updates: ClickStack Cloud entered private preview, AI Notebooks entered beta for Managed ClickStack, and a ClickStack MCP server was introduced in open-source ClickStack. ClickStack Cloud is a fully managed, serverless platform that accepts OpenTelemetry data through a managed endpoint, while Managed ClickStack is generally available for teams controlling collectors, ingestion, compute sizing, schema design, and tuning. AI Notebooks provide a persistent, shareable investigation workspace in which prompts, queries, charts, reasoning steps, findings, branches, intermediate results, and generated queries remain visible and editable. The MCP server exposes the investigative tools used by Notebooks to external agents, routing requests through optimized searches, aggregations, and visualizations rather than relying entirely on ad hoc SQL generation; it can also persist findings in dashboards. Together, the announcements present a choice between managed convenience, operational control, and programmable observability workflows.


### [Thank you for building with us](https://yomu.fyi/post/thank-you-for-building-with-us.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At its 2026 user conference in San Francisco, ClickHouse reports surpassing 4,000 customers and $250 million in annual run-rate revenue, while marking nearly ten years since Alexey Milovidov open-sourced the database. The company attributes its growth to a community of contributors and describes use across AI products, financial services, observability, analytics, and infrastructure. New offerings include ClickHouse Agents, a fully managed agentic analytics service powered by Anthropic’s Claude; CostBench, an open benchmark using each warehouse’s real billing model; and House Mates, a partner program launching with more than 50 partners across six continents. Additional updates include Managed Postgres in public beta, Managed ClickStack and Langfuse for AI-era observability, generally available full-text search, and automatic JOIN optimization, with cost-performance identified as the central constraint for high-concurrency, low-latency workloads.


### [ClickHouse tops $250M ARR and 4,000 customers, launches Claude-powered agents at Open House 2026](https://yomu.fyi/post/clickhouse-tops-250m-arr-and-4-000-customers-launches-claude-powered-a.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At Open House 2026, ClickHouse announced that its serverless cloud offering exceeded $250 million in annual run-rate revenue and reached 4,000 customers, with ARR more than tripling year-over-year and more than 1,000 net new customers added since January. The company launched ClickHouse Agents, a fully managed, no-code agentic analytics service powered by Anthropic’s Claude, with a chat interface, sandboxed code interpreter, artifacts, skills, memory, multi-agent workflows, and MCP-compatible integrations. It also published CostBench, an open benchmark applying each vendor’s real compute billing model to the same workload; ClickHouse Cloud was reported to have 23x better cost-performance than the nearest cloud data warehouse. Other announcements included Managed Postgres in public beta, managed ClickStack, generally available full-text search, cross-region replication, and the House Mates partner program.


### [Amazon Research Awards recipients announced](https://yomu.fyi/post/amazon-research-awards-recipients-announced.md)
- Company: amazon.science
- Author: Amazon Research Awards team
- Published: May 27, 2026
- Type: Announcement

Amazon Research Awards has announced 70 award recipients across 49 universities in 11 countries for its fall 2025 funding cycle. The funded research spans six distinct tracks: AI for Information Security, Agentic AI, Automated Reasoning, AWS Cryptography, Cybersecurity and Anti-Abuse Technologies, and Sustainability. Each selected academic team receives unrestricted financial grants, AWS Promotional Credits to access specialized cloud AI and machine learning services, and access to more than 700 Amazon public datasets. Awardees are paired with an Amazon research contact who provides advisory support, consultation, and access to company training sessions. Furthermore, the program explicitly encourages recipients to publish their discoveries, deliver presentations at Amazon offices worldwide, and release associated software implementations under open-source licenses.


### [Reachy Mini goes fully local](https://yomu.fyi/post/reachy-mini-goes-fully-local.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amir Mahla, Andres Marafioti
- Published: May 27, 2026
- Type: Tutorial

Reachy Mini’s conversation app can now run its speech stack locally instead of sending audio to a server, avoiding cloud, API keys, and data leaving the machine. The tutorial uses speech-to-speech as a cascaded VAD → STT → LLM → TTS pipeline exposed through a Realtime API-compatible /v1/realtime WebSocket, with Silero VAD v5, Parakeet-TDT 0.6B v3, Qwen3-TTS, and a selectable LLM backend. It shows llama.cpp serving Gemma 4 E4B with parallel slots, 64k context, flash attention, and full sliding-window attention, then covers vLLM, MLX, Transformers, hosted Responses API endpoints, and network connection to a wireless robot. Once the backend and app are connected, conversations run through the local loop, while individual stages can be exchanged to trade latency, quality, language coverage, and hardware choices.


### [Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL](https://yomu.fyi/post/shipping-a-trillion-parameters-with-a-hub-bucket-delta-weight-sync-in.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amine Dirhoussi, Quentin Gallouédec, Kashif Rasul, Lewis Tunstall, Edward Beeching, Albert Villanova del Moral, Leandro von Werra, Sergio Paniego
- Published: May 27, 2026
- Type: Problem & solution

Async RL weight synchronization currently requires the trainer to send a complete model to the inference engine after each optimizer step, creating a critical-path transfer that can reach roughly a terabyte for a frontier 1T-parameter checkpoint. The article attributes the opportunity for reduction to bf16 arithmetic: at RL learning rates, many updates fall below the bf16 visibility threshold, leaving about 99% of weights bit-identical between steps and more than 98% even in the reported worst case. TRL's delta-weight-sync implementation records changed elements around the optimizer step, encodes them in sparse safetensors, uploads each delta to a Hugging Face Bucket, and lets vLLM fetch and apply it. On Qwen3-0.6B, this reduces each per-step payload from 1.2 GB to 20–35 MB, while a demonstrated setup connected separate trainer, vLLM, and Wordle Spaces through one bucket. Open issues include extra CPU bf16 snapshots, fixed anchor cadence, and unmeasured multi-node FSDP2 support.


### [5 countries, 12 days, one ClickHouse: inside Alexey's APJ AI Tour](https://yomu.fyi/post/5-countries-12-days-one-clickhouse-inside-alexey-s-apj-ai-tour.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

ClickHouse CTO and creator Alexey Milovidov completed a 12-day APJ tour spanning India, Singapore, Indonesia, South Korea, and Japan, with customer meetings, industry events, executive sessions, press interviews, and community meetups. The itinerary covered six cities and included three Confluent Data Streaming World Tour events, four ClickHouse meetups, and more than 10 enterprise customer meetings across varied industries and workloads. Discussions centered on real-time analytics, connected products, and analytics at industrial scale, while in-person conversations supplied feedback that the post says helps improve ClickHouse. The trip concluded with the ClickHouse Japan Launch Event on April 27, marking the company’s official entry into that market. The account presents physical presence as a way to extend ClickHouse’s open-building culture beyond GitHub and strengthen its feedback loop with APJ customers and communities.


### [Introducing CostBench: an open benchmark for data warehouse cost-performance](https://yomu.fyi/post/introducing-costbench-an-open-benchmark-for-data-warehouse-cost-perfor.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Tom Schreiber, Lionel Palacin
- Published: May 27, 2026
- Type: Benchmark

Traditional data warehouse benchmarks measure query execution speed without accounting for the varying compute costs across platforms. CostBench introduces an open benchmarking framework designed to evaluate data warehouses on a performance-per-dollar metric across both read and write workloads. Its initial release evaluates ClickHouse Cloud, Snowflake, Databricks, BigQuery, and Redshift using 43 production-derived analytical queries on an anonymized dataset combined with each vendor's compute billing model. In these read-side tests, ClickHouse Cloud maintained high speed at lower costs as data scaled, outperforming the nearest competitor by 23 times in cost-performance. The benchmark repository openly publishes its scripts, configurations, pricing assumptions, and raw JSON results to ensure all comparative metrics remain fully inspectable and reproducible.


### [Integrating the Rust Delta Kernel into ClickHouse](https://yomu.fyi/post/integrating-the-rust-delta-kernel-into-clickhouse.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Melvyn Peignon, Kseniia Sumarokova, Raúl Marín
- Published: May 22, 2026
- Type: Problem & solution

ClickHouse initially built native protocol support to read Delta Lake tables, but maintaining pace with the format's evolving specification created high complexity and fragmented features. To resolve this, engineers integrated the Rust Delta Kernel to offload protocol handling while retaining ClickHouse's optimized Parquet data reading. The Delta Kernel parses JSON transaction logs, resolves snapshots, manages transactional write metadata, and exposes schema evolution mapping alongside Change Data Feed streams. ClickHouse also contributed upstream enhancements to the Rust kernel, adding dynamic runtime logging and asynchronous metadata processing via FFI handles. While ClickHouse can attach to existing Delta tables, the integration cannot yet initialize empty Delta Lake tables.


### [OlmoEarth v1.1: A more efficient family of Earth observation models](https://yomu.fyi/post/olmoearth-v1-1-a-more-efficient-family-of-earth-observation-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kyle Wiggers
- Published: May 19, 2026
- Type: Announcement

OlmoEarth v1.1 is a new family of Earth observation models designed to reduce the compute required for large-scale satellite imagery predictions. Compute dominates the lifecycle from data export through preprocessing, inference, and post-processing, motivating a focus on transformer token sequence length. For Sentinel-2, the models collapse three resolution-specific tokens into one, reducing token counts threefold; this initially caused a 10 ppt drop on the m-eurosat kNN benchmark, so the pre-training regimen was modified. OlmoEarth v1.1 runs up to three times cheaper than v1 while maintaining similar performance across a mix of research benchmarks and partner-built tasks, although some regressions remain, and the same dataset is used to isolate methodological effects.


### [What's new in ClickStack - April 2026](https://yomu.fyi/post/what-s-new-in-clickstack-april-2026.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: The ClickStack Team
- Published: May 19, 2026
- Type: Announcement

ClickStack's April release introduces several core enhancements across querying, alerting, and dashboard visualization capabilities. The platform added SQL-powered alerting, allowing teams to express complex statistical rules like rolling baselines and anomaly detection using native ClickHouse SQL and window functions without external pipelines. To improve search and filtering performance across common observability workloads, the default otel\_logs schema was redesigned with ClickHouse text indexes and adjusted ordering keys. Furthermore, heatmaps transitioned into a first-class chart type inside the shared visualization system for latency distribution tracking. The update also delivers persistent alert execution error reporting, expanded threshold comparison operators, and per-series number formatting across dashboards.


### [Introducing the Ettin Reranker Family](https://yomu.fyi/post/introducing-the-ettin-reranker-family.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen
- Published: May 19, 2026
- Type: Announcement

Tom Aarsen introduces six Sentence Transformers CrossEncoder rerankers, ranging from 17M to 1B parameters, alongside their training data and full recipe. Each model uses an Ettin ModernBERT encoder with unpadded attention, RoPE, GeGLU, up to 8,192 tokens of context, and a modular classification head whose ablations favored CLS pooling over mean pooling. Training applies pointwise MSE distillation to scores from mixedbread-ai/mxbai-rerank-large-v2 using a broad and retrieval-focused dataset mixture. The conclusion reports state-of-the-art results at every released size, with the 400M model within 0.0024 and the 1B model within 0.0001 of the 1.54B teacher on MTEB, while all models outperform the ms-marco-MiniLM-L\*-v2 family on MTEB and NanoBEIR.


### [Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality](https://yomu.fyi/post/granite-embedding-multilingual-r2-open-apache-2-0-multilingual-embeddi.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Radu Florian, Parul Awasthy, Aashka Trivedi, Madison Lee
- Published: May 14, 2026
- Type: Announcement

Granite Embedding Multilingual R2 introduces two Apache 2.0 multilingual embedding models built on ModernBERT: a 97M-parameter compact model and a 311M full-size model. Both cover more than 200 languages, receive enhanced retrieval training for 52 languages and nine programming languages, and support context lengths up to 32,768 tokens, compared with 512 for the R1 models. The compact model scores 60.3 on MTEB Multilingual Retrieval, while the full-size model scores 65.2; the reported results place them among the strongest open models in their parameter ranges. R2 replaces the XLM-RoBERTa architecture with alternating attention lengths, rotary position embeddings, newer multilingual and code tokenizers, and Flash Attention 2.0 support, while the 311M model adds Matryoshka embeddings and both models provide ONNX and OpenVINO weights.


### [Faster Queries and New Capabilities with the Open-Source Databricks JDBC Driver](https://yomu.fyi/post/faster-queries-and-new-capabilities-with-the-open-source-databricks-jd.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Toussaint Webb, Gopal Lal, Kaitlin Baumgardner
- Published: May 12, 2026
- Type: Announcement

The open-source Databricks JDBC driver 3.x and later releases address connectivity needs affecting dashboards, spreadsheets, applications, and other data workflows. Compared with the legacy 2.x driver, they deliver up to 30% faster retrieval of large query results while preserving backward compatibility. Architectural changes add full Arrow-based transfer for JDK 16+, a first-class asynchronous execution API, stream-based ingestion into Databricks Volumes, and integration with the Statement Execution API. The driver also supports stored procedures, multi-statement transactions, Unity Catalog metric views, query tags, geospatial and complex data types, plus built-in client telemetry for query latency, metrics, and errors. Its open-source ownership and Databricks maintenance are presented as enabling faster fixes, new features, transparency, contributions, and tighter platform integration.


### [ClickHouse Release 26.4](https://yomu.fyi/post/clickhouse-release-26-4.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: May 8, 2026
- Type: Announcement

ClickHouse Release 26.4 introduces 39 new features, 45 performance optimizations, and 238 bug fixes, focusing on broader SQL compatibility and indexing improvements. The update expands standard SQL support by allowing VALUES clauses as table expressions with column aliases, PostgreSQL-style EXTRACT units, NATURAL JOIN, and SET TIME ZONE commands. For text search, LIKE and ILIKE pattern queries using splitByNonAlpha tokenizers scan inverted index dictionaries instead of executing full-table scans when the dedicated evaluation setting is active. The release also provides the JSONAllValues function to extract all leaf values from a JSON column into a string array, enabling text skip indexes that significantly reduce scanned data and improve query latency.


### [Introducing Postgres Query Insights in ClickHouse Cloud](https://yomu.fyi/post/introducing-postgres-query-insights-in-clickhouse-cloud.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Amog Iska
- Published: May 7, 2026
- Type: Announcement

ClickHouse Cloud introduced Query Insights in preview for Managed Postgres to help teams diagnose slow database queries. Powered by the open-source extension pg\_stat\_ch, the tool streams per-statement telemetry directly into ClickHouse to enable fast aggregations across billions of execution rows. Postgres normalizes query literals into placeholders during the parse-analyze phase before telemetry leaves the database, preventing sensitive data exposure. Telemetry emission maintains around three percent producer overhead by utilizing a non-blocking try-lock on a shared-memory ring buffer. Users can examine high-level database health metrics, rank query patterns by runtime or latency percentiles, and inspect execution flyouts detailing disk spills, cache hits, and parallel worker behavior.


### [Fixing JavaScript observability, one library at a time](https://yomu.fyi/post/fixing-javascript-observability-one-library-at-a-time.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Abdelrahman Awad
- Published: May 7, 2026
- Type: Problem & solution

JavaScript application performance monitoring tools rely on monkey-patching via require-in-the-middle and import-in-the-middle, which breaks with ECMAScript Modules, bundlers, and non-Node runtimes. To solve this, an initiative is replacing monkey-patching with Node's built-in diagnostics\_channel TracingChannel API across server-side JavaScript libraries. Under this pattern, libraries publish structured events while monitoring vendors subscribe to them with zero overhead when unlistened. The author leveraged Claude Code workflows alongside direct maintainer communication to draft proposals, implement code, and manage reviews across 44 target packages. Ten libraries including mysql2, node-redis, ioredis, and unjs modules have merged support, while active efforts continue on shared OpenTelemetry mapper registries and remaining ecosystem packages.


### [Adding Benchmaxxer Repellant to the Open ASR Leaderboard](https://yomu.fyi/post/adding-benchmaxxer-repellant-to-the-open-asr-leaderboard.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Eric Bezzam, Steven Zheng, Eustache Le Bihan, Sergio Bruccoleri, Jeanine Sinanan-Singh, Casey Ford, Guanbo Wang, Yukai Huang, Ke Li, Yufeng Hao, Liao Xiaoling
- Published: May 6, 2026
- Type: Announcement

The Open ASR Leaderboard is adding private English speech-recognition datasets from Appen Inc. and DataoceanAI to evaluate scripted and conversational speech across several accents. To reduce benchmark-specific optimization (“benchmaxxing”) and test-set contamination, the datasets remain private, while the default Average WER continues to use public datasets only. A toggle exposes the private data, with macroaverages for overall, scripted, conversational, American, and non-American subsets; individual split scores are withheld to discourage provider- or accent-specific optimization. The leaderboard uses a shared Hub dataset, normalized transcripts and outputs, and open-source evaluation code, while private evaluation is run after public-set verification when models are submitted.


[Newer posts](https://yomu.fyi/topic/open-source/page/10.md) · [Older posts](https://yomu.fyi/topic/open-source/page/12.md)