---
title: "Open Source"
description: "568 posts about Open Source, summarised, each linking to the original."
---

# Open Source
> 568 posts about Open Source, summarised, each linking to the original.

## Articles

### [Welcoming Diel Duarte, our new Open source Engineer](https://yomu.fyi/post/welcoming-diel-duarte-our-new-open-source-engineer.md)
- Company: [Resend](https://yomu.fyi/company/resend.md)
- Published: Apr 30, 2026
- Type: Announcement

Resend introduces Diel Duarte as its new open source engineer, bringing more than 13 years of experience building web applications and SDKs. His focus is developer experience, especially API design with clean ergonomics that lets developers concentrate on building rather than wrestling with interfaces. Diel describes a path from studying programming and computer fundamentals at Cotemig to following Resend as a user and joining after an interview process he found unusually human; he cites Resend’s DX and craft, along with Zeno’s open source work, as major references. Outside work, he draws inspiration from deliberately curated feeds, new languages and APIs, visual-design communities, and creator friends, while keeping music and interior design as interests; he names Warp Terminal as his favorite tool and Japan as a memorable destination.


### [Granite 4.1 LLMs: How They’re Built](https://yomu.fyi/post/granite-4-1-llms-how-they-re-built.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Yousaf Shah
- Published: Apr 29, 2026
- Type: Explainer

Granite 4.1 is a family of dense decoder-only language models in 3B, 8B, and 30B sizes, trained on approximately 15T tokens. Its five-phase pre-training pipeline moves from broad web-scale data to progressively curated mixtures, increases math and code emphasis, and extends context from 4K to 512K tokens through staged long-context training. Supervised fine-tuning uses about 4.1M curated samples evaluated with LLM-as-Judge and rule-based checks, while reinforcement learning uses on-policy GRPO with DAPO loss. The 8B instruct model matches or surpasses the earlier Granite 4.0-H-Small 32B-A9B MoE model despite its simpler dense architecture and fewer parameters, and the Granite 4.1 models are released under Apache 2.0.


### [From init to deploy: building with agents and Stripe Projects](https://yomu.fyi/post/from-init-to-deploy-building-with-agents-and-stripe-projects.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Anna Spysz
- Published: Apr 29, 2026
- Type: Tutorial

The post describes an internal agentic transcription app built to reduce the manual effort required to correct auto-generated subtitles for Stripe Developers videos. The app uses a Next.js interface with transcript, chat, and corrected-output panels, while Stripe Projects CLI links Vercel and OpenRouter and populates required environment variables. An LLM client supports selectable models, iterative instructions, and a transcription agent whose system prompt emphasizes preserving transcript structure; the agent can also use Stripe documentation search through MCP. In a benchmark on a 771-line meetup script, Gemini 2.5 Pro reached 99% accuracy, while the tested models reduced editing from more than 30 minutes to just over two minutes of generated edits plus a human check. The reported overall reduction was 4x, with testing costs totaling $2.54.


### [Comparing ClickHouse versions with clickhousectl](https://yomu.fyi/post/comparing-clickhouse-versions-with-clickhousectl.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Mark Needham
- Published: Apr 29, 2026
- Type: Tutorial

Evaluating query performance differences across ClickHouse versions previously required managing Docker containers or manually downloading historical binaries from GitHub releases. The clickhousectl command-line interface simplifies this process by downloading specific ClickHouse binaries, running multiple background server instances concurrently with automatic port assignment, and executing queries across local environments. Users can copy table schemas between local instances by piping raw output from SHOW CREATE TABLE queries and replicate datasets using ClickHouse's remote table function. In a benchmark comparing ClickHouse 25.12 and 26.3 on an S3-backed Parquet dataset, version 26.3 introduced a Parquet metadata cache that reduced repeat query times from roughly 9 seconds to 1–2 seconds. System tables like system.server\_settings and system.metrics enable operators to inspect cache configurations, eviction policies, and memory consumption directly.


### [Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents](https://yomu.fyi/post/introducing-nvidia-nemotron-3-nano-omni-long-context-multimodal-intell.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tuomas Rintamaki, Amala Sanjay Deshmukh, Nabin Mulepati, Collin McCarthy, Pritam Biswas, Arushi Goel, Alexandre Milesi, Danial Mohseni Taheri, Kateryna Chumachenko, Isabel Hulseman, Zhehuai Chen, Karan, Tao, Leili Tavabi, Ehsan Hosseini Asl
- Published: Apr 28, 2026
- Type: Announcement

NVIDIA Nemotron 3 Nano Omni is introduced as an omni-modal understanding model for document analysis, multiple-image reasoning, automatic speech recognition, long audio-video understanding, agentic computer use, and general reasoning. It extends the Nemotron multimodal line from vision-language processing to text, image, video, and audio, targeting long, dense inputs such as 100-plus-page documents and mixed audio-visual evidence. Its unified encoder-projector-decoder combines a Nemotron 3 Nano 30B-A3B hybrid Mamba-Transformer MoE backbone with C-RADIOv4-H and Parakeet-TDT-0.6B-v2 encoders; dynamic visual resolution, Conv3D tubelets, and Efficient Video Sampling support multimodal context compression. The post reports leading or top results across document, video, audio-video, voice, and ASR benchmarks, alongside up to 9x higher throughput and 2.9x single-stream reasoning speed on multimodal use cases, and provides BF16, FP8, and NVFP4 checkpoints.


### [Introducing the OSSCAR Index](https://yomu.fyi/post/introducing-the-osscar-index.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: prashant
- Published: Apr 28, 2026
- Type: Announcement

Supabase has launched the OSSCAR Index, a quarterly ranking of fastest-growing open source organizations, with the site, data, and scoring code released as open source. It addresses the limits of rankings based on accumulated stars, downloads, or contributors by measuring community growth within two divisions: Emerging organizations below 1,000 GitHub stars at quarter-start and Scaling organizations at or above that threshold. Scores combine normalized net new stars, unique contributors, and npm, PyPI, and Cargo downloads using an L² norm, with minimum thresholds and no penalty for missing or declining signals. The Q1 2026 edition identifies Openclaw as a breakout project, places AI-agent projects prominently in Emerging, and reports global representation; quarterly updates, expanded package coverage, an RFC process, and historical rankings are planned.


### [Formatting an entire 25 million line codebase overnight: the rubyfmt story](https://yomu.fyi/post/formatting-an-entire-25-million-line-codebase-overnight-the-rubyfmt-st.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Fable Tales
- Published: Apr 28, 2026
- Type: Problem & solution

Stripe’s 25-million-line Ruby codebase needed a true autoformatter; before rubyfmt, no industry-wide Ruby formatter existed, and earlier tools had crashed on its files. rubyfmt began as a zero-configuration open-source project motivated by formatting disputes, Ruby’s slow startup, and the uncertainty faced by engineers arriving from other languages. To meet a 100ms target, it initially used a hand-authored C launcher and later moved formatting into Rust, with ripper parse trees first passing through JSON and eventually being deserialized directly from Ruby VALUE objects. Stripe adopted per-file opt-in, expanded coverage cautiously, and formatted the codebase on a Saturday in 2024; the excerpt reports that all 42-million-line Ruby codebase is now formatted. In 2025, migrating from ripper to Prism removed the need to link a Ruby VM and made rubyfmt smaller and dramatically faster.


### [How to build scalable web apps with OpenAI's Privacy Filter](https://yomu.fyi/post/how-to-build-scalable-web-apps-with-openai-s-privacy-filter.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: yuvraj sharma, Freddy Boulton, Abubakar Abid
- Published: Apr 27, 2026
- Type: Tutorial

The article presents three web applications built around OpenAI’s open-source Privacy Filter, a PII detector that labels eight categories in one forward pass over a 128k-token context. Document Privacy Explorer extracts PDF or DOCX text and returns category-labeled spans, while Image Anonymizer combines Tesseract word boxes with character offsets to produce editable pixel redaction rectangles. SmartRedact Paste replaces detected spans with category placeholders, stores public and token-gated views, and evicts expired pastes in a daemon thread. Across the apps, gradio.Server’s @server.api routes model work through Gradio’s queue and gradio\_client, while plain FastAPI routes serve custom HTML, views, and lightweight lookups. The implementations keep browser-side filtering, canvas edits, and PNG export out of server round-trips, and the paste service is about 200 lines including storage.


### [DeepSeek-V4: a million-token context that agents can actually use](https://yomu.fyi/post/deepseek-v4-a-million-token-context-that-agents-can-actually-use.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: ben burtenshaw
- Published: Apr 24, 2026
- Type: Explainer

DeepSeek V4 is presented as an open model family aimed at long-running agentic workloads, with Pro and Flash MoE checkpoints offering 1M-token context windows. Its central architecture alternates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), compressing KV entries by 4x and 128x respectively, while using FP8/BF16 storage and FP4 indexing. The post-training design preserves reasoning across user turns when tool calls are present, introduces a dedicated XML tool-call schema, and relies on DSec, a Rust sandbox platform for RL rollouts. At 1M tokens, Pro uses 27% of V3.2's single-token FLOPs and 10% of its KV cache, while Flash uses 10% and 7%; V4-Pro-Max reports 80.6% on SWE Verified and 0.59 MRCR 8-needle retrieval at 1M.


### [Do you still need Elasticsearch for log analytics? ClickHouse says no.](https://yomu.fyi/post/do-you-still-need-elasticsearch-for-log-analytics-clickhouse-says-no.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Tom Schreiber, Lionel Palacin
- Published: Apr 23, 2026
- Type: Benchmark

ClickHouse introduced a redesigned full-text search powered by inverted indexes that feeds matching documents directly into its vectorized analytical engine for observability and log analytics workloads. To evaluate this capability, the open-source versions of ClickHouse and Elasticsearch were benchmarked on single-node AWS EC2 instances across synthetic OpenTelemetry log datasets scaling up to 50 billion rows. The benchmark evaluated nine representative query patterns combining text search with filtering, grouping, counting, and time-series aggregations under both cold and hot cache conditions. ClickHouse achieved a roughly fivefold smaller storage footprint on disk while running cold queries four to six times faster and hot queries approximately 1.7 to 2.6 times faster than Elasticsearch. Furthermore, ClickHouse ingested the 50-billion-row dataset in under four hours out of the box, whereas the tuned Elasticsearch setup required approximately five days.


### [QIMMA قِمّة ⛰: A Quality-First Arabic LLM Leaderboard](https://yomu.fyi/post/qimma-a-quality-first-arabic-llm-leaderboard.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Leen AlQadi, Ahmed Alzubaidi, Mohammed Alyafeai, Maitha Alhammadi, Shaikha Alsuwaidi, Omar saif alkaabi, Basma Boussaha, Hakim Hacid
- Published: Apr 21, 2026
- Type: Benchmark

QIMMA is an Arabic LLM leaderboard built to address fragmented evaluation, translation artifacts, weak benchmark quality checks, limited coverage, and reproducibility gaps. It treats benchmark validation as a prerequisite for reported scores rather than an afterthought. The suite consolidates 109 subsets from 14 source benchmarks into more than 52,000 samples across seven domains, with 99% native Arabic content, Arabic-adapted HumanEval+ and MBPP+ coding tasks, and six prompt templates. Before model evaluation, Qwen3-235B-A22B-Instruct and DeepSeek-V3-671B assess samples using a 10-point rubric, flagged cases receive native-speaker review, and code statements are refined without changing task identifiers, reference solutions, or tests. In the April 2026 rankings, Qwen/Qwen3.5-397B-A17B-FP8 ranks first, while Arabic-specialized models lead cultural and linguistic tasks and multilingual models lead coding.


### [AI and the Future of Cybersecurity: Why Openness Matters](https://yomu.fyi/post/ai-and-the-future-of-cybersecurity-why-openness-matters.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Margaret Mitchell, Yacine Jernite, Clem 🤗
- Published: Apr 21, 2026
- Type: Explainer

Mythos is presented as a frontier AI system that rapidly finds and patches software vulnerabilities, with the post arguing that its surrounding system matters more than the language model alone. Its recipe combines substantial compute, software-relevant training data, vulnerability-probing and patching scaffolding, speed, and some autonomy; together, these elements can uncover vulnerabilities, find exploits, and build patches. The post argues that open code and tooling can distribute detection, verification, coordination, and patch propagation across communities, while reducing capability asymmetry between attackers and defenders. For defensive use, it recommends semi-autonomous agents with prespecified actions, human approval, auditable logs, and private deployment using open components and existing security tools. This approach is especially relevant to high-stakes organizations that need inspectable, controllable systems and protection for sensitive data.


### [otel.fyi - OTel Collector docs made simple](https://yomu.fyi/post/otel-fyi-otel-collector-docs-made-simple.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: The ClickStack Team
- Published: Apr 17, 2026
- Type: Problem & solution

OpenTelemetry Collector documentation is often fragmented across separate contrib repositories as individual README files, making component configuration difficult and time-consuming to navigate. To address this friction, the team created otel.fyi as a dedicated platform to aggregate, simplify, and structure documentation from across the ecosystem. The implementation parses contrib repositories to pull configuration examples and normalize metadata across receivers, processors, exporters, and extensions into a unified format. By surfacing structured component information upfront, the site allows practitioners to assemble working configurations rapidly without digging through multiple GitHub repositories. Originally developed for internal operational workflows, the platform has been released publicly to assist the broader community, with future plans to introduce an AI assistant.


### [April 2026 newsletter](https://yomu.fyi/post/april-2026-newsletter.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Mark Needham
- Published: Apr 16, 2026
- Type: Announcement

ClickHouse 26.3 LTS introduces experimental materialized common table expressions, WebAssembly user-defined functions, and enables asynchronous inserts by default. Alongside the release, Alasdair Brown launched clickhousectl to manage local installations and cloud deployments with built-in agentic development features and authentication safeguards. User case studies show Goldsky accelerating historical blockchain backfills by twelve times through direct ClickHouse reads using Apache Arrow, replacing their prior Kafka and Avro architecture. Additionally, the engineering team redesigned the full-text search index with front-coding compression, a sparse lookup index, and adaptive posting lists to handle expensive random reads on object storage. ClickHouse also previewed community architecture patterns for lightweight SIEM setups using Clickdetect and Wazuh log collection.


### [Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents](https://yomu.fyi/post/ecom-rlve-adaptive-verifiable-environments-for-e-commerce-conversation.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rahul Bajaj, Jaya Nupur, Anuj Garg, ben burtenshaw
- Published: Apr 16, 2026
- Type: Explainer

Ecom-RLVE extends the RLVE framework from single-turn reasoning puzzles to multi-turn, tool-augmented e-commerce conversations. EcomRLVE-GYM supplies eight environments spanning product discovery, substitution, cart building, returns, order tracking, policy QA, bundle planning, and multi-intent journeys, with procedural problem generation, a 12-axis difficulty curriculum, and algorithmically verifiable rewards. Each episode requires tool use and world-state changes; rewards combine task completion, efficiency, and a penalty for recommending product IDs not retrieved, while malformed outputs and illegal calls fail immediately. Adaptive scheduling advances each environment independently when its success rate is reliable. An initial study trained Qwen 3 8B with DAPO for 300 steps on cart building and reported progressively increasing difficulty reached, consistent with a steady learning signal.


### [The PR you would have opened yourself](https://yomu.fyi/post/the-pr-you-would-have-opened-yourself.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pedro Cuenca, Awni Hannun
- Published: Apr 16, 2026
- Type: Announcement

mlx-lm contributors can use a Skill and a separate test harness to port language models from transformers, with the goal of making new architectures available shortly after their upstream implementations land. The work addresses rising agent-generated pull-request volume and the difficulty of preserving implicit codebase conventions, user contracts, and performance-sensitive behavior that agents may not understand. Given a conversion prompt, the Skill sets up environments, finds model variants and checkpoints, reads transformers modeling code, writes an MLX implementation, and iterates through tests including dtype checks, generation examples, numerical comparisons, and per-layer comparisons. The resulting pull requests disclose agent assistance and include detailed verification data, while contributors remain responsible for reviewing and accepting the results. Current limitations include shared-utility refactors, vision-language models, quantized uploads, and thinking-specific tests.


### [Getting started with clickhousectl: the ClickHouse CLI for local and cloud](https://yomu.fyi/post/getting-started-with-clickhousectl-the-clickhouse-cli-for-local-and-cl.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Al Brown
- Published: Apr 16, 2026
- Type: Tutorial

ClickHouse developers often face friction managing local versions, isolated instances, project scaffolding, and Cloud operations without relying on complex shell scripts. The open-source command-line tool clickhousectl addresses this by unifying local environment management and ClickHouse Cloud workflows. Locally, it downloads shared binaries, initializes standardized directory structures, and spins up named, isolated servers with automatic port assignments to facilitate upgrade testing. For cloud workflows, the CLI enforces a split authentication model where browser OAuth provides read-only access and API keys permit state mutations like service creation and scaling. Furthermore, the tool includes Claude Code agent skills covering best practices and deployment workflows, alongside support for CI/CD pipeline automation via environment variables and JSON output.


### [Autoresearch isn’t just for training models](https://yomu.fyi/post/autoresearch-isn-t-just-for-training-models.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 15, 2026
- Type: Problem & solution

Repeated visual-regression failures made Polaris CI feedback cycles take about 30 minutes, prompting an attempt to optimize build time rather than fix individual tasks. The approach uses Pi to create an Autoresearch extension that repeatedly forms a hypothesis, measures the build against a 19.1-second baseline, and keeps only changes that improve the metric, discarding crashes and slower runs. The loop identified wasted VRT work: the full component pipeline ran before Storybook recompilation, while TypeScript transformed all 580 component files although 105 needed it. Removing those inefficiencies made the build 65% faster, although the system also produced unacceptable hacks that were discarded. The extension later gained multi-metric support, consistent iteration scripts, auto commits, and open-source adoption, with reported internal improvements including unit tests running 300 times faster and Playwright tests becoming faster.


### [Index-based pruning in ClickHouse](https://yomu.fyi/post/index-based-pruning-in-clickhouse.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Mark Needham
- Published: Apr 15, 2026
- Type: Explainer

ClickHouse accelerates analytical queries by minimizing the amount of data read from disk across granules. A primary key determines data part sort order and records the first value per granule, enabling binary search to skip unneeded granules during filtered queries. For non-primary key filters, lightweight projections act like secondary indexes by storing alternative sort keys alongside base table offset pointers without duplicating full table rows. Skip indexes, such as minmax indexes, record column boundaries per granule and operate without disk duplication, though effective pruning requires some correlation with the primary key. Demonstrations using a UK property price dataset show how primary key filters and minmax skip indexes eliminate large fractions of total granules.


### [What's new in ClickStack - March 2026](https://yomu.fyi/post/what-s-new-in-clickstack-march-2026.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: The ClickStack Team
- Published: Apr 14, 2026
- Type: Announcement

ClickStack announced several observability updates in its March 2026 release to improve root cause analysis, dashboard organization, and local development. The Event Deltas tool now displays baseline span distributions immediately, uses deterministic cityHash64 sampling for trace heatmaps, and applies proportional comparison scoring to accurately evaluate attributes across uneven sample sizes. Furthermore, users can directly include or exclude values from attribute comparison bars to streamline query creation during incident investigations. Dashboard capabilities expanded with a curated OpenTelemetry template gallery, multi-value filtering with optional lookup WHERE clauses, and broadened metric display units. Finally, local mode introduces browser localStorage persistence through createEntityStore, allowing developers to save dashboards and favorites without a running backend API.


[Newer posts](https://yomu.fyi/topic/open-source/page/11.md) · [Older posts](https://yomu.fyi/topic/open-source/page/13.md)