---
title: "Open Source"
description: "568 posts about Open Source, summarised, each linking to the original."
---

# Open Source
> 568 posts about Open Source, summarised, each linking to the original.

## Articles

### [NVIDIA Cosmos-H-Dreams: Bringing Real-Time Generative Simulation to Surgical Robotics](https://yomu.fyi/post/nvidia-cosmos-h-dreams-bringing-real-time-generative-simulation-to-sur.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Lukas Zbinden, Javier Gamazo, Mostafa Toloui, Sean Huver
- Published: Jul 27, 2026
- Type: Announcement

Surgical robotics needs safer, faster ways to train and evaluate vision-language-action policies because physical experiments are costly, slow to reproduce, and vulnerable to damaging instruments or biological material. NVIDIA introduces Cosmos-H-Dreams, a real-time, action-conditioned generative simulator that specializes Cosmos-H-Surgical-Simulator for da Vinci Research Kit tabletop suturing and generates future frames from an initial RGB image and live robot kinematics. Its causal student is trained through teacher fine-tuning, causal warmup, and self-forcing distillation, including failure episodes and long autoregressive horizons, while few-step diffusion reduces generation cost. Served through FlashDreams with streaming KV caching, CUDA Graph capture, and model compilation, it reaches approximately 160 frames per second on one NVIDIA RTX PRO 6000, compared with roughly 10 frames per second for standard inference. The system supports human or learned-policy control, but is presented as a research and development platform rather than a diagnostic system or physical-robot controller.


### [Six Agent Harness Capabilities for Higher Model Performance](https://yomu.fyi/post/six-agent-harness-capabilities-for-higher-model-performance.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jul 27, 2026
- Type: Announcement

NVIDIA Labs released NVIDIA Labs Object-Oriented Agents (NOOA), an open-source research preview treating AI agents as single Python classes. The harness implements six model-facing capabilities: typed inputs and outputs, passing live objects by reference, code-based actions, programmable loops, explicit object state, and model-callable harness APIs. Instead of context compaction, passing live references keeps transcripts append-only, preserving prompt cache validity across long sessions. NOOA also includes a curated SQLite-backed long-term memory system forming a typed knowledge graph. Across evaluations, NOOA achieved 82.2 percent on SWE-bench Verified with GPT-5.5 using roughly 1.1M tokens per task, 86.8 percent on CyberGym L1 without network access, and 85.1 percent mean RHAE on ARC-AGI-3 using GPT-5.6-sol at under twenty dollars per game.


### [Amazon is investing in the Lean Focused Research Organization](https://yomu.fyi/post/amazon-is-investing-in-the-lean-focused-research-organization.md)
- Company: amazon.science
- Author: Byron Cook, Shawn Bice
- Published: Jul 26, 2026
- Type: Announcement

As AI agents handle high-stakes operational and financial decisions, traditional software testing fails to provide exhaustive correctness guarantees across all possible inputs. Mathematical proof offers deterministic verification that systems cannot behave incorrectly, but requires accessible tooling to function at modern software scale. To address this, Amazon announced substantial long-term financial backing for the Lean Focused Research Organization, representing the largest donation in the organization's history. Internally, Amazon relies on Lean for verifying differential-privacy mechanisms in AWS Clean Rooms, hardware compilation in AWS Neuron, and policy constraints within Amazon Bedrock AgentCore. Developing Lean externally through an open-source research organization provides independent auditability for regulators and accelerates community-driven tooling for neurosymbolic AI safety.


### [Debugging Ray Tracing Applications Using NVIDIA OptiX Toolkit](https://yomu.fyi/post/debugging-ray-tracing-applications-using-nvidia-optix-toolkit.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 23, 2026
- Type: Problem & solution

NVIDIA OptiX ray tracing applications often present difficult debugging challenges, including invalid API arguments, black frames, and concurrency issues across thousands of GPU threads. The open-source NVIDIA OptiX Toolkit addresses these diagnostic hurdles by offering unified error checking and targeted device-side debug logging. Host-side macros like OTK\_ERROR\_CHECK wrap OptiX, CUDA runtime, and CUDA driver API calls to enforce uniform error validation while capturing file and line diagnostics. For device-side troubleshooting, the toolkit introduces a DebugLocation utility to isolate logging to specific launch indices instead of flooding the output stream with concurrent thread data. This device mechanism also supports interactive one-shot debug dumps and screen highlights around the inspected pixel.


### [Observability for every Supabase project with Grafana Cloud](https://yomu.fyi/post/observability-for-every-supabase-project-with-grafana-cloud.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: alex\_hall, matt\_linkous, raminder\_singh
- Published: Jul 23, 2026
- Type: Announcement

Supabase now offers a one-click Grafana Cloud integration from its Dashboard, with the setup available on every plan, including the free plan. The setup provisions authentication, metric scraping, and a pre-built observability dashboard that tracks more than 200 metrics through Supabase’s Metrics API. Those metrics cover CPU and memory pressure, disk I/O, network traffic, connections and pooler behavior, replication lag, checkpoints, database size, and query statistics, while the same open-source dashboard definitions support self-hosted Grafana, the existing Grafana Cloud integration, and the new setup. The release is focused on metrics and is designed to accommodate logs without architectural changes; support for log drains to Grafana Cloud is already in progress. Users can start with Grafana Cloud’s free tier without a credit card on either side.


### [Bringing Nunchaku 4-bit Diffusion Inference to Diffusers](https://yomu.fyi/post/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pham Hong Vinh, Sayak Paul
- Published: Jul 23, 2026
- Type: Announcement

Modern diffusion transformers running in BF16 often require 20–30 GB of VRAM, limiting access on consumer GPUs; the integration brings Nunchaku Lite checkpoints into Diffusers to reduce memory and denoising latency. SVDQuant handles activation outliers by moving them into weights, preserving a small 16-bit low-rank branch while quantizing residual weights to 4-bit, enabling W4A4 compute in transformer attention and MLP projections. Nunchaku Lite patches standard nn.Linear modules at runtime, loads Hub-hosted CUDA kernels through the kernels package, and exposes ordinary from\_pretrained() loading without local compilation; AWQ W4A16 handles modulation and normalization projections. On an RTX 5090, the cited NVFP4 checkpoint generates 1024×1024 images in about 1.7 seconds at roughly 12 GB peak memory versus about 24 GB for BF16, while the bare-bones Nunchaku Lite path delivers around 30% speedup; torch.compile raises reported end-to-end speedup from 1.35x to 1.8x.


### [Devin Outposts on Modal](https://yomu.fyi/post/devin-outposts-on-modal.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: Jul 21, 2026
- Type: Announcement

Devin, an AI software engineer by Cognition, plans, writes, tests, and ships code semi-autonomously. Cognition now extends Devin with Outposts, moving execution into user-controlled environments while reasoning stays in Cognition’s cloud. Modal is a launch partner with modal-devin, an open-source integration running Outpost sessions in Modal Sandboxes with pre-built custom images, fast cold-starts, snapshotting, and cost-efficient bursting. For example, a task that requires a GPU fails immediately in the default sandbox but succeeds on a Modal-backed Outpost that spins up a GPU on demand and scales to zero when done. One command, uvx modal-devin init, sets up secure infrastructure with a scheduler and worker functions, and each team can have its own outpost with separate credentials.


### [Grabette: an open system to record robot-manipulation data](https://yomu.fyi/post/grabette-an-open-system-to-record-robot-manipulation-data.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Steve Nguyen, Claire Houziel, Gaelle Lannuzel, Simon Le Goff, Jeremy Laville, Étienne
- Published: Jul 21, 2026
- Type: Announcement

Grabette is an open, low-cost system for recording robot-manipulation data without requiring a robot, lab, or teleoperation rig. It uses a handheld gripper with a wide fisheye observation camera, an RGBD tracking camera, an IMU, and gripper encoders to capture synchronized demonstrations and recover camera-local 6-DoF Cartesian pose plus gripper state. Users save episodes on a Raspberry Pi, then select them in a browser dashboard for upload, RTAB-MAP SLAM, trajectory verification, and conversion into a LeRobot dataset on the Hugging Face Hub. The release also includes a motorized Gripette gripper and an example that trains a Diffusion Policy on 200 demonstrations and evaluates it on an OpenArm 7-DoF arm via gRPC, demonstrating the recording-to-training workflow.


### [Integrate NVIDIA Omniverse RTX Sensor Simulation Into Existing Apps](https://yomu.fyi/post/integrate-nvidia-omniverse-rtx-sensor-simulation-into-existing-apps.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 20, 2026
- Type: Explainer

NVIDIA Omniverse libraries, included in NVIDIA Agent Toolkit, provide modular components for incorporating physical AI features into current tools and services. As part of this stack, the ovrtx prerelease library offers a lightweight C and Python SDK for real-time RTX sensor simulation and visualization using OpenUSD scenes. Host applications retain architectural and workflow control by stepping the renderer directly from their internal loops. Integration relies on ovstage for shared OpenUSD data management alongside RenderProducts and RenderVars to configure camera, lidar, radar, and semantic segmentation outputs. PTC Onshape Render Studio is exploring ovrtx to embed RTX rendering directly into cloud-native CAD and PDM workflows.


### [How Unity Catalog managed tables bring interoperability, performance, and unified governance to the Lakehouse](https://yomu.fyi/post/how-unity-catalog-managed-tables-bring-interoperability-performance-an.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Alex Jiang, Tathagata “TD” Das
- Published: Jul 16, 2026
- Type: Announcement

Unity Catalog external access to Unity Catalog managed Delta tables is now in Public Preview, allowing external engines to create, read, and write while governance remains centralized. The announcement addresses the previous multi-engine trade-off: external tables enabled access but lacked managed-table performance optimizations and governance guarantees. Catalog commits coordinate writes through Unity Catalog, making it the source of truth for table state and enabling safe external writes, multi-statement transactions, and auditing of external operations. Predictive Optimization cleans storage, collects query statistics, and selects Liquid clustering columns as query patterns change; the post says these capabilities can deliver up to 50% storage cost savings and 20x faster queries. Support includes Spark, Flink, Starburst, DuckDB, and StreamNative, with open APIs and Delta Kernel extending integrations across Databricks UC and UC OSS.


### [Newer Models, Same Advantage](https://yomu.fyi/post/newer-models-same-advantage.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Erick Lachmann, Gabriel Pimenta de Freitas Cardoso, Francisco de Almeida Rocha Alves, Victor Gabriel Ferreira Barbosa
- Published: Jul 16, 2026
- Type: Benchmark

DharmaOCR, a Brazilian Portuguese-focused OCR model, is evaluated against newer Mistral OCR4 and Unlimited-OCR systems on a Portuguese-only benchmark. The article attributes its performance to a two-stage training pipeline: supervised fine-tuning on varied Portuguese documents, followed by Direct Preference Optimization (DPO) using comparative output preferences. Fine-tuning concentrates model capacity on Brazilian Portuguese vocabulary, syntax, and document structures, while DPO targets repetitive or incoherent generation, reducing inference time and cost and improving stability according to the article. DharmaOCR scored 0.925, compared with 0.798 for Mistral OCR4 and 0.7587 for Unlimited-OCR, while examples from handwritten ENEM essays show the competitors misreading Brazilian names and phrases that DharmaOCR handled correctly. The conclusion is that newer architectures can raise absolute performance, but directing finite resources at one domain can preserve a measurable specialist advantage.


### [July 2026 newsletter](https://yomu.fyi/post/july-2026-newsletter.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jul 16, 2026
- Type: Announcement

The July 2026 newsletter rounds up ClickHouse releases, community projects, benchmarks, engineering work, and upcoming events in real-time data warehousing. Version 26.6 adds hypothetical skip indexes, cascading refreshable materialized-view dependencies, experimental continuous queries, AI embedding functions, PNG output, CLI documentation, and geospatial improvements. A CostBench comparison running the full analytics path over 28 hours found better cost-performance for ClickHouse Cloud than Snowflake Interactive Tables, which required a continuously running refresh warehouse for pre-aggregated data. Other features cover conversational analytics with ClickHouse Cloud, LibreChat, and MCP; Postgres change-data capture through Debezium and Kafka; local agent queries with chDB; high-concurrency tuning; and Silk, whose fiber runtime targets lower I/O tail latency. Quick reads report ingestion, caching, security, and multi-tenant SQL developments.


### [Migrating from Apollo Tooling to GraphQL Codegen at Yelp](https://yomu.fyi/post/migrating-from-apollo-tooling-to-graphql-codegen-at-yelp.md)
- Company: [Yelp](https://yomu.fyi/company/yelp.md)
- Author: Igor Kusakov, Software Engineer
- Published: Jul 16, 2026
- Type: Problem & solution

Yelp migrated its frontend React monorepo from the deprecated apollo-tooling to GraphQL Codegen to ensure ongoing maintenance and build reliability. Because the monorepo spans over 500 packages with thousands of generated type files and custom nested type imports, the Client Data team prioritized a transparent migration that avoided codebase-wide changes. The team rejected approaches requiring syntax updates or altered import paths, choosing instead to configure GraphQL Codegen and contribute upstream plugin changes so output structures and naming conventions matched the legacy output. Migrating packages incrementally through small pull requests enabled continuous feedback while allowing developers across dozens of teams to maintain their workflows without editing existing code. Completing the migration to GraphQL Codegen v6 simplified Yelp's build tooling, eliminated deprecated dependencies, and left upstream compatibility enhancements available for the wider open-source community.


### [Build a Multi-Camera 3D Tracking Application with NVIDIA DeepStream 9.1 Skills](https://yomu.fyi/post/build-a-multi-camera-3d-tracking-application-with-nvidia-deepstream-9.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 15, 2026
- Type: Announcement

NVIDIA DeepStream 9.1 introduces AutoMagicCalib and Multi-View 3D Tracking skills to resolve the limitations of single-camera 2D tracking and manual camera calibration in large-scale video analytics. Multi-View 3D Tracking projects detections into a shared 3D coordinate system, matching tracklets across cameras via MQTT and maintaining globally consistent object IDs. AutoMagicCalib automates camera intrinsic and extrinsic parameter estimation by analyzing object trajectories across video feeds, optionally using a Visual Geometry Grounded Transformer. These capabilities are integrated into 13 agentic skills that allow coding agents to set up, configure, and execute end-to-end vision AI pipelines from natural language prompts. Outputs include an on-screen display grid, bird's-eye view trajectories, and per-frame protobuf metadata streamed through Kafka.


### [Develop Lightweight USD Runtimes Faster with AI Agents](https://yomu.fyi/post/develop-lightweight-usd-runtimes-faster-with-ai-agents.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jul 15, 2026
- Type: Explainer

Building custom Universal Scene Description (USD) implementations traditionally required adapting large existing codebases to meet specific memory footprints, application binary interfaces (ABIs), or performance requirements. The nanousd-labs project introduces an alternative approach by using AI agents to generate lightweight USD runtimes directly from the Alliance for OpenUSD machine-readable USD Core Specification. Agents parse the specification section by section, write conforming code, and validate output against specification-derived tests under engineer guidance. Written in C++ with a public C API, nanousd functions as an independent data layer that handles parsing, composition, queries, and writing without rendering pixels. This methodology enables developers to regenerate runtimes for varying deployment constraints while maintaining standard compliance through reusable workflows and skill graphs.


### [Welcome Inkling by Thinking Machines](https://yomu.fyi/post/welcome-inkling-by-thinking-machines.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: ben burtenshaw, merve, Pedro Cuenca, Aritra Roy Gosthipaty, Andres Marafioti
- Published: Jul 15, 2026
- Type: Announcement

Thinking Machines Lab has released Inkling and the smaller Inkling-Small family as open multimodal models that accept image, text, and audio inputs. Inkling is described as a roughly 1T-parameter decoder-only Mixture-of-Experts model, with 975B total and 41B active parameters, a 1M-token context window, and training on 45 trillion multimodal tokens. Its architecture combines relative attention, alternating global and sliding-window attention, short convolution, shared experts, hierarchical image patchification, and discretized mel-spectrogram audio processing. The release includes BF16 and NVFP4 checkpoints, speculative MTP layers, day-zero support in Transformers, SGLang, vLLM, and llama.cpp, plus deployment configurations and interactive voice and image demos. Reported benchmarks show Inkling and Inkling-Small achieving strong but varied results across reasoning, coding, vision, audio, factuality, chat, and safety evaluations.


### [@clickhouse/rowbinary: when your library is also a parser compiler](https://yomu.fyi/post/clickhouse-rowbinary-when-your-library-is-also-a-parser-compiler.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jul 14, 2026
- Type: Explainer

ClickHouse's @clickhouse/rowbinary is a Node.js reader and writer for the RowBinary, RowBinaryWithNames, and RowBinaryWithNamesAndTypes formats, offering both a generic parser and an accompanying Agent Skill. RowBinary avoids per-row overhead but is difficult to consume in JavaScript because nested types, LEB128 varints, precision-sensitive DateTime64 values, and recursive self-describing types require extensive runtime dispatch; JSON alternatives can also round large UInt64 values. The package provides small, composable, monomorphizable reading primitives, while SKILL.md instructs a coding agent to assemble query-specific parsers with inlined operations and selected policies such as BigInt handling and array materialization. Generated parsers run 1.5–3.4x faster than composing library functions in loops, with a fixed-width benchmark parser reaching 3.41x faster, and produce reviewable source that can be tested and benchmarked like ordinary application code.


### [How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo](https://yomu.fyi/post/how-to-run-an-autoresearch-workflow-with-rl-agent-skills-and-nvidia-ne.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 14, 2026
- Type: Tutorial

Setting up reinforcement learning workflows and running repetitive experimentation often consumes significant engineering time before meaningful metrics emerge. To address this, an autonomous autoresearch workflow uses a frontier coding agent running Codex with GPT 5.5 to manage NVIDIA NeMo RL and NeMo Gym on an NVIDIA Brev GPU instance. The agent executes full-stack setup, creates a novel NeMo Gym visual counting environment, tunes Qwen3-VL-2B-Instruct accuracy from 25.0% to 96.9%, and implements an off-policy reinforcement learning algorithm directly from a research paper. Reusable skills, including brev-etiquette, session-memory, and autoresearch, enforce local conventions and prevent context drift during long executions. This setup offloads environment configuration, dependency resolution, and experiment tracking while preserving human oversight for goals and strategy.


### [Extreme Event Likelihoods with Guided Generative Models](https://yomu.fyi/post/extreme-event-likelihoods-with-guided-generative-models.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 13, 2026
- Type: Explainer

Estimating the likelihood of extreme, high-impact events using brute-force Monte Carlo simulations requires excessive model iterations for expensive physics-based climate models. While guided diffusion models steer generation toward rare events like tropical cyclones, oversampling distorts the true probability distribution. To address this, researchers guide the diffusion-based climate emulator NVIDIA cBottle toward specific events and compute log-odds ratios between guided and unguided distributions. Reweighting the guided samples via these odds ratios enables importance sampling under the original climate distribution. Implemented in NVIDIA Earth2Studio, this technique reduced standard error by 25% compared to simple Monte Carlo sampling when evaluating tropical cyclone states.


### [How to Evaluate General-Purpose Robot Policies for Real-World Deployment](https://yomu.fyi/post/how-to-evaluate-general-purpose-robot-policies-for-real-world-deployme.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Brad Nemire
- Published: Jul 12, 2026
- Type: Problem & solution

Evaluating robotics foundation models accurately in simulation remains challenging due to visual domain overlap, benchmark saturation, diagnostic gaps, and small sample sizes. RoboLab addresses these deficiencies by providing a robot-agnostic simulation platform that supports rapid task creation and diagnostic evaluation suites. Tasks in RoboLab can be compiled across varying robot embodiments and target visual, procedural, and relational competencies. Statistical analysis using the Clopper-Pearson method reveals that narrowing the confidence interval around a 90 percent success rate from ten to two percentage points demands increasing rollouts from 70 to 1,030. Developed by NVIDIA Research, RoboLab powers the open-source Isaac Lab-Arena simulation framework to establish scalable evaluation for real-world robotic policies.


[Newer posts](https://yomu.fyi/topic/open-source/page/6.md) · [Older posts](https://yomu.fyi/topic/open-source/page/8.md)