# Latest reads > The engineering internet, summarised so you can actually read it. ## Articles ### [Differential Transformer V2](https://yomu.fyi/post/differential-transformer-v2.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Li Dong - Published: Jan 20, 2026 Differential Transformer V2 introduces an architectural refinement to differential attention that pairs and subtracts query heads within the same grouped-query attention group. The architecture doubles query heads while preserving key-value head counts, which avoids increasing key-value cache demands and removes the need for custom kernels. DIFF V2 replaces the per-head RMSNorm of its predecessor with a per-token, per-head projected lambda passed through a sigmoid function. This formulation expands the context root-mean-square range to between zero and root two, preventing massive gradient magnifications, mitigating attention sinks, and stabilizing large learning rate pretraining. Ablations confirm that sharing keys and values across subtracted heads and constraining lambda scaling are necessary to prevent training instability and elevated loss. ### [Introducing Waypoint-1: Real-time interactive video diffusion from Overworld](https://yomu.fyi/post/introducing-waypoint-1-real-time-interactive-video-diffusion-from-over.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Andrew Lapp, Louis Castricato, Scott Fox, Shahbuland Matiana, David Rossi - Published: Jan 20, 2026 Overworld introduced Waypoint-1, a real-time interactive video diffusion model controllable through text prompts, mouse movements, and keyboard inputs. The architecture uses a frame-causal rectified flow transformer trained on compressed latent representations across 10,000 hours of video game footage paired with control signals. Pre-training uses diffusion forcing with causal attention, while post-training incorporates self forcing via DMD to mitigate autoregressive rollout error accumulation, enable one-pass classifier-free guidance, and support few-step denoising. To achieve low latency on consumer hardware, Overworld developed WorldEngine, a pure Python streaming inference library. Utilizing AdaLN feature caching, static rolling key-value caching with Flex Attention, fused matrix multiplications, and PyTorch compilation, the 2.3B parameter Waypoint-1-Small model reaches up to 60 frames per second on an RTX 5090 GPU. ### [From deployment slop to production reality: How BriX bridges the gap with enterprise-grade AI infrastructure](https://yomu.fyi/post/from-deployment-slop-to-production-reality-how-brix-bridges-the-gap-wi.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Sneh Agrawal - Published: Jan 16, 2026 Internal AI prototypes frequently fail enterprise rollouts due to diverging versions, security oversights, hardcoded credentials, and infrastructure bottlenecks. BriX addresses this deployment gap by turning AI rollout into a configuration-driven platform rather than an engineering rewrite. Built on a synchronous streaming architecture, it routes user prompts through a React frontend using Server-Sent Events, a FastAPI gateway, and LangGraph orchestration. The platform integrates model switching, centralized prompt locks, and standardized Model Context Protocols for governed enterprise data access. ### [MongoDB.local San Francisco 2026: Ship Production AI, Faster](https://yomu.fyi/post/mongodb-local-san-francisco-2026-ship-production-ai-faster.md) - Company: [MongoDB](https://yomu.fyi/company/mongodb.md) - Author: Peter Richards - Published: Jan 15, 2026 MongoDB announced several features aimed at streamlining AI application development across its database and search ecosystem. The Voyage 4 embedding model family is now generally available, introducing cross-model compatibility within a shared embedding space alongside the video-capable voyage-multimodal-3.5 model. For in-database operations, MongoDB Community Edition introduced Automated Embedding in public preview to eliminate external synchronization pipelines for vector generation. Vector Search gained Lexical Prefilters, replacing the knnBeta operator with a dedicated vectorSearch operator and vector data type inside $search aggregations. Additionally, MongoDB made its intelligent assistant generally available in Compass and released the source code for its mongot search engine under the SSPL. ### [Open Responses: What you need to know](https://yomu.fyi/post/open-responses-what-you-need-to-know.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: shaun smith, ben burtenshaw, merve, Pedro Cuenca - Published: Jan 15, 2026 Open Responses is a new open inference standard initiated by OpenAI, developed by the open source AI community, and backed by Hugging Face. While agentic workloads require multi-step planning and tool execution, the ecosystem has remained constrained by the legacy Chat Completion format and proprietary Responses implementations. Open Responses addresses this mismatch by standardizing semantic event streaming, provider-specific configuration options, and distinct roles for model providers and intermediary routers. The specification formalizes reasoning visibility across raw traces, protected encrypted content, and sanitized summaries while supporting provider-managed sub-agent execution loops. Developers can now orchestrate multi-step tool calls and stream reasoning deltas across heterogeneous providers through a single consistent API schema. ### [AI in Investment Management: 2026 Outlook (Part I)](https://yomu.fyi/post/ai-in-investment-management-2026-outlook-part-i.md) - Company: [Two Sigma](https://yomu.fyi/company/two-sigma.md) - Author: Emily Majewski - Published: Jan 12, 2026 Senior leaders and researchers at Two Sigma analyze the expanding role of artificial intelligence across quantitative investment management for 2026. Rapid model improvements are inverting traditional quantitative research workflows by vastly expanding hypothesis generation and shifting operational bottlenecks toward rapid evaluation. Rather than relying on large language models to execute trades independently, firms are integrating agentic workflows as an underlying operating system across data pipelines and portfolio rooms. However, autonomous agents present alignment challenges because they optimize proxy objective functions relentlessly without intrinsic contextual awareness. Sustained success relies heavily on human supervision, rigorous safety monitoring, and disciplined research execution rather than mere compute scale or model complexity. ### [Vision RAG: Enabling Search on Any Documents](https://yomu.fyi/post/vision-rag-enabling-search-on-any-documents.md) - Company: [MongoDB](https://yomu.fyi/company/mongodb.md) - Author: Thibaut Gourdel - Published: Jan 12, 2026 Traditional retrieval-augmented generation relies on text parsing and optical character recognition to extract data from complex visual documents. These legacy techniques introduce high engineering effort, inconsistent accuracy, and heavy scaling costs. Vision RAG resolves these challenges by embedding entire visual documents directly through multimodal models like Voyage AI's voyage-multimodal-3.5, removing the need for text extraction pipelines. In this pipeline, dense vector embeddings map both text queries and visual assets into a unified representation space. At query time, the system retrieves relevant document images and supplies them alongside the text prompt to Anthropic's Claude 3.5 Sonnet to generate visually grounded responses. ### [Lyft’s Feature Store: Architecture, Optimization, and Evolution](https://yomu.fyi/post/lyft-s-feature-store-architecture-optimization-and-evolution.md) - Company: [Lyft](https://yomu.fyi/company/lyft.md) - Author: Rohan Varshney - Published: Jan 6, 2026 Lyft built a centralized Feature Store to optimize feature management, deployment, and data consistency for machine learning workflows across its rideshare platform. The system ingests batch features defined by Spark SQL and JSON metadata into Airflow DAGs, which simultaneously write data to Hive for model training and to an online serving layer for low-latency inference. Real-time streaming features are processed with Apache Flink and Apache Kafka before being ingested into the serving infrastructure. The online layer, dsfeatures, couples DynamoDB with a ValKey write-through LRU cache and OpenSearch to serve embeddings. Infrastructure optimizations reduced standard P95 read latency by a third while supporting over a trillion additional read and write operations. ### [NVIDIA Cosmos Reason 2 Brings Advanced Reasoning To Physical AI](https://yomu.fyi/post/nvidia-cosmos-reason-2-brings-advanced-reasoning-to-physical-ai.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Tsung-Yi Lin, Debraj Sinha - Published: Jan 5, 2026 NVIDIA released Cosmos Reason 2, an open reasoning vision-language model designed to improve multi-step planning, uncertainty handling, and physical AI decision-making. Available in 2B and 8B parameter configurations, the model increases input context support to 256K tokens, up from 16K in the prior version. The release adds spatial perception capabilities including optical character recognition, 2D and 3D point localization, bounding boxes, and trajectory generation for robotics. In domain adaptation tests on autonomous vehicle video captioning, fine-tuning the 8B model improved BLEU scores by 10.6% and LingoQA scores by 13.8%. The model leads the Physical AI Bench and Physical Reasoning leaderboards for open visual understanding models. ### [Introducing Falcon-H1-Arabic: Pushing the Boundaries of Arabic Language AI with Hybrid Architecture](https://yomu.fyi/post/introducing-falcon-h1-arabic-pushing-the-boundaries-of-arabic-language.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Basma Boussaha, Mohammed Alyafeai, Ahmed Alzubaidi, Leen AlQadi, Shaikha Alsuwaidi, Omar saif alkaabi, Hamza Alobeidli, Hakim Hacid - Published: Jan 5, 2026 The Technology Innovation Institute introduced Falcon-H1-Arabic, a family of Arabic language models released in 3B, 7B, and 34B parameter sizes. Built on a hybrid architecture, each block runs Mamba State Space Models and Transformer attention in parallel before fusing their representations at the output projection. Pre-training incorporated around 300 billion tokens across Arabic dialects, English, and multilingual content filtered via linguistic analysis. A post-training pipeline combining supervised fine-tuning and direct preference optimization addresses long-context utilization and alignment. Across evaluations on the Open Arabic LLM Leaderboard, the 34B model achieves roughly 75%, exceeding the performance of larger baseline systems like Llama-3.3-70B. ### [AprielGuard: A Guardrail for Safety and Adversarial Robustness in Modern LLM Systems](https://yomu.fyi/post/aprielguard-a-guardrail-for-safety-and-adversarial-robustness-in-moder.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Jaykumar Kasundra - Published: Dec 23, 2025 Modern language model systems face security and safety threats spanning prompt injection, jailbreaks, and tool manipulation across long-context agentic workflows. To address these vulnerabilities, researchers introduced AprielGuard, an 8B parameter causal decoder-only transformer based on a downscaled Apriel-1.5 Thinker Base model. The model detects 16 safety risk categories alongside binary adversarial attacks across standalone prompts, multi-turn dialogues, and agentic workflows. Trained on synthetic datasets created with Mixtral-8x7B, NVIDIA NeMo Curator, and SyGra, AprielGuard operates in both explainable reasoning and fast non-reasoning modes. Evaluation shows high precision and recall on safety and adversarial benchmarks, though trade-offs exist between explainability latency and non-reasoning performance. ### [Demystifying user journeys: Revolutionizing troubleshooting with auto tracking](https://yomu.fyi/post/demystifying-user-journeys-revolutionizing-troubleshooting-with-auto-t.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Alex Prokofiev - Published: Dec 23, 2025 Tracking user journeys across hundreds of mobile UI components via manual clickstream instrumentation frequently caused data gaps and prolonged issue resolution. Grab designed AutoTrack, an SDK that automatically records application lifecycle states, launch triggers, user interactions, and screen contexts. The system monitors transitions across foreground, background, and inactive states alongside entry points like deep links and push notifications. On the client side, AutoTrack captures complete screen hierarchy metadata from native Android activities, fragments, and iOS view controllers, redacting sensitive data to comply with privacy regulations. This automated trace enables engineering teams to reproduce elusive bugs, route incidents to domain owners, and generate automated UI test cases from real-world usage patterns. ### [That’s a Wrap: MongoDB’s 2025 in Review & 2026 Predictions](https://yomu.fyi/post/that-s-a-wrap-mongodb-s-2025-in-review-2026-predictions.md) - Company: [MongoDB](https://yomu.fyi/company/mongodb.md) - Author: Kevin O'Rourke - Published: Dec 22, 2025 MongoDB recapped its product releases, executive transitions, and enterprise adoption milestones throughout 2025 alongside predictions for 2026. The company acquired embedding and reranking specialist Voyage AI to mitigate large language model hallucinations and launched MongoDB AMP to accelerate legacy application modernization. Vector search and full-text search capabilities were expanded from MongoDB Atlas to Community Edition and Enterprise Server, enabling on-premises and hybrid retrieval-augmented generation workloads. Enterprise implementations included pharmaceutical distributor McKesson scaling tracking infrastructure to handle 1.2 billion containers annually after replacing its monolithic legacy system. Executive predictions for 2026 emphasized human-in-the-loop verification, grounding language models with factual retrieval, and building multi-cloud resilience against outages and data sovereignty constraints. ### [Token-count-based Batching: Faster, Cheaper Embedding Inference for Queries](https://yomu.fyi/post/token-count-based-batching-faster-cheaper-embedding-inference-for-quer.md) - Company: [MongoDB](https://yomu.fyi/company/mongodb.md) - Author: Chengcheng Pei, Yuan Lin - Published: Dec 18, 2025 Serving embedding models for short search queries often suffers from poor GPU efficiency because traffic is spiky and memory-bound. Traditional time-window and request-count batching strategies lead to inconsistent GPU utilization, while tensor padding wastes compute on empty tokens. To resolve these bottlenecks, Voyage AI implemented token-count-based batching paired with padding removal in inference engines like vLLM. The architecture uses Redis with Lua scripts to atomically aggregate pending requests until reaching an optimal hardware saturation token threshold. Across production deployments, this approach achieved a 50% reduction in GPU inference latency with 3X fewer GPUs and improved throughput by up to 8×. ### [How Grab is accelerating growth with real-time personalization using Customer Data Platform scenarios](https://yomu.fyi/post/how-grab-is-accelerating-growth-with-real-time-personalization-using-c.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Saubhagya Awaneesh - Published: Dec 18, 2025 Grab previously relied on daily batch attribute updates in its Customer Data Platform, which created engineering bottlenecks and hindered time-sensitive engagement opportunities. To address this limitation, the team introduced Scenarios, a self-serve real-time personalization capability embedded within the platform. The architecture processes real-time event triggers from Grab's Scribe platform using Apache Flink, enriches incoming events with historical context from StarRocks, and evaluates pre-trained machine learning classifiers. Computed outputs sync to Kafka streams or Amphawa, an internal feature store powered by AWS DynamoDB, maintaining end-to-end latencies under fifteen seconds. Across more than a dozen production deployments, including real-time interventions for subscription abandonment within fifteen minutes, the platform achieved over a 3% conversion uplift compared to batch campaigns. ### [Tokenization in Transformers v5: Simpler, Clearer, and More Modular](https://yomu.fyi/post/tokenization-in-transformers-v5-simpler-clearer-and-more-modular.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Ita Zaporozhets, Aritra Roy Gosthipaty, Arthur Zucker, Sergio Paniego, merve, Pedro Cuenca - Published: Dec 18, 2025 Transformers v5 overhauls its tokenization framework by separating tokenizer architecture from trained vocabularies. In contrast to v4's dual slow Python and fast Rust files, v5 consolidates each model tokenizer into a single file defaulting to the Rust-backed TokenizersBackend. The pipeline stages—normalizer, pre-tokenizer, model algorithm such as BPE or Unigram, post-processor, and decoder—are now directly exposed and configurable rather than buried in serialized files. Practitioners can instantiate blank tokenizer architectures and train custom vocabularies directly from iterators using native methods like train\_new\_from\_iterator while retaining model-specific formatting rules. The wrapper layer continues to bridge raw tokenization and model requirements by managing chat templates, context limits, and special token insertion. ### [The Open Evaluation Standard: Benchmarking NVIDIA Nemotron 3 Nano with NeMo Evaluator](https://yomu.fyi/post/the-open-evaluation-standard-benchmarking-nvidia-nemotron-3-nano-with.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Seph Mard, Isabel Hulseman, Besmira Nushi, Piotr Januszewski, Grzegorz Chlebus, VivienneZhang, Wojciech Prazuch, Pablo Ribalta, Nik Spirin, Ferenc Galko - Published: Dec 17, 2025 Assessing whether large language model improvements stem from genuine advances or underspecified evaluation conditions remains a major challenge across the industry. Most published model evaluations omit critical execution parameters, prompt templates, harness versions, and runtime configurations. In response, NVIDIA released Nemotron 3 Nano 30B A3B alongside its complete, reproducible evaluation recipe built with the open-source NeMo Evaluator library. The library serves as an orchestration layer that unifies diverse benchmark harnesses under standard configurations while decoupling evaluation logic from underlying inference backends. Developers can execute the identical evaluation pipeline against local deployments or hosted endpoints using published YAML configurations and structured logging. ### [From Python3.8 to Python3.10: Our Journey Through a Memory Leak](https://yomu.fyi/post/from-python3-8-to-python3-10-our-journey-through-a-memory-leak.md) - Company: [Lyft](https://yomu.fyi/company/lyft.md) - Author: Jay Patel - Published: Dec 15, 2025 During an initiative to upgrade Python services from version 3.8 to 3.10, Lyft engineers encountered severe latency spikes and timeouts in one service within their test environment. Profiling DynamoDB queries revealed that gevent thread joins were taking up to 30 seconds while pod memory consumption climbed steadily. To isolate the issue, engineers used an internal tracemalloc-based profiler triggered via SIGUSR2 signals, temporarily disabling Gunicorn preload to prevent workers from terminating prematurely on signal receipt. Memory traces pointed to botocore and an incompatibility between weakref.finalize and gevent monkey patching in urllib3 version 1.26.16, which prevented connections from returning to the pool. Downgrading urllib3 to version 1.26.15 immediately resolved both the timeouts and the memory leak before a permanent fix arrived in gevent and urllib3 updates. ### [CUGA on Hugging Face: Democratizing Configurable AI Agents](https://yomu.fyi/post/cuga-on-hugging-face-democratizing-configurable-ai-agents.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Jim Laredo, Avi Yaeli, Sami Marreed, Ayhan Sebin, Merve Unuvar - Published: Dec 15, 2025 Many existing AI agent frameworks suffer from brittleness, tool misuse, and failures when executing complex workflows. To address these limitations, the open-source Configurable Generalist Agent (CUGA) introduces structured orchestration that decomposes user goals into programmatic subtasks tracked by a dynamic task ledger. The framework integrates agentic patterns like planner-executor and code-act, delegating subtasks to specialized agents that generate pseudo-code before running execution in a secure sandbox. Released under the Apache 2.0 license, CUGA integrates with Langflow for low-code visual workflow assembly and supports multi-tool environments through OpenAPI specs, MCP servers, and LangChain. Testing on inference platforms like Groq with open models such as gpt-oss-120b demonstrates rapid response times during multi-step planning and validation. ### [New in llama.cpp: Model Management](https://yomu.fyi/post/new-in-llama-cpp-model-management.md) - Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md) - Author: Xuan-Son Nguyen, Victor Mustar - Published: Dec 11, 2025 llama.cpp server now includes a router mode that enables dynamic loading, unloading, and switching between multiple LLMs without restarting the server. The architecture runs each model in an isolated process to ensure a single model crash does not affect other active instances. Running llama-server without specifying a model activates auto-discovery across the cache directory or a designated folder of GGUF files. In addition to on-demand loading and least-recently-used eviction capped by default at four models, the server provides endpoints for manual loading, unloading, and listing model statuses. Models inherit global configuration options or use dedicated configuration presets while also integrating directly into the built-in web UI. [Newer posts](https://yomu.fyi/page/23.md) · [Older posts](https://yomu.fyi/page/25.md)