---
title: "Machine Learning"
description: "622 posts about Machine Learning, summarised, each linking to the original."
---

# Machine Learning
> 622 posts about Machine Learning, summarised, each linking to the original.

## Articles

### [Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents](https://yomu.fyi/post/introducing-nvidia-nemotron-3-nano-omni-long-context-multimodal-intell.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tuomas Rintamaki, Amala Sanjay Deshmukh, Nabin Mulepati, Collin McCarthy, Pritam Biswas, Arushi Goel, Alexandre Milesi, Danial Mohseni Taheri, Kateryna Chumachenko, Isabel Hulseman, Zhehuai Chen, Karan, Tao, Leili Tavabi, Ehsan Hosseini Asl
- Published: Apr 28, 2026
- Type: Announcement

NVIDIA Nemotron 3 Nano Omni is introduced as an omni-modal understanding model for document analysis, multiple-image reasoning, automatic speech recognition, long audio-video understanding, agentic computer use, and general reasoning. It extends the Nemotron multimodal line from vision-language processing to text, image, video, and audio, targeting long, dense inputs such as 100-plus-page documents and mixed audio-visual evidence. Its unified encoder-projector-decoder combines a Nemotron 3 Nano 30B-A3B hybrid Mamba-Transformer MoE backbone with C-RADIOv4-H and Parakeet-TDT-0.6B-v2 encoders; dynamic visual resolution, Conv3D tubelets, and Efficient Video Sampling support multimodal context compression. The post reports leading or top results across document, video, audio-video, voice, and ASR benchmarks, alongside up to 9x higher throughput and 2.9x single-stream reasoning speed on multimodal use cases, and provides BF16, FP8, and NVFP4 checkpoints.


### [Connecting Agents to Decisions](https://yomu.fyi/post/connecting-agents-to-decisions.md)
- Company: [Palantir](https://yomu.fyi/company/palantir.md)
- Author: Palantir
- Published: Apr 28, 2026
- Type: Explainer

Palantir’s Ontology is presented as a decision-centric architecture for human-agent operations, intended to represent how enterprises make and execute decisions rather than only storing or analyzing data. It combines data, logic, action, and security in a shared, dynamic representation that can connect operational data, computational processes, workflow execution, and policy controls. The data layer includes structured, streaming, edge, unstructured, imagery, and decision data, with decision lineage recording the data version, application, and decision context; logic binding connects heterogeneous business, modeling, planning, and optimization assets for human and agent workflows. The architecture exposes approved actions to agents with configurable human review and guardrails, while capturing decisions as learning material for improving workflows and agentic memory. It concludes that a real-time ontology can anchor AI in enterprise conditions while controlling where automation is allowed.


### [From Clicks to Conversions: Architecting Shopping Conversion Candidate Generation at Pinterest](https://yomu.fyi/post/from-clicks-to-conversions-architecting-shopping-conversion-candidate.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: Apr 27, 2026
- Type: Problem & solution

Pinterest built a dedicated shopping candidate generation model to optimize for lower-funnel purchase conversions rather than relying strictly on engagement retrieval models. Because offsite conversion events are sparse and delayed, engineers trained a single multi-surface two-tower model using conversion data supplemented by duration-weighted click engagement and unengaged ad impressions as hard negatives. The two-tower retrieval architecture incorporates a parallel DCN v2 and MLP layer structure that allows simultaneous learning from the same inputs without information bottlenecks. Later revisions transitioned the system from a multi-head loss structure to a unified single-head multi-task architecture paired with an advertiser-level objective. Deployed across more than 600 million monthly active users, the system increased shopping conversion volume by 2.3% and boosted return on ad spend by 3.1% for United States shopping campaigns.


### [How to Use Transformers.js in a Chrome Extension](https://yomu.fyi/post/how-to-use-transformers-js-in-a-chrome-extension.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nico Martin
- Published: Apr 23, 2026
- Type: Tutorial

The guide explains how to build a Chrome extension that runs local AI with Transformers.js under Manifest V3, using a browser assistant as the reference architecture. It assigns model orchestration, lifecycle management, tool execution, and conversation state to a background service worker, while a side panel handles chat and a content script handles page extraction and highlighting. The implementation uses Gemma 4 E2B in q4f16 for text generation and all-MiniLM-L6-v2 in fp32 for embeddings, with WebGPU, DynamicCache-based KV caching, explicit download progress, and reusable pipelines. Typed message contracts, model-output parsing, and separate internal and UI transcripts support a deterministic tool loop, while chrome.storage.local, IndexedDB, and background caches place data according to lifecycle and access needs; the guide concludes that separating these runtimes keeps the UI responsive and avoids duplicate model loads.


### [Flow generation through natural language: An agentic modeling approach](https://yomu.fyi/post/flow-generation-through-natural-language-an-agentic-modeling-approach.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 22, 2026
- Type: Explainer

Shopify built a tool-calling agent for Sidekick that converts natural-language requests into Shopify Flow workflows, addressing the cold-start problem by working backward from thousands of anonymized production workflows. The team generated synthetic queries and tool trajectories, fine-tuned Qwen3-32B, and changed Flow’s nested JSON DSL into an equivalent Python representation connected by a round-trip transpiler. Using Python improved syntactic correctness by 22 points and semantic correctness by 13 points on a 300-example benchmark, while matching production tool names, ordering, responses, and prompts proved essential to avoid degradation. Production evaluation exposed a 35% gap masked by benchmark parity, so Shopify built a weekly feedback loop that scores conversations, filters training data, identifies slice gaps, retrains, and deploys. The resulting agent serves most production traffic, runs 2.2 times faster at 68% lower cost, and outperforms closed models.


### [Machine Learning Solutions: A Complete Implementation Guide](https://yomu.fyi/post/machine-learning-solutions-a-complete-implementation-guide.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Apr 21, 2026
- Type: Tutorial

The guide presents machine learning delivery as a lifecycle problem, arguing that failures arise more often from planning, scoping, communication, and expectations than from algorithmic limitations. It recommends assessing data readiness before development by inventorying sources, checking quality and coverage, planning labeling, and engineering features. Once a measurable business outcome is defined, teams should match the learning paradigm and model complexity to the problem, establish cross-validation and holdout or temporal validation, and estimate compute and storage early. Baselines, reproducible experiments, deployment monitoring, model versioning, CI/CD, and retraining workflows support the transition from prototype to production. The guide also calls for representative proof of concepts and A/B testing or control groups to attribute business impact before scaling.


### [A Practical Guide to LLM Fine Tuning](https://yomu.fyi/post/a-practical-guide-to-llm-fine-tuning.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Apr 21, 2026
- Type: Tutorial

The guide presents LLM fine tuning as a way to adapt a pre-trained model to specific tasks, domains, or applications without full pretraining. It explains when prompting or few-shot learning is sufficient and when fine tuning is justified by quality gaps, domain terminology, latency, cost, or behavioral control. The recommended lifecycle covers scoping, data collection and cleaning, base-model and method selection, training, evaluation, deployment, and monitoring, with production-format consistency and held-out splits treated as important safeguards. It compares supervised and full fine tuning with parameter-efficient fine tuning (PEFT), including LoRA and QLoRA, which update fewer parameters and reduce compute and memory requirements. The main risks are noisy data, overfitting, and catastrophic forgetting; the proposed default is a pilot that compares a PEFT-adapted model with its base model, while combining fine tuning with RAG and prompt engineering when appropriate.


### [QIMMA قِمّة ⛰: A Quality-First Arabic LLM Leaderboard](https://yomu.fyi/post/qimma-a-quality-first-arabic-llm-leaderboard.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Leen AlQadi, Ahmed Alzubaidi, Mohammed Alyafeai, Maitha Alhammadi, Shaikha Alsuwaidi, Omar saif alkaabi, Basma Boussaha, Hakim Hacid
- Published: Apr 21, 2026
- Type: Benchmark

QIMMA is an Arabic LLM leaderboard built to address fragmented evaluation, translation artifacts, weak benchmark quality checks, limited coverage, and reproducibility gaps. It treats benchmark validation as a prerequisite for reported scores rather than an afterthought. The suite consolidates 109 subsets from 14 source benchmarks into more than 52,000 samples across seven domains, with 99% native Arabic content, Arabic-adapted HumanEval+ and MBPP+ coding tasks, and six prompt templates. Before model evaluation, Qwen3-235B-A22B-Instruct and DeepSeek-V3-671B assess samples using a 10-point rubric, flagged cases receive native-speaker review, and code statements are refined without changing task identifiers, reference solutions, or tests. In the April 2026 rankings, Qwen/Qwen3.5-397B-A17B-FP8 ranks first, while Arabic-specialized models lead cultural and linguistic tasks and multilingual models lead coding.


### [What is Generative AI in Marketing?](https://yomu.fyi/post/what-is-generative-ai-in-marketing.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Apr 16, 2026
- Type: Explainer

Generative AI in marketing creates content, insights and recommendations that can personalize experiences, optimize campaigns and improve performance. Unlike traditional analytics, which mainly reports on past results, it produces net-new outputs such as ad copy, audience segments, product recommendations, visual assets and strategic summaries. A typical workflow prepares campaign, customer and brand data, grounds or fine-tunes models, generates outputs, applies targeting and optimization, and uses human review and refinement. It distinguishes pretrained tools such as ChatGPT, Claude and Perplexity from customized models and broader AI transformation, noting trade-offs between speed, investment, relevance and strategic alignment. Successful adoption depends on high-quality, consented data, governance, privacy controls, human oversight, monitoring and cross-functional ownership; the conclusion presents AI as a force multiplier rather than a replacement for human creativity and judgment.


### [Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents](https://yomu.fyi/post/ecom-rlve-adaptive-verifiable-environments-for-e-commerce-conversation.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rahul Bajaj, Jaya Nupur, Anuj Garg, ben burtenshaw
- Published: Apr 16, 2026
- Type: Explainer

Ecom-RLVE extends the RLVE framework from single-turn reasoning puzzles to multi-turn, tool-augmented e-commerce conversations. EcomRLVE-GYM supplies eight environments spanning product discovery, substitution, cart building, returns, order tracking, policy QA, bundle planning, and multi-intent journeys, with procedural problem generation, a 12-axis difficulty curriculum, and algorithmically verifiable rewards. Each episode requires tool use and world-state changes; rewards combine task completion, efficiency, and a penalty for recommending product IDs not retrieved, while malformed outputs and illegal calls fail immediately. Adaptive scheduling advances each environment independently when its success rate is reliable. An initial study trained Qwen 3 8B with DAPO for 300 steps on cart building and reported progressively increasing difficulty reached, consistent with a steady learning signal.


### [The PR you would have opened yourself](https://yomu.fyi/post/the-pr-you-would-have-opened-yourself.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pedro Cuenca, Awni Hannun
- Published: Apr 16, 2026
- Type: Announcement

mlx-lm contributors can use a Skill and a separate test harness to port language models from transformers, with the goal of making new architectures available shortly after their upstream implementations land. The work addresses rising agent-generated pull-request volume and the difficulty of preserving implicit codebase conventions, user contracts, and performance-sensitive behavior that agents may not understand. Given a conversion prompt, the Skill sets up environments, finds model variants and checkpoints, reads transformers modeling code, writes an MLX implementation, and iterates through tests including dtype checks, generation examples, numerical comparisons, and per-layer comparisons. The resulting pull requests disclose agent assistance and include detailed verification data, while contributors remain responsible for reviewing and accepting the results. Current limitations include shared-utility refactors, vision-language models, quantized uploads, and thinking-specific tests.


### [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://yomu.fyi/post/training-and-finetuning-multimodal-embedding-reranker-models-with-sent.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen
- Published: Apr 16, 2026
- Type: Tutorial

Sentence Transformers provides a Python library for training and using embedding and reranker models across text, images, audio, and video in retrieval and semantic search applications. The post shows how to finetune multimodal models on task-specific data, using Visual Document Retrieval as the example: matching text queries with document page images while preserving charts, tables, and layout. It describes the SentenceTransformerTrainer pipeline, model and processor configuration, dataset handling, and alternative Router-based encoders, while also covering multimodal reranker training with BinaryCrossEntropyLoss and direction-specific datasets. On its evaluation data, finetuning Qwen/Qwen3-VL-Embedding-2B raised NDCG@10 from 0.888 to 0.947 and outperformed all tested VDR models, including models up to 4x its size.


### [Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents](https://yomu.fyi/post/inside-vakra-reasoning-tool-use-and-failure-modes-of-agents.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Ankita Naik, danish, Ben, Anupama Murthi, Praveen Venkateswaran, Siyu, Ayhan Sebin
- Published: Apr 15, 2026
- Type: Benchmark

VAKRA is an executable, tool-grounded benchmark for evaluating whether AI agents can complete enterprise-like workflows that combine API calls, document retrieval, and natural-language constraints. It provides more than 8,000 locally hosted APIs backed by real databases across 62 domains, plus domain-aligned documents, and uses full execution traces to assess multi-step behavior. The four capability groups cover API chaining, dashboard-API tool selection, multi-hop dashboard reasoning, and multi-hop multi-source reasoning with multi-turn dialogue and optional tool-use policies; tasks can require three to seven reasoning steps, while some segments specify one to twelve tool calls or up to five logical hops. Reported analyses find that models perform best on single-hop questions, degrade as API hops increase, and often violate policies or fail to retrieve enough information when constraints restrict the relevant source, exposing a gap between isolated tool competence and end-to-end reliability.


### [New Research Reveals Overcoming Legacy Tech Issues Key to AI Success](https://yomu.fyi/post/new-research-reveals-overcoming-legacy-tech-issues-key-to-ai-success.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Dr. William Lee
- Published: Apr 14, 2026
- Type: Benchmark

An IDC survey of 1,400 organizations across eight Asia/Pacific markets reveals that legacy architectures and accumulated data debt severely hinder enterprise artificial intelligence initiatives. While 43% of surveyed organizations cite legacy systems as a major obstacle to application development, 95% report project delays and 90% have experienced failed modernization attempts. In contrast, high-performing organizations treat modernization as a continuous discipline across people, processes, and cloud-centric data platforms capable of processing both structured and unstructured workloads. These leading enterprises achieve nearly three times more digital revenue growth than peers still constrained by technical debt. IDC notes that addressing technical and data debt proactively avoids escalating AI failure rates and delivers superior return on investment compared to superficial lift-and-shift migrations.


### [Waypoint-1.5: Higher-Fidelity Interactive Worlds for Everyday GPUs](https://yomu.fyi/post/waypoint-1-5-higher-fidelity-interactive-worlds-for-everyday-gpus.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Andrew Lapp, Louis Castricato, Scott Fox, Shahbuland Matiana, David Rossi
- Published: Apr 9, 2026
- Type: Announcement

Overworld’s Waypoint-1.5 is a real-time video world model aimed at making interactive generative worlds usable on consumer hardware rather than limited to datacenter-scale compute. It offers two tiers: a 720p model for higher-performance desktop GPUs, capable of up to 60 FPS on RTX 3090 through 5090 hardware, and a 360p model for broader deployment, including gaming laptops and soon Apple Silicon Macs. Training used nearly 100 times more data than Waypoint-1, while more efficient video-modeling techniques reduce redundant computation across frames to support coherent motion and responsive exploration. Users can run it locally through the Biome desktop client, try it in a browser via Overworld Stream, or use the World Engine inference library and third-party clients. The release presents improved visual fidelity, broader local hardware support, and continued progress toward real-time interactive world simulation.


### [Multimodal Embedding & Reranker Models with Sentence Transformers](https://yomu.fyi/post/multimodal-embedding-reranker-models-with-sentence-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen
- Published: Apr 9, 2026
- Type: Tutorial

Sentence Transformers v5.4 adds multimodal embedding and reranking support for text, images, audio, and video through its Python API. Embedding models map modalities into a shared space, while multimodal Cross Encoder rerankers score relevance for mixed-modality pairs, enabling cross-modal similarity, retrieval, and multimodal RAG workflows. The guide shows model loading, URL-based image encoding, encode\_query() and encode\_document() prompt handling, similarity calculation, and ranking, with a Qwen3-VL-Embedding-2B example producing 2,048-dimensional image vectors and matching text queries to images. It notes that cross-modal scores can be lower because of the modality gap, while relative ordering can still support retrieval, and that rerankers generally offer higher quality at the cost of slower per-pair processing; installation extras and GPU memory requirements vary by modality and model.


### [Safetensors is Joining the PyTorch Foundation](https://yomu.fyi/post/safetensors-is-joining-the-pytorch-foundation.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Luc Georges, Lysandre
- Published: Apr 8, 2026
- Type: Announcement

Safetensors has joined the PyTorch Foundation as a foundation-hosted project under the Linux Foundation, alongside DeepSpeed, Helion, Ray, vLLM, and PyTorch. It began as a Hugging Face project addressing the risk of arbitrary code execution in pickle-based model-weight formats. Its format stores tensor metadata in a JSON header capped at 100MB, followed by raw tensor data, and supports zero-copy mapping and lazy loading. The move places the trademark, repository, and governance with the Linux Foundation, while Luc and Daniel remain on the Technical Steering Committee and lead day-to-day work. Existing users get the same format, APIs, and Hub integration without breaking changes, while contributors gain a documented maintainer path; planned work includes PyTorch core serialization, accelerator-aware loading, parallel loading, and quantization support.


### [MongoDB Predictive Auto-Scaling: An Experiment](https://yomu.fyi/post/mongodb-predictive-auto-scaling-an-experiment.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: A. Jesse Jiryu Davis, Matthieu Humeau
- Published: Apr 7, 2026
- Type: Problem & solution

MongoDB Atlas initially relied on a reactive auto-scaler that took minutes to adjust instance sizes and only transitioned between adjacent tiers. Because these delays caused transient server over-utilization or unnecessary customer costs, researchers prototyped predictive auto-scaling to forecast demand. The experimental architecture used short-term and long-term forecasters for customer-driven metrics, an estimator using boosted decision trees to project CPU utilization, and a planner to pick the most cost-effective tier. In testing, the prototype kept replica sets closer to target CPU ranges while saving an average of nine cents per replica set hourly. MongoDB subsequently implemented a production version that scales replica sets up proactively before expected spikes while retaining reactive downscaling.


### [Welcome Gemma 4: Frontier multimodal intelligence on device](https://yomu.fyi/post/welcome-gemma-4-frontier-multimodal-intelligence-on-device.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: merve, Pedro Cuenca, Sergio Paniego, ben burtenshaw, Steven Zheng, Alvaro Bartolome, Nathan Habib
- Published: Apr 2, 2026
- Type: Announcement

Google DeepMind’s Gemma 4 family is available on Hugging Face as Apache 2-licensed, multimodal models intended for use across agents, inference engines, fine-tuning libraries, and on-device deployments. The family accepts image and text inputs across five sizes, while E2B, E4B, and 12B Unified also support audio; context windows range from 128k to 256k tokens. Its design combines alternating local and global attention, dual RoPE configurations, Per-Layer Embeddings, shared KV caches, and model-specific vision and audio handling, including an encoder-free unified 12B variant. The post documents integrations for Transformers, llama.cpp, MLX, WebGPU, Rust, and other tools, alongside fine-tuning examples and demos. Reported instruction-tuned results include a 1452 estimated text-only LMArena score for 31B and benchmark results spanning reasoning, coding, vision, audio, and long-context tasks.


### [Falcon Perception](https://yomu.fyi/post/falcon-perception.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: FalconPerception
- Published: Apr 1, 2026
- Type: Announcement

Falcon Perception is a 0.6B-parameter early-fusion Transformer for open-vocabulary grounding and segmentation from natural-language prompts. It processes image patches, text, and task tokens in one sequence, using hybrid attention so image tokens receive bidirectional context while text and task tokens decode causally. Its Chain-of-Perception interface predicts each instance as coordinate, size, and segmentation tokens, with specialized Fourier-feature and dot-product heads for dense outputs. The post also introduces PBench, which profiles attributes, OCR-guided identification, spatial understanding, relations, and crowded scenes. Falcon Perception reaches 68.0 Macro-F1 on SA-Co versus 62.3 for SAM 3, while presence calibration remains weaker at MCC 0.64 versus 0.82; the 0.3B Falcon OCR model scores 80.3 on olmOCR and 88.6 on OmniDocBench.


[Newer posts](https://yomu.fyi/topic/machine-learning/page/15.md) · [Older posts](https://yomu.fyi/topic/machine-learning/page/17.md)