---
title: "Machine Learning"
description: "622 posts about Machine Learning, summarised, each linking to the original."
---

# Machine Learning
> 622 posts about Machine Learning, summarised, each linking to the original.

## Articles

### [Introducing RTEB: A New Standard for Retrieval Evaluation](https://yomu.fyi/post/introducing-rteb-a-new-standard-for-retrieval-evaluation.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Frank Liu, Kenneth Enevoldsen, Solomatin Roman, Isaac Chung, Tom Aarsen, Fődi, Zoltán
- Published: Oct 1, 2025
- Type: Announcement

Public embedding benchmarks often fail to measure true generalization because training sets overlap with public evaluation data, rewarding memorization over general search quality. The Retrieval Embedding Benchmark (RTEB) beta introduces a standardized evaluation framework designed for real-world enterprise retrieval applications. To prevent overfitting while preserving transparency, RTEB combines fully public datasets with private evaluation sets managed exclusively by MTEB maintainers. The suite spans 20 languages across domains like healthcare, law, finance, and code, measuring ranked retrieval quality using NDCG@10 on datasets of at least 1,000 documents. While currently limited to text-only retrieval and containing repurposed question-answering datasets, the benchmark provides clear signals when models degrade on unseen data.


### [Accelerating Qwen3-8B Agent on Intel® Core™ Ultra with Depth-Pruned Draft Models](https://yomu.fyi/post/accelerating-qwen3-8b-agent-on-intel-coretm-ultra-with-depth-pruned-dr.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Igor Margulis, Ofir Zafrir, Shira Guskin, Guy Boudoukh, Pedro Cuenca
- Published: Sep 29, 2025
- Type: Problem & solution

Local agentic AI workflows generate multi-step reasoning traces that increase token counts and strain real-time inference on personal computers. To accelerate execution, researchers implemented speculative decoding using OpenVINO.GenAI on an Intel Lunar Lake integrated GPU, pairing a 4-bit Qwen3-8B target model with a smaller Qwen3-0.6B draft model. By measuring angular distance across layers, the team pruned six of the draft model's twenty-eight layers to minimize draft latency and fine-tuned it on synthetic data generated from 500,000 prompts. This depth-pruned draft elevated speculative decoding throughput from a 1.3x speedup to a 1.4x speedup over the standalone baseline. Finally, deploying the optimized model alongside the Hugging Face smolagents framework allowed local agents to perform tool invocation and code generation tasks efficiently.


### [User foundation models for Grab](https://yomu.fyi/post/user-foundation-models-for-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Abhinav Rai
- Published: Sep 26, 2025
- Type: Explainer

Grab describes a foundation model that learns from both tabular data and sequential clickstream interactions to represent users, merchants, and drivers across its superapp. Traditional recommender systems depended on manually engineered, task-specific features, which required substantial effort and did not capture evolving behavior holistically. The custom architecture addresses heterogeneous modalities including text, numerical values, IDs, images, and location data, combining long-term profiles with immediate intent. Grab pre-trains the model on a diverse corpus, then either extracts embeddings for downstream recommendation and prediction tasks or fine-tunes it to predict directly. The resulting reusable representations are reported to support applications such as ad optimisation, dual app prediction, fraud detection, and churn probability, while future work targets unified data, improved architecture, and greater training and inference efficiency.


### [Swift Transformers Reaches 1.0 – and Looks to the Future](https://yomu.fyi/post/swift-transformers-reaches-1-0-and-looks-to-the-future.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pedro Cuenca, Christopher Fleetwood, Mattt, Vaibhav Srivastav
- Published: Sep 26, 2025
- Type: Announcement

swift-transformers has released version 1.0 to provide a stable foundation for running local models on Apple Silicon platforms. The library supplies components missing from Core ML and MLX, focusing heavily on tokenization, chat templates, and Hugging Face Hub integration. In this major release, Tokenizers and Hub become standalone top-level modules, enabling developers to import only the components they require. The update also integrates an updated swift-jinja library for faster template processing, adopts Modern Core ML stateful models and MLTensor APIs, removes unnecessary CLI dependencies, and adds full Swift 6 support. Future development will prioritize deeper integration with MLX frameworks and agentic workflows like MCP.


### [Smol2Operator: Post-Training GUI Agents for Computer Use](https://yomu.fyi/post/smol2operator-post-training-gui-agents-for-computer-use.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amir Mahla, merve, Sergio Paniego, Vaibhav Srivastav, Lewis Tunstall
- Published: Sep 23, 2025
- Type: Problem & solution

Lightweight vision-language models often lack the precise visual grounding and action execution capabilities required for graphical user interface automation. To resolve this deficiency, the authors constructed a post-training pipeline that evolves base models into agentic GUI coders through supervised fine-tuning. The process standardizes disparate mobile and desktop dataset action spaces into a unified functional format utilizing normalized coordinates relative to image size. Training is structured into two sequential phases that first establish basic visual perception and subsequently introduce explicit reasoning steps with masked assistant loss. On the ScreenSpot-v2 perception benchmark, fine-tuning SmolVLM2-2.2B-Instruct increased element localization accuracy to 61.71 percent, while a compact 460-million-parameter model achieved approximately 58 percent accuracy.


### [SyGra: The One-Stop Framework for Building Data for LLMs and SLMs](https://yomu.fyi/post/sygra-the-one-stop-framework-for-building-data-for-llms-and-slms.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Bidyapati Pradhan, Vipul Mittal, Amit Kumar Saha, Surajit Dasgupta
- Published: Sep 22, 2025
- Type: Problem & solution

Training and aligning large language models and small language models requires datasets tailored for complex reasoning, preference tuning, multi-turn questioning, and format conversion. To eliminate the need for bespoke data preparation scripts, SyGra provides a low-code and no-code Python framework for automated dataset creation, transformation, and alignment. The library integrates with diverse inference backends, including vLLM, Hugging Face TGI, Triton, and Ollama, allowing teams to focus on prompt engineering rather than pipeline infrastructure. It supports data workflows ranging from converting PDFs and knowledge bases into question-answering pairs to filtering low-quality samples and generating Direct Preference Optimization datasets. These plug-and-play workflows reduce manual curation effort while accelerating model fine-tuning and retrieval-augmented generation pipelines.


### [MongoDB.local NYC 2025: Definiendo la base de datos ideal para la era de la IA](https://yomu.fyi/post/mongodb-local-nyc-2025-definiendo-la-base-de-datos-ideal-para-la-era-d.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Dev Ittycheria, President and CEO, MongoDB
- Published: Sep 18, 2025
- Type: Announcement

At MongoDB.local NYC 2025, MongoDB introduced new capabilities and product updates tailored for artificial intelligence workloads. MongoDB 8.2 was unveiled alongside public previews of full-text and vector search for both MongoDB Community Edition and Enterprise Server. The company also detailed its integration with Voyage AI embedding and reranking models to deliver precision retrieval across raw data, metadata, and vectors. To address legacy infrastructure migration hurdles, MongoDB launched the Application Modernization Platform, which combines AI-assisted tooling and specialized workflows to accelerate legacy application refactoring two to three times faster. These architectural updates aim to position the JSON document model as the persistent memory and context store required by autonomous, agentic AI workflows.


### [MongoDB.local NYC 2025: Definition der idealen Datenbank für das KI-Zeitalter](https://yomu.fyi/post/mongodb-local-nyc-2025-definition-der-idealen-datenbank-fu-r-das-ki-ze.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Dev Ittycheria, President and CEO, MongoDB
- Published: Sep 18, 2025
- Type: Announcement

MongoDB unveiled several core database updates aimed at supporting artificial intelligence workloads during MongoDB.local NYC 2025. The release of MongoDB 8.2 introduces performance enhancements alongside Voyage AI embedding models and rerankers for AI applications. Furthermore, Search and Vector Search are now available in public preview for both MongoDB Community Edition and MongoDB Enterprise Server. To address the costs and operational friction of legacy systems, MongoDB launched the Application Modernization Platform (AMP), which combines AI-powered tools with specialized talent to accelerate database migrations. These capabilities position the JSON-based document model to supply the persistent state, memory, and retrieval mechanisms required by emerging agentic AI architectures.


### [Democratizing AI Safety with RiskRubric.ai](https://yomu.fyi/post/democratizing-ai-safety-with-riskrubric-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Gal Moyal
- Published: Sep 18, 2025
- Type: Announcement

Cloud Security Alliance and Noma Security introduced RiskRubric.ai to provide standardized, transparent risk assessments across the open AI model ecosystem. The framework evaluates AI models across six pillars—transparency, reliability, security, privacy, safety, and reputation—using over 1,000 reliability tests, 200 adversarial security probes, automated code scanning, and harmful content evaluations. Each model receives 0–100 scores and A–F letter grades, supplemented by specific vulnerability findings and recommended mitigation strategies to assist deployment filtering. Initial benchmark results across models showed composite scores ranging from 47 to 94 with a median of 81, revealing polarized safety distributions and indicating that security hardening directly correlates with reduced safety risks.


### [Public AI on Hugging Face Inference Providers 🔥](https://yomu.fyi/post/public-ai-on-hugging-face-inference-providers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Joseph Low, Joshua Tan, Célina Hanouti, Julien Chaumond, Simon Brandeis, Lucain Pouget
- Published: Sep 17, 2025
- Type: Announcement

Hugging Face has integrated Public AI as a supported Inference Provider on the Hugging Face Hub. Public AI operates as a nonprofit, open-source project providing access to sovereign and public models from institutions such as the Swiss AI Initiative and AI Singapore. Its distributed infrastructure combines a vLLM-powered backend serving OpenAI-compatible APIs across partner-donated clusters with a global load-balancing routing layer. Users can access these models through the Hugging Face web UI, Python client SDK, and JavaScript SDK using either direct provider API keys or routed Hugging Face tokens. At the time of announcement, inference through the Public AI provider is free of charge, supported by donated GPU time and advertising subsidies.


### [\`LeRobotDataset:v3.0\`: Bringing large-scale datasets to \`lerobot\`](https://yomu.fyi/post/lerobotdataset-v3-0-bringing-large-scale-datasets-to-lerobot.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Francesco Capuano, Michel Aractingi, Quentin Lhoest, Caroline Pascal, Pepijn Kooijmans, Jade Choghari, Remi Cadene, Simon  Alibert, Adil Zouitine, Martino Russi, Steven Palma
- Published: Sep 16, 2025
- Type: Announcement

LeRobotDataset:v3.0 is a new release for lerobot, Hugging Face’s robotics library, addressing file-system limitations encountered when LeRobotDataset:v2 stored one episode per file at million-episode scale. Version 3 packs multiple episodes into shared Parquet and MP4 files, while relational JSON and chunked episode metadata identify episode boundaries and locate the corresponding tabular and visual data. The format separates tabular data, visual data, and metadata; it stores low-dimensional readings in Apache Parquet, concatenates camera frames into MP4 videos, and supports native temporal windowing through delta\_timestamps. It also introduces StreamingLeRobotDataset for processing batches from the Hugging Face Hub without downloading the full dataset locally, and provides a one-line conversion utility for datasets in the LeRobotDataset format. The release is planned for lerobot-v0.4.0 and is presented as a step toward datasets with millions of episodes.


### [Visible Watermarking with Gradio](https://yomu.fyi/post/visible-watermarking-with-gradio.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Margaret Mitchell
- Published: Sep 15, 2025
- Type: Announcement

Generative artificial intelligence tools produce images, video, audio, and text that have become increasingly difficult to distinguish from real-world media captures. To improve synthetic media transparency, Hugging Face introduced visible watermarking capabilities directly into the Gradio web application framework. Developers can now overlay visual watermarks on image and video outputs by specifying a single watermark parameter using file paths, open image objects, or NumPy arrays. The implementation also supports QR code watermarks, which can match the visual style of generated media while conveying detailed background information. Furthermore, the Gradio Chatbot component accepts a text watermark parameter that automatically attaches attribution metadata whenever users copy generated responses to their clipboards.


### [Introducing the Palmyra-mini family: Powerful, lightweight, and ready to reason!](https://yomu.fyi/post/introducing-the-palmyra-mini-family-powerful-lightweight-and-ready-to.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rakshith, Tom Peres
- Published: Sep 11, 2025
- Type: Announcement

WRITER released three open models in the Palmyra-mini family ranging between 1.5B and 1.7B parameters, designed for efficient inference and specialized reasoning tasks. Built on the Qwen architecture, the release includes the standard palmyra-mini base model alongside two Chain of Thought variants, palmyra-mini-thinking-a and palmyra-mini-thinking-b. In benchmark evaluations, palmyra-mini achieved 52.6% on Big Bench Hard, while palmyra-mini-thinking-a reached 82.87% on GSM8K and palmyra-mini-thinking-b reached 92.5% on AMC23. For palmyra-mini-thinking-b, applying reinforcement learning fine-tuning to an OpenReasoning-Nemotron-1.5B base improved single-shot pass@1 accuracy while reducing sampling diversity and majority@64 performance. The models are available in GGUF and MLX-BF16 quantizations and support inference engines including vLLM, SGLang, TRTLLM, and TGI.


### [Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers](https://yomu.fyi/post/tricks-from-openai-gpt-oss-you-can-use-with-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Aritra Roy Gosthipaty, Sergio Paniego, Vaibhav Srivastav, Pedro Cuenca, Arthur Zucker, Nathan Habib, Cyril Vallez
- Published: Sep 11, 2025
- Type: Announcement

To support OpenAI's GPT-OSS series of models, the transformers library introduced several performance upgrades that apply across supported architectures. The release integrates zero-build custom kernels downloadable directly from the Hub, reducing external dependency bloat and compilation friction for operations like Liger RMSNorm and MegaBlocks MoE. Native support for MXFP4 quantization groups vector elements into 32-value blocks with shared scales, allowing GPT-OSS 20B to fit in roughly 16 GB of VRAM and GPT-OSS 120B in roughly 80 GB. In addition, transformers incorporates Flash Attention 3 with attention sinks, continuous batching via the generate\_batch API for experimentation, and automatic memory pre-allocation to speed up model loading on GPUs.


### [Fine-tune Any LLM from the Hugging Face Hub with Together AI](https://yomu.fyi/post/fine-tune-any-llm-from-the-hugging-face-hub-with-together-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Zain Hasan, Artem Chumachenko, Egor Timofeev, Max Ryabinin
- Published: Sep 10, 2025
- Type: Announcement

Together AI and Hugging Face have introduced an integration enabling developers to fine-tune compatible Hugging Face Hub models directly on Together AI's managed infrastructure. To launch a fine-tuning job via the Python SDK, users provide a base model from Together's catalog as a configuration template alongside the target Hugging Face repository identifier. This base model template dictates GPU allocation, memory configuration, training pipelines, and inference setup for custom models with matching architectures and sizes. The workflow operates bidirectionally, pulling from public or token-authenticated private repositories and optionally pushing completed checkpoints back to the Hub upon completion. Geared toward CausalLM models under 100 billion parameters, the capability enables faster iteration cycles and domain adaptation without custom DevOps infrastructure.


### [Jupyter Agents: training LLMs to reason with notebooks](https://yomu.fyi/post/jupyter-agents-training-llms-to-reason-with-notebooks.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Baptiste Colle, Hanna Yukhymenko, Leandro von Werra
- Published: Sep 10, 2025
- Type: Problem & solution

Small language models often struggle to compete with large frontier models on complex, agentic data science tasks. To improve notebook-based reasoning, researchers simplified agent scaffolding down to roughly two hundred lines of code with dedicated execution and final answer tools, boosting baseline easy accuracy on the DABStep benchmark from 44.4 percent to 59.7 percent. They constructed a curated training dataset by deduplicating two terabytes of Kaggle notebooks, automatically fetching five terabytes of linked datasets, and scoring educational value and relevance using Qwen3-32B. Question-answer pairs grounded in verified execution traces were generated to fine-tune compact Qwen3-4B thinking and instruct models. The team released the trained models alongside the dataset and execution sandboxes, establishing a foundation for reinforcement learning and distillation on notebook workflows.


### [mmBERT: ModernBERT goes Multilingual](https://yomu.fyi/post/mmbert-modernbert-goes-multilingual.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Marc Marone, Orion Weller, William Fleshman, Eugene Yang, Dawn Lawrie, Ben Van Durme
- Published: Sep 9, 2025
- Type: Announcement

mmBERT is a massively multilingual encoder model trained on more than 3T tokens across over 1,800 languages to improve upon existing multilingual architectures like XLM-R. Built upon ModernBERT with a Gemma 2 tokenizer, mmBERT employs a three-phase training curriculum consisting of pre-training on 60 languages, mid-training on 110 languages, and a final decay phase covering 1,833 languages. The training pipeline integrates an inverse mask ratio schedule, dynamic language temperature annealing, and TIES merging across three decay variants. Benchmark evaluations demonstrate strong natural language understanding on English GLUE and multilingual XTREME, as well as competitive retrieval performance on MTEB v2 and CoIR. The release includes standard base and small models alongside open training data and checkpoints.


### [Welcome EmbeddingGemma, Google's new efficient embedding model](https://yomu.fyi/post/welcome-embeddinggemma-google-s-new-efficient-embedding-model.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen, Joshua, Alvaro Bartolome, Aritra Roy Gosthipaty, Pedro Cuenca, Sergio Paniego
- Published: Sep 4, 2025
- Type: Announcement

Google DeepMind released EmbeddingGemma, a multilingual embedding model with 308 million parameters and a 2048-token context window designed for on-device applications. Based on the Gemma3 transformer backbone, the architecture replaces causal attention with bidirectional attention to function as an encoder, followed by mean pooling and two dense layers producing 768-dimensional vectors. The model incorporates Matryoshka Representation Learning, allowing outputs to be truncated down to 512, 256, or 128 dimensions for reduced memory and storage footprints. Trained on approximately 320 billion multilingual tokens across more than 100 languages, the quantized model operates under 200 MB of RAM. In domain-specific evaluations on the MIRIAD dataset, fine-tuning increased NDCG@10 from 0.8340 to 0.8862, outperforming larger baselines.


### [SAIR: Accelerating Pharma R&D with AI-Powered Structural Intelligence](https://yomu.fyi/post/sair-accelerating-pharma-r-d-with-ai-powered-structural-intelligence.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Arman Zaribafiyan, Georgia Channing, Rudi Plesch, Zane Beckwith
- Published: Sep 2, 2025
- Type: Announcement

SandboxAQ released the Structurally Augmented IC50 Repository (SAIR), an open-source dataset containing 5.24 million computationally co-folded 3D protein-ligand structures paired with empirical IC50 binding potency data. The repository addresses training data scarcity in structure-based drug discovery, where over 40 percent of the included target proteins lack experimental structures in the Protein Data Bank. To construct the dataset, engineers executed over 130,000 GPU hours of the Boltz1 co-folding model on 760 NVIDIA H100 processors hosted via NVIDIA DGX Cloud on Google Cloud Platform. Infrastructure optimizations maintained over 95 percent GPU compute utilization, compressing the generation timeline from three months to three weeks. Quality validation using PoseBusters confirmed that 97 percent of the predicted complexes met physical plausibility and chemical sanity standards.


### [Building production-ready agentic systems: Lessons from Shopify Sidekick](https://yomu.fyi/post/building-production-ready-agentic-systems-lessons-from-shopify-sidekic.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 26, 2025
- Type: Explainer

Shopify’s Sidekick evolved from a simple tool-calling system into an agentic assistant that helps merchants manage stores through natural language, exposing scaling challenges in architecture, evaluation, and training. As its inventory grew beyond 50 tools, the system prompt accumulated conflicting special cases and became difficult to maintain; JIT instructions instead return relevant guidance with tool data. For evaluation, the team uses Ground Truth Sets based on production distributions, human labels from at least three product experts, statistical agreement measures, specialized LLM judges calibrated against humans, and merchant simulation. Judge correlation rose from Cohen’s Kappa 0.02 to 0.61 against a human baseline of 0.69, while syntax validation improved from about 93% to 99% after reward-hacking fixes. GRPO training exposed opt-out, tag, and schema hacking, and the post recommends modularity, procedural plus semantic validation, user simulation, and iterative judge refinement for production agentic systems.


[Newer posts](https://yomu.fyi/topic/machine-learning/page/21.md) · [Older posts](https://yomu.fyi/topic/machine-learning/page/23.md)