---
title: "Machine Learning"
description: "622 posts about Machine Learning, summarised, each linking to the original."
---

# Machine Learning
> 622 posts about Machine Learning, summarised, each linking to the original.

## Articles

### [Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains](https://yomu.fyi/post/introducing-mellum2-a-12b-mixture-of-experts-model-by-jetbrains.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nikita Pavlichenko
- Published: Jun 1, 2026
- Type: Announcement

JetBrains is releasing Mellum2, a 12B-parameter Mixture-of-Experts model trained from scratch on natural language and code. It targets latency-sensitive, high-throughput workloads including routing, retrieval-augmented generation (RAG), summarization, sub-agents, and coding features. The model has 12B total parameters but activates 2.5B per token, with its architecture keeping total capacity high while making inference more efficient for real-time use. Released under the Apache 2.0 license, Mellum2 is described as competitive with similarly sized open models on code generation, reasoning, science, and math benchmarks while delivering more than 2x faster inference. Its text-and-code focus supports high-throughput production workloads and private deployments where proprietary code or internal data may be involved.


### [Executable UDFs are now in public beta on ClickHouse Cloud](https://yomu.fyi/post/executable-udfs-are-now-in-public-beta-on-clickhouse-cloud.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 30, 2026
- Type: Announcement

Executable UDFs are available in public beta on ClickHouse Cloud, allowing Python functions packaged as zip files to be called from SQL like built-in functions. ClickHouse runs them in a pool of long-lived sandboxed processes, keeping model code near the data and supporting use in ad-hoc queries, joins, and materialized views. The demonstration wraps a roughly 270K-parameter PyTorch autoencoder in an embed\_trade UDF that consumes 14 features, produces a 32-dimensional embedding and reconstruction error, and scores trades as they enter through a materialized view. Training uses streamed Parquet chunks, incremental StandardScaler fitting, early stopping, and saved model artifacts; the deployed pool keeps the 2MB model hot, so inference occurs once per trade and downstream queries read the resulting table.


### [Winning under CMS TEAM: Building the learning health system to realize success in VBC today and tomorrow](https://yomu.fyi/post/winning-under-cms-team-building-the-learning-health-system-to-realize.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Kayla Grieme, Benjamin Goldsteen, Shankara Ettigi
- Published: May 29, 2026
- Type: Problem & solution

CMS’s Transforming Episode Accountability Model (TEAM), beginning January 1, 2026, requires selected organizations to manage total cost and quality across five surgical episode categories and a 30-day post-discharge window. The post argues that retrospective dashboards and delayed claims data cannot support intervention or prepare hospitals for downside risk, especially amid complex integration across EHR, claims, post-acute, social-determinants, and provider data. It proposes a modern, cloud-native lakehouse foundation with unified data, operationalized AI/ML, embedded clinical workflows, and scalable architecture. Recommended capabilities include risk stratification, next-best-action decision support, post-acute optimization, provider benchmarking, and predictive performance alerts. In a composite 500-bed medical-center example, the approach is associated with 15% lower SNF costs, 8% better risk-capture completeness, and 12% fewer readmissions.


### [Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler](https://yomu.fyi/post/profiling-in-pytorch-part-1-a-beginner-s-guide-to-torch-profiler.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Aritra Roy Gosthipaty, Sayak Paul, Sergio Paniego, Rémi Ouazan Reboul, Pedro Cuenca
- Published: May 29, 2026
- Type: Tutorial

The post introduces torch.profiler through a beginner-oriented walkthrough of profiling a matrix multiplication and addition, explaining how to configure the profiler and interpret its outputs. It uses a small function with torch.matmul and torch.add, annotates it with record\_function, captures CPU and CUDA activities after warm-up iterations, and exports a profiler table plus a Chrome trace. The table summarizes event counts and CPU or GPU time, while the trace shows execution order, call hierarchy, CPU-GPU gaps, kernel launches, and overlap. Comparing 64x64 and 4096x4096 matrices demonstrates a shift from overhead-bound behavior to compute-bound behavior; the guide also covers dispatch chains, hidden device-to-device copies, profiler buffer activity, and torch.compile's runtime and caching behavior.


### [Amazon Research Awards recipients announced](https://yomu.fyi/post/amazon-research-awards-recipients-announced.md)
- Company: amazon.science
- Author: Amazon Research Awards team
- Published: May 27, 2026
- Type: Announcement

Amazon Research Awards has announced 70 award recipients across 49 universities in 11 countries for its fall 2025 funding cycle. The funded research spans six distinct tracks: AI for Information Security, Agentic AI, Automated Reasoning, AWS Cryptography, Cybersecurity and Anti-Abuse Technologies, and Sustainability. Each selected academic team receives unrestricted financial grants, AWS Promotional Credits to access specialized cloud AI and machine learning services, and access to more than 700 Amazon public datasets. Awardees are paired with an Amazon research contact who provides advisory support, consultation, and access to company training sessions. Furthermore, the program explicitly encourages recipients to publish their discoveries, deliver presentations at Amazon offices worldwide, and release associated software implementations under open-source licenses.


### [Reachy Mini goes fully local](https://yomu.fyi/post/reachy-mini-goes-fully-local.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amir Mahla, Andres Marafioti
- Published: May 27, 2026
- Type: Tutorial

Reachy Mini’s conversation app can now run its speech stack locally instead of sending audio to a server, avoiding cloud, API keys, and data leaving the machine. The tutorial uses speech-to-speech as a cascaded VAD → STT → LLM → TTS pipeline exposed through a Realtime API-compatible /v1/realtime WebSocket, with Silero VAD v5, Parakeet-TDT 0.6B v3, Qwen3-TTS, and a selectable LLM backend. It shows llama.cpp serving Gemma 4 E4B with parallel slots, 64k context, flash attention, and full sliding-window attention, then covers vLLM, MLX, Transformers, hosted Responses API endpoints, and network connection to a wireless robot. Once the backend and app are connected, conversations run through the local loop, while individual stages can be exchanged to trade latency, quality, language coverage, and hardware choices.


### [Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL](https://yomu.fyi/post/shipping-a-trillion-parameters-with-a-hub-bucket-delta-weight-sync-in.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amine Dirhoussi, Quentin Gallouédec, Kashif Rasul, Lewis Tunstall, Edward Beeching, Albert Villanova del Moral, Leandro von Werra, Sergio Paniego
- Published: May 27, 2026
- Type: Problem & solution

Async RL weight synchronization currently requires the trainer to send a complete model to the inference engine after each optimizer step, creating a critical-path transfer that can reach roughly a terabyte for a frontier 1T-parameter checkpoint. The article attributes the opportunity for reduction to bf16 arithmetic: at RL learning rates, many updates fall below the bf16 visibility threshold, leaving about 99% of weights bit-identical between steps and more than 98% even in the reported worst case. TRL's delta-weight-sync implementation records changed elements around the optimizer step, encodes them in sparse safetensors, uploads each delta to a Hugging Face Bucket, and lets vLLM fetch and apply it. On Qwen3-0.6B, this reduces each per-step payload from 1.2 GB to 20–35 MB, while a demonstrated setup connected separate trainer, vLLM, and Wordle Spaces through one bucket. Open issues include extra CPU bf16 snapshots, fixed anchor cadence, and unmeasured multi-node FSDP2 support.


### [Diverse reasoning traces teach LLMs to make better decisions](https://yomu.fyi/post/diverse-reasoning-traces-teach-llms-to-make-better-decisions.md)
- Company: amazon.science
- Author: Sheng Jia, Xiao Wang, Shiva Kasiviswanathan
- Published: May 26, 2026
- Type: Explainer

Large language models typically rely on supervised fine-tuning with a single human-verified reasoning trace per example, which limits reasoning breadth and causes mode collapse when attempting parallel reasoning. Researchers presented set-supervised fine tuning (SSFT) and global forking policy optimization (GFPO) to teach models diverse problem-solving strategies. SSFT represents reasoning as a set of complete solution paths paired with global forking tokens, utilizing bipartite matching to achieve order-invariant training and specialized tokens. GFPO applies lightweight reinforcement learning to select the optimal forking token upfront, detaching gradients from generated rollouts to ensure stable optimization. On standard reasoning and coding benchmarks, the combined framework increased single-shot accuracy by 5% to 7% while improving pass@k diversity without degrading pass@1 accuracy.


### [Harness, Scaffold, and the AI Agent Terms Worth Getting Right](https://yomu.fyi/post/harness-scaffold-and-the-ai-agent-terms-worth-getting-right.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sergio Paniego, Aritra Roy Gosthipaty
- Published: May 25, 2026
- Type: Explainer

AI-agent terminology has outpaced shared definitions, leaving terms such as harness and scaffold blurred across frameworks and contexts. The glossary separates the model, which produces text but has no memory or execution loop, from scaffolding, which defines instructions, tools, output parsing, and context management, and the harness, which runs model calls, routes tool use, and decides when to stop. It presents an agent as a model plus the surrounding system that lets it act repeatedly, while distinguishing tools, skills, sub-agents, and orchestrators by their roles. For training, it defines the environment, trainer, rollout, and reward, including verifiable or learned scores and sparse or dense feedback. Because usage is not universal, the glossary offers a practical mental model rather than enforcing one vocabulary.


### [Making User-Sequence Data More Cost-Efficient, Faster, and Easier to Use](https://yomu.fyi/post/making-user-sequence-data-more-cost-efficient-faster-and-easier-to-use.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: May 21, 2026
- Type: Problem & solution

User sequences serve as core primitives powering machine learning models across ranking, retrieval, and recommendation surfaces at Pinterest. Managing these sequences previously caused high infrastructure costs, operational fragility, and data drift between offline training pipelines and online serving systems. In response, engineering teams redesigned the platform around a single definition executed across multiple runtimes, including real-time streaming, batch backfills, and online inference. The updated architecture pairs a unified enrichment engine with columnar time-partitioned storage and standardized validation tiers. Consequently, the redesign achieved lower storage and network costs, faster signal onboarding via configuration, and improved engagement metrics across major recommendation surfaces.


### [Transforming industries with conversational AI: Partner solutions built on Databricks Genie](https://yomu.fyi/post/transforming-industries-with-conversational-ai-partner-solutions-built.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Amit Singh
- Published: May 21, 2026
- Type: Announcement

Databricks presents a first cohort of partner-built industry solutions that use Genie to reduce the “Analyst Bottleneck,” where leaders wait for custom SQL or dashboard updates. Genie provides a conversational analytics layer grounded in Unity Catalog metadata and business semantics, using specialized AI agents to return governed, secure, traceable answers to natural-language questions. The featured accelerators span communications, media and entertainment, financial services, healthcare and life sciences, manufacturing and energy, public sector, and retail, with examples including churn prediction, next-best-action recommendations, billing anomaly investigation, KYC monitoring, actuarial analysis, and commercial insights. Across the examples, solutions combine domain expertise with Databricks platform capabilities such as machine learning, Agent Bricks, Lakebase, Serverless SQL, and auditable governance to support self-service analysis and operational decisions.


### [OlmoEarth v1.1: A more efficient family of Earth observation models](https://yomu.fyi/post/olmoearth-v1-1-a-more-efficient-family-of-earth-observation-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kyle Wiggers
- Published: May 19, 2026
- Type: Announcement

OlmoEarth v1.1 is a new family of Earth observation models designed to reduce the compute required for large-scale satellite imagery predictions. Compute dominates the lifecycle from data export through preprocessing, inference, and post-processing, motivating a focus on transformer token sequence length. For Sentinel-2, the models collapse three resolution-specific tokens into one, reducing token counts threefold; this initially caused a 10 ppt drop on the m-eurosat kNN benchmark, so the pre-training regimen was modified. OlmoEarth v1.1 runs up to three times cheaper than v1 while maintaining similar performance across a mix of research benchmarks and partner-built tasks, although some regressions remain, and the same dataset is used to isolate methodological effects.


### [Scaling Real-Time Traffic Forecasting with a Graph-Aware Transformer](https://yomu.fyi/post/scaling-real-time-traffic-forecasting-with-a-graph-aware-transformer.md)
- Company: [Uber](https://yomu.fyi/company/uber.md)
- Published: May 19, 2026
- Type: Explainer

DeepETT is Uber’s real-time traffic forecasting system for estimating traversal times on roughly 100 million road segments, supporting routing and arrival-time estimates. It replaces a legacy blend of recent observations and historical patterns with a deep-learning model designed for rapidly changing conditions, sparse roads, and long trips. To keep deployment predictable at global scale, the system uses fixed-size computation over pre-aggregated segment, graph, region, hourly, weekly, and event views rather than a dynamic graph, while predicting segment ETTs independently of routing. Training targets segment-level mean squared error, while trip-level mean absolute error is evaluated offline after routing sums segment forecasts; calibration and continual real-time correction address downstream error. In a global A/B test, long-trip arrival-time accuracy improved 6%, variance explained rose 19%, navigation defects fell 2.73%, and mobility gross bookings increased by an estimated $100 million annually, with forecasts exceeding 2 million per second.


### [Introducing the Ettin Reranker Family](https://yomu.fyi/post/introducing-the-ettin-reranker-family.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen
- Published: May 19, 2026
- Type: Announcement

Tom Aarsen introduces six Sentence Transformers CrossEncoder rerankers, ranging from 17M to 1B parameters, alongside their training data and full recipe. Each model uses an Ettin ModernBERT encoder with unpadded attention, RoPE, GeGLU, up to 8,192 tokens of context, and a modular classification head whose ablations favored CLS pooling over mean pooling. Training applies pointwise MSE distillation to scores from mixedbread-ai/mxbai-rerank-large-v2 using a broad and retrieval-focused dataset mixture. The conclusion reports state-of-the-art results at every released size, with the 400M model within 0.0024 and the 1B model within 0.0001 of the 1.54B teacher on MTEB, while all models outperform the ms-marco-MiniLM-L\*-v2 family on MTEB and NanoBEIR.


### [Making LLMs faster without sacrificing accuracy](https://yomu.fyi/post/making-llms-faster-without-sacrificing-accuracy.md)
- Company: amazon.science
- Author: Tao Yu, Youngsuk Park
- Published: May 15, 2026
- Type: Problem & solution

Standard scaling laws optimize parameter count and training data budgets to minimize loss but ignore internal Transformer architecture configurations. To address throughput disparities among equally sized models, researchers developed a scaling framework incorporating hidden size, the multilayer perceptron to attention parameter ratio, and grouped-query attention. The method calibrates a correction surface over standard Chinchilla scaling laws to independently optimize separable architectural parameters for target accuracy. Evaluated across more than 200 models up to three billion parameters, the framework produced the Panda model family for maximum accuracy and the Surefire family for Pareto efficiency. Surefire models matched or exceeded LLaMA-3.2 accuracy while increasing inference throughput by 12% to 47% across vLLM and SGLang serving systems on modern GPUs.


### [Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality](https://yomu.fyi/post/granite-embedding-multilingual-r2-open-apache-2-0-multilingual-embeddi.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Radu Florian, Parul Awasthy, Aashka Trivedi, Madison Lee
- Published: May 14, 2026
- Type: Announcement

Granite Embedding Multilingual R2 introduces two Apache 2.0 multilingual embedding models built on ModernBERT: a 97M-parameter compact model and a 311M full-size model. Both cover more than 200 languages, receive enhanced retrieval training for 52 languages and nine programming languages, and support context lengths up to 32,768 tokens, compared with 512 for the R1 models. The compact model scores 60.3 on MTEB Multilingual Retrieval, while the full-size model scores 65.2; the reported results place them among the strongest open models in their parameter ranges. R2 replaces the XLM-RoBERTa architecture with alternating attention lengths, rotary position embeddings, newer multilingual and code tokenizers, and Flash Attention 2.0 support, while the 311M model adds Matryoshka embeddings and both models provide ONNX and OpenVINO weights.


### [Beyond Prediction: Solving the Multiple Knapsack Problem at Scale: How Uber Optimizes Incentives](https://yomu.fyi/post/beyond-prediction-solving-the-multiple-knapsack-problem-at-scale-how-u.md)
- Company: [Uber](https://yomu.fyi/company/uber.md)
- Published: May 14, 2026
- Type: Explainer

Uber’s Tarot platform frames incentive allocation as a Multiple Knapsack Problem (MKP), replacing FIFO contention handling and reactive budget management. It combines segmentation, budget pacing, uplift predictions, multi-objective value scoring, and a plug-and-play optimizer coordinated by an orchestrator. For each run, Tarot forms user-lever opportunities, predicts incremental ROI and cost, reconciles available budget, and selects assignments under team, business, and concurrency constraints before publishing them to domain services. The initial HiGHS linear programming approach took more than 24 hours to find a feasible solution for 100,000 users and one lever, while CP-SAT solved the same dataset in minutes and handled millions of users across multiple levers within stated SLA timeframes. The result is a production architecture that moves beyond heuristics by combining constraint programming with budget pacing, while future work targets automated exploration and model feedback.


### [Promptimus: Improving already good LLM prompts with zero manual engineering](https://yomu.fyi/post/promptimus-improving-already-good-llm-prompts-with-zero-manual-enginee.md)
- Company: amazon.science
- Author: Zhengyuan Shen, Yunfei Bai, Sullam Jeoung, Shuai Wang
- Published: May 14, 2026
- Type: Problem & solution

Enterprise applications rely on mature large language model prompts that incorporate intricate business logic and regulatory constraints, making further optimization and cross-model migration challenging. Promptimus automates prompt refinement through a four-step iterative loop that evaluates candidate prompts against decomposed metric checkpoints, diagnoses failure bottlenecks, and generates targeted improvement strategies. The system operates either via full prompt rewrites in standard mode or through surgical, programmatic find-and-replace edits in edit mode to preserve structured taxonomies and schemas. Tested across multimodal classification and code generation tasks, the framework achieves performance gains with small development datasets of 20 to 50 samples. Amazon is making the model-agnostic optimization system available through Amazon Bedrock to support enterprise generative-AI model migration without manual engineering.


### [Unlocking asynchronicity in continuous batching](https://yomu.fyi/post/unlocking-asynchronicity-in-continuous-batching.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rémi Ouazan Reboul, Pedro Cuenca, Aritra Roy Gosthipaty
- Published: May 14, 2026
- Type: Tutorial

Continuous batching improves GPU utilization with tightly packed batches, but its synchronous loop makes the CPU and GPU take turns, leaving the GPU idle while the CPU prepares each next batch. Profiling 8K-token generation with a batch size of 32 and an 8B model recorded 300.6 seconds, with the GPU idle for 24.0% of runtime. The post builds asynchronous batching by overlapping CPU preparation with GPU computation, using non-default CUDA streams, non-blocking transfers, and synchronization events to coordinate dependencies. In the same experiment, GPU activity rose to 99.4% and total generation time fell to 234.5 seconds, a 22% speedup, without new kernels or model changes while maintaining model accuracy.


### [Building Blocks for Foundation Model Training and Inference on AWS](https://yomu.fyi/post/building-blocks-for-foundation-model-training-and-inference-on-aws.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Keita Watanabe, Pavel Belevich, Aman Shanbhag
- Published: May 11, 2026
- Type: Explainer

The post explains how foundation-model scaling now spans pre-training, post-training, and test-time compute, creating shared infrastructure requirements rather than a single pre-training curve. It examines how AWS infrastructure and open-source tools support these workloads across accelerated compute, networking, storage, orchestration, ML frameworks, and observability. Technical detail includes NVIDIA H100, H200, B200, and B300 GPU characteristics, NVLink/NVSwitch scale-up communication, and EFA-based scale-out networking for distributed collectives. The architecture also incorporates EC2 P instances, tiered storage, Slurm, Kubernetes, SageMaker HyperPod, CUDA, NCCL, PyTorch, Prometheus, and Grafana. The conclusion presents integration points between these layers as the basis for diagnosing bottlenecks and making scaling decisions across the foundation-model lifecycle.


[Newer posts](https://yomu.fyi/topic/machine-learning/page/13.md) · [Older posts](https://yomu.fyi/topic/machine-learning/page/15.md)