Loading…
Topic
617 posts about Machine Learning. Every summary links to the original.
See the idea
A walkthrough of how a 3D CT model turns a voxel volume into a structured diagnostic report: isotropic resampling, 3D patch tokens, spatial encoding, and language-model reasoning.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
amazon.science
Amazon's middle-mile logistics network coordinates tens of millions of packages across hundreds of facilities while managing subtle demand variations and major operational disruptions. To solve this mixed-integer optimization challenge without combinatorial explosion, Amazon models routes at coarse time resolutions augmented by precomputed 15-minute timing bounds. The system uses Monte Carlo methods alongside a graph attention network featuring interconnected site graphs and origin-destination graphs to capture complex spatial correlations and flow interdependencies. By generating synthetic flow permutations, planners stress-test candidate network designs against hundreds of plausible scenarios to prioritize routing optionality. This risk-aware modeling approach preserves network flexibility, absorbs severe shocks, and yields potential cost savings while safeguarding delivery promises.
This guide surveys practical machine learning applications across industries, from medical imaging and fraud detection to demand forecasting, document processing, and customer-service automation. It defines supervised, unsupervised, semi-supervised, and reinforcement learning, then relates technique selection to the business question, data type, and availability of labels. Technical coverage includes convolutional neural networks for image analysis, transformers and large language models for generative AI, and time-series workflows using cleaning, lag features, and forecasting. The operational guidance addresses feature stores, experiment tracking, CI/CD, drift monitoring, retraining, cost optimization, fairness, privacy, explainability, and model risk management. Templates and checklists structure projects around business problems, data sources, metrics, architecture, measured outcomes, validation, governance, and escalation, while cited Databricks case studies and tools provide implementation examples.
The Open ASR Leaderboard is adding private English speech-recognition datasets from Appen Inc. and DataoceanAI to evaluate scripted and conversational speech across several accents. To reduce benchmark-specific optimization (“benchmaxxing”) and test-set contamination, the datasets remain private, while the default Average WER continues to use public datasets only. A toggle exposes the private data, with macroaverages for overall, scripted, conversational, American, and non-American subsets; individual split scores are withheld to discourage provider- or accent-specific optimization. The leaderboard uses a shared Hub dataset, normalized transcripts and outputs, and open-source evaluation code, while private evaluation is run after public-set verification when models are submitted.
amazon.science
Amazon integrates responsible AI practices across four core model development phases: pretraining, post-training, evaluation, and third-party monitoring. During pretraining, researchers augment training corpuses with specialized safety datasets, multimodality alignments, and learning exercises to teach foundational safety concepts instead of purely filtering harmful text. In post-training, reinforcement learning from human feedback uses human preference rankings, auxiliary-reward models, and independent LLM judges to ensure model outputs adhere to safety policies. For specialized use cases requiring access to sensitive domains like security testing, researchers apply low-rank adaptors to alter model behaviors surgically without retraining base weights. Cross-functional policy teams guide these technical stages by mapping risks against eight responsible AI dimensions and continuously updating behavioral boundaries to reflect evolving regulatory frameworks.
Pinterest deployed a root-leaf architecture for online machine learning serving to decouple CPU-based feature preprocessing from GPU-based inference. Because the root cluster retrieved a union of features across models and fanned out the full set to every leaf, network bandwidth became a severe bottleneck. After initial tests with fbthrift LZ4 compression yielded a 20% bandwidth reduction at the cost of CPU and latency overhead, the team implemented Feature Trimmer. This mechanism uses exported model signatures as the source of truth to prune unused features on the root before transmission. By dispatching only required features per model, the system reduced network bandwidth, decreased latency, and enabled cluster rightsizing that saved over $4M annually.
BKK, Budapest’s unified transport authority, manages public transit, shared mobility, infrastructure, and traffic systems for a city of 1.7 million people. Its legacy on-premises warehouse and fragmented Excel- and PowerBI-based reporting struggled with growing, varied data, limiting efficient access and analysis. BKK phased a migration to Azure Databricks, starting with mobility datasets and carefully modeling and validating data from vehicle GPS, passenger sensors, boarding schedules, and other systems. The Databricks Lakehouse lets analysts process large geospatial datasets in shared SQL, Python, or R notebooks, supporting minute-level shared-mobility tracking, route-performance analysis, airport-bus forecasting, and dynamic scheduling. These capabilities provide faster insights and more responsive decisions, while governed access and cost visibility support broader collaboration and BKK’s vision for a digital twin of Budapest’s mobility system.
This guide distinguishes artificial intelligence, a broad field covering systems that perform tasks associated with human intelligence, from large language models (LLMs), specialized generative AI models for language and code. It places LLMs within generative AI and deep learning, and explains how transformer self-attention processes tokens while training data, parameter scale, and context windows shape capability and limitations. Examples include content drafting, code generation, translation, customer-service chatbots, sentiment analysis, extraction, summarization, and tool-connected agentic workflows, while traditional machine learning remains suited to structured labels and numeric predictions. It recommends retrieval-augmented generation, human review, bias testing, privacy controls, and evidence-based pilots with defined workflows, metrics, budgets, realistic data, and logged outputs.
This guide distinguishes data engineering from data science by their roles in the data value chain. Data engineers build and maintain ingestion pipelines, warehouses, lakes, access controls, and production systems, using tools such as SQL, Python, Scala, Java, Apache Spark, Kafka, orchestration platforms, and cloud infrastructure. Data scientists work with clean, accessible data to perform exploratory analysis, engineer features, train and validate statistical or machine-learning models, and communicate findings through visualization and storytelling. The roles collaborate through feedback on data quality, shared documentation, model deployment infrastructure, and versioned schemas, feature pipelines, and experiments. For choosing a path, the guide recommends trying a two-week pipeline project and a two-week ML project; it also outlines portfolio deliverables for each and says both roles require strong analytical and computer-science fundamentals.
The guide compares DevOps and MLOps, framing both as practices for reliable production systems but distinguishing traditional application code from machine-learning systems involving data and models. DevOps uses source control, automated testing, staging, and CI/CD, while MLOps governs the “holy trinity” of code, data, and models through separate versioning, quality checks, lineage, and access controls. Its central operational difference is Continuous Training: MLOps pipelines can validate model quality, monitor drift, and trigger retraining when data distributions shift, alongside testing model-serving infrastructure and promoting models through a registry. The guide also assigns responsibilities across data scientists, ML engineers, data engineers, and IT operations, and recommends dedicated MLOps for business-critical, frequently retrained models, DevOps for applications without ML, and split pipelines for hybrid products.
Data warehouse tools are presented as a strategic choice for analytics and ML teams facing fragmented estates of warehouses, lakes, and standalone ML systems. The guide proposes evaluating platforms across query performance, scalability, data integration, BI connectivity, total cost of ownership, and governance and security, with attention to MPP, columnar storage, and decoupled compute and storage. It contrasts traditional warehouses, data lakes, and lakehouses: warehouses favor structured SQL analytics, lakes offer native-format flexibility but weaker quality and performance guarantees, and lakehouses combine these capabilities using open formats such as Delta Lake and Apache Iceberg. The stated conclusion is that a lakehouse can provide a single governed foundation for SQL, BI, streaming, ML, and AI workloads, while teams should select tools according to workload, scale, budget, and future AI needs.
The NHS Federated Data Platform (NHS FDP), delivered by NHS England through a consortium led by Palantir, connects operational data held in separate NHS systems while keeping each organisation’s data under local control. Built on Palantir Foundry, its federated tenants use the Canonical Data Model, encryption, purpose-based access, and Privacy Enhancing Technology for national data flows. As of end December 2025, the post reports 110,078 additional theatre procedures, 797,728 patients safely removed from waiting lists, and 297,943 safely discharged patients, alongside around a 15% reduction in observed discharge delays. The inaugural Build with FDP hackathon brought together over 120 NHS developers and analysts, who built 25 working prototypes on a notional Canonical Data Model to address frontline operational challenges. Projects included early-warning, bed-demand prediction, CAMHS demand balancing, and genomic-optimised waitlist tools.
Video bounding-box annotations can contain ID swaps and position jumps that degrade training data for object-tracking models, especially when long videos are split among annotators and later merged. Uber's system runs in uLabel, combining visual, motion, and coordinate features from an 11-frame sliding window with an XGBoost classifier trained on synthetic anomalies from six open-source datasets. It scores each frame for error probability, then groups predictions with frame-index and confidence clustering into regions that operators can review, fix, or dismiss. The current iteration prioritizes jump and ID-swap detection, while freeze and scale/aspect errors remain planned for future releases; the deployed workflow is intended to reduce the cost and time of sequential edit-review while processing client data statelessly without retaining it or using it to train models.
Gas turbines produce millions of daily readings—vibration, temperature, pressure, flow rates, and electrical output—but maintenance teams may learn about warning signals only when an unplanned outage occurs. The post argues that predictive maintenance has struggled less because of model capability than because decision-makers lack fluid access to model findings and operational context. Databricks Genie provides a conversational interface to sensor, maintenance, dispatch, cost, and regulatory data, allowing questions about elevated vibration against maintenance baselines and timing maintenance around outage cycles. It is presented as decision support, not automation, with time-series analysis, maintenance history, integrated costs, and compliance context helping asset managers investigate fleets and act with more confidence.
Granite 4.1 is a family of dense decoder-only language models in 3B, 8B, and 30B sizes, trained on approximately 15T tokens. Its five-phase pre-training pipeline moves from broad web-scale data to progressively curated mixtures, increases math and code emphasis, and extends context from 4K to 512K tokens through staged long-context training. Supervised fine-tuning uses about 4.1M curated samples evaluated with LLM-as-Judge and rule-based checks, while reinforcement learning uses on-policy GRPO with DAPO loss. The 8B instruct model matches or surpasses the earlier Granite 4.0-H-Small 32B-A9B MoE model despite its simpler dense architecture and fewer parameters, and the Granite 4.1 models are released under Apache 2.0.
Thumbtack evaluates generative AI features across customer and service professional experiences to prevent unsupported claims, tone inconsistencies, and safety issues. The engineering organization transitioned from decentralized evaluation efforts to a dedicated cross-functional Evals team that provides shared infrastructure while preserving product team autonomy. System architectures combine rule-based checks, AI-as-a-judge scorers powered by DeepEval, trust and safety assessments, and crowdsourced human reviews. Teams leverage three parallel orchestration paths, including an MLflow-tracked stack, nightly Databricks batch jobs integrated with data warehouses, and multi-layer human-in-the-loop pipelines. Future milestones target centralized LLM judge registries, full task journey evaluations, automated judge-writing workflows, and cost-aware evaluation orchestration.
NVIDIA Nemotron 3 Nano Omni is introduced as an omni-modal understanding model for document analysis, multiple-image reasoning, automatic speech recognition, long audio-video understanding, agentic computer use, and general reasoning. It extends the Nemotron multimodal line from vision-language processing to text, image, video, and audio, targeting long, dense inputs such as 100-plus-page documents and mixed audio-visual evidence. Its unified encoder-projector-decoder combines a Nemotron 3 Nano 30B-A3B hybrid Mamba-Transformer MoE backbone with C-RADIOv4-H and Parakeet-TDT-0.6B-v2 encoders; dynamic visual resolution, Conv3D tubelets, and Efficient Video Sampling support multimodal context compression. The post reports leading or top results across document, video, audio-video, voice, and ASR benchmarks, alongside up to 9x higher throughput and 2.9x single-stream reasoning speed on multimodal use cases, and provides BF16, FP8, and NVFP4 checkpoints.
Palantir’s Ontology is presented as a decision-centric architecture for human-agent operations, intended to represent how enterprises make and execute decisions rather than only storing or analyzing data. It combines data, logic, action, and security in a shared, dynamic representation that can connect operational data, computational processes, workflow execution, and policy controls. The data layer includes structured, streaming, edge, unstructured, imagery, and decision data, with decision lineage recording the data version, application, and decision context; logic binding connects heterogeneous business, modeling, planning, and optimization assets for human and agent workflows. The architecture exposes approved actions to agents with configurable human review and guardrails, while capturing decisions as learning material for improving workflows and agentic memory. It concludes that a real-time ontology can anchor AI in enterprise conditions while controlling where automation is allowed.
Pinterest built a dedicated shopping candidate generation model to optimize for lower-funnel purchase conversions rather than relying strictly on engagement retrieval models. Because offsite conversion events are sparse and delayed, engineers trained a single multi-surface two-tower model using conversion data supplemented by duration-weighted click engagement and unengaged ad impressions as hard negatives. The two-tower retrieval architecture incorporates a parallel DCN v2 and MLP layer structure that allows simultaneous learning from the same inputs without information bottlenecks. Later revisions transitioned the system from a multi-head loss structure to a unified single-head multi-task architecture paired with an advertiser-level objective. Deployed across more than 600 million monthly active users, the system increased shopping conversion volume by 2.3% and boosted return on ad spend by 3.1% for United States shopping campaigns.
The guide explains how to build a Chrome extension that runs local AI with Transformers.js under Manifest V3, using a browser assistant as the reference architecture. It assigns model orchestration, lifecycle management, tool execution, and conversation state to a background service worker, while a side panel handles chat and a content script handles page extraction and highlighting. The implementation uses Gemma 4 E2B in q4f16 for text generation and all-MiniLM-L6-v2 in fp32 for embeddings, with WebGPU, DynamicCache-based KV caching, explicit download progress, and reusable pipelines. Typed message contracts, model-output parsing, and separate internal and UI transcripts support a deterministic tool loop, while chrome.storage.local, IndexedDB, and background caches place data according to lifecycle and access needs; the guide concludes that separating these runtimes keeps the UI responsive and avoids duplicate model loads.
Shopify built a tool-calling agent for Sidekick that converts natural-language requests into Shopify Flow workflows, addressing the cold-start problem by working backward from thousands of anonymized production workflows. The team generated synthetic queries and tool trajectories, fine-tuned Qwen3-32B, and changed Flow’s nested JSON DSL into an equivalent Python representation connected by a round-trip transpiler. Using Python improved syntactic correctness by 22 points and semantic correctness by 13 points on a 300-example benchmark, while matching production tool names, ordering, responses, and prompts proved essential to avoid degradation. Production evaluation exposed a 35% gap masked by benchmark parity, so Shopify built a weekly feedback loop that scores conversations, filters training data, identifies slice gaps, retrains, and deploys. The resulting agent serves most production traffic, runs 2.2 times faster at 68% lower cost, and outperforms closed models.
The best engineering reads, once a week.
