Loading…
Topic
620 posts about Machine Learning. Every summary links to the original.
See the idea
A walkthrough of how a 3D CT model turns a voxel volume into a structured diagnostic report: isotropic resampling, 3D patch tokens, spatial encoding, and language-model reasoning.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
An adoption bottleneck slows artificial intelligence integration because standalone tools lack connectivity with everyday enterprise workflows. While models can draft responses or process text, they fail to access siloed background information from emails, chat channels, and shared documents. Consequently, human users must expend disproportionate effort gathering context, granting tool permissions, and editing preliminary outputs. Michael Foree explains that overcoming this hurdle requires context engineering, where practitioners deliberately observe their own thought processes to identify relevant information sources and filter out distracting data. Testing mock prompts and iteratively refining context architectures allows practitioners to bridge the gap between standalone model capabilities and autonomous task execution.
Salesforce needed to deliver major releases across 34 languages as product localization volume surged by more than 35% under fixed budgets and unchanged delivery windows. To address these scaling constraints, the engineering team replaced their legacy translation workflows with an LLM-powered orchestration pipeline integrating prompt engineering, context engineering, and multi-stage validation. The architecture transforms existing style guides, glossaries, and terminology databases into structured context, routing interface strings through specialized prompt stages—up to 85 stages for some languages—to enforce product context, grammar, and branding rules. Combined with an automated validation loop spanning AI editing, AI validation, and human review, this system reduced localization costs by 50 to 90% while accelerating turnaround times across enterprise releases.
Adapting open models to specialized tasks often requires significant infrastructure, technical expertise, and GPU resources. This tutorial demonstrates how developers can customize NVIDIA Nemotron 3 Nano using Prime Intellect Lab for hosted reinforcement learning with verifiable rewards. The workflow establishes a baseline evaluation on a Python Math task, trains a LoRA adapter using a TOML configuration, and reevaluates performance. Across 32 held-out problems capped at five assistant turns, the model improves its accuracy from 21.9% to 90.6% for less than five dollars in compute spend. The same hosted training approach applies to larger models in the family, including Nemotron 3 Super and Nemotron 3 Ultra.
Modern diffusion transformers running in BF16 often require 20–30 GB of VRAM, limiting access on consumer GPUs; the integration brings Nunchaku Lite checkpoints into Diffusers to reduce memory and denoising latency. SVDQuant handles activation outliers by moving them into weights, preserving a small 16-bit low-rank branch while quantizing residual weights to 4-bit, enabling W4A4 compute in transformer attention and MLP projections. Nunchaku Lite patches standard nn.Linear modules at runtime, loads Hub-hosted CUDA kernels through the kernels package, and exposes ordinary from_pretrained() loading without local compilation; AWQ W4A16 handles modulation and normalization projections. On an RTX 5090, the cited NVFP4 checkpoint generates 1024×1024 images in about 1.7 seconds at roughly 12 GB peak memory versus about 24 GB for BF16, while the bare-bones Nunchaku Lite path delivers around 30% speedup; torch.compile raises reported end-to-end speedup from 1.35x to 1.8x.
An ML training run was exhausting worker memory before training could start because a cold Snowflake fetch peaked at 88.8 GiB RSS, versus 17.6 GiB when loading local Parquet. The shared Snowflake-to-pandas path created Python objects and SQLAlchemy Row proxies before pandas built the DataFrame, with temporary representations requiring roughly three times the final DataFrame size at peak. An Apache Arrow path for supported reads reduced median peak process RSS growth by 60–79% across three query shapes, while peak Python allocations fell by 69–87%. The narrow and wide tests also ran faster, but aggregate timing was mixed, so elapsed time was not the headline claim. Existing jobs adopted the default without query rewrites, with normalization preserving the pandas contract and SQLAlchemy retained for unsupported reads; in one workflow, lower fetch peaks doubled the training-data window on the same cluster size.
Long-running NVIDIA TensorRT engine builds can freeze workflows and waste GPU hours when they lack visibility or abort mechanisms. Developers can resolve this by subclassing the IProgressMonitor API in Python or C++ and attaching it to the builder configuration. The monitor tracks hierarchical build phases through start, step completion, and finish callbacks, requiring thread synchronization because TensorRT calls monitor methods across internal builder threads. Returning false from the step completion callback requests early build cancellation, which unwinds active phases in reverse order at the next step boundary. Beyond terminal rendering with ANSI escape codes, teams can stream these progress events to IDE protocols, HTTP service endpoints, or agent tool-call traces.
Industrial edge environments often face significant unplanned downtime yet lack reliable cloud connectivity to run large-scale artificial intelligence models. To address this limitation, an offline-first reference architecture shifts generative inference to edge devices while leveraging cloud services for preparation and lifecycle management. The workflow uses Amazon Bedrock with Amazon Nova Pro to create labeled dataset pairs, Amazon SageMaker AI Pipelines to fine-tune a compact model, and AWS IoT Greengrass to orchestrate edge deployments. Local execution relies on Ollama, Strands Agents, and a CPU-based ChromaDB retrieval pipeline that consumes zero GPU VRAM. In evaluations across three foundation model judges, the fine-tuned gpt-oss-20b model combined with retrieval-augmented generation consistently outperformed the base model configuration across accuracy, completeness, and relevance metrics.
Dow built the Carbon Footprint Ledger (CFL) on Databricks to provide a transparent, verifiable way to track and communicate product carbon footprints amid customer Scope 3 commitments and tighter environmental disclosure requirements. The system combines carbon-accounting methodology assured against ISO 14067 and the GHG Protocol Product Standard with a calculation engine and certificate-issuing ledger that maintains residual balances. Within Dow’s Integrated Data Hub, Apache Spark unifies supply-chain, sustainability, operations, and other source data, while Unity Catalog governs access and Delta tables provide ACID transactions, time travel, and efficient upserts. An advanced optimization model, developed and deployed with MLflow, identifies lowest-greenhouse-gas production pathways and sends versioned, monitored results through the pipeline with lineage tracking. The implementation reduced PCF processing from weeks to a fraction of that, enabled portfolio-scale calculations, and supports verifiable certificates and commercial use of decarbonization investments.
Frontier AI pre-training increasingly relies on mixture of experts architectures, which lower compute per token while introducing severe all-to-all communication bottlenecks. To address these scaling limits, the NVIDIA GB300 NVL72 rack-scale system co-designs Blackwell Ultra GPUs, fifth-generation NVLink scale-up fabrics, and 800 Gbps SuperNIC scale-out networking. Testing on DeepSeek-V3 671B across 256 GPUs achieved a pre-training benchmark of 1,648 TFLOPs per GPU using Megatron Core. In addition to hardware co-design, ongoing software optimizations across Megatron Core, TorchTitan, and JAX contributed major throughput gains over time. Scaling from 256 to 1,024 GPUs sustained over 97% per-GPU performance efficiency across all evaluated frameworks by overlapping scale-out gradient traffic behind compute windows.
Agentic AI workloads shift critical execution paths onto CPUs through sandboxed environments that execute code, invoke tools, and manage runtime states. The NVIDIA Vera CPU addresses these compute requirements via Olympus cores built to maximize single-thread performance and instructions per cycle under full socket load. To handle irregular control flows and pointer-heavy structures, Olympus combines a ten-wide decode engine, neural branch prediction supporting two taken branches per cycle, dependency-breaking acceleration, and dedicated graph prefetchers. Across scale-up systems, dual-socket Vera configurations leverage second-generation NVLink-C2C interconnects to maintain a clean two-NUMA-node topology alongside PCIe 6.4, CXL 3.1, and Arm CCA confidential computing. These architectural choices provide predictable latency and memory bandwidth across highly concurrent reinforcement learning and agent loops.
Airbnb replaced hundreds of hand-crafted ranking features with a Transformer-based sequence model that captures both long-term booking history and short-term browsing behavior. To keep latency low and throughput high, sequence representations are generated via daily batch jobs and combined with real-time queries using a co-trained setwise ranker.
Grabette is an open, low-cost system for recording robot-manipulation data without requiring a robot, lab, or teleoperation rig. It uses a handheld gripper with a wide fisheye observation camera, an RGBD tracking camera, an IMU, and gripper encoders to capture synchronized demonstrations and recover camera-local 6-DoF Cartesian pose plus gripper state. Users save episodes on a Raspberry Pi, then select them in a browser dashboard for upload, RTAB-MAP SLAM, trajectory verification, and conversion into a LeRobot dataset on the Hugging Face Hub. The release also includes a motorized Gripette gripper and an example that trains a Diffusion Policy on 200 demonstrations and evaluates it on an OpenArm 7-DoF arm via gRPC, demonstrating the recording-to-training workflow.
Thumbtack developed a transformer-based category recommender to personalize email and push notification suggestions across roughly 500 home service categories. Because user activity is sparse while contextual signals like location and climate are rich, the architecture represents search history and context features as sequences of tokens evaluated against candidate category tokens using cross-attention. Initial training with naive cross-entropy suffered from severe popularity bias, skewing recommendations toward head categories and yielding a low tail hit rate. To resolve this issue, the team implemented adaptive mixed sampling, combining in-batch and random negative candidates and keeping only top-scoring hard negatives for cross-entropy loss computation. This method raised tail hit rate from 2.1% to 28.4%, improved overall HR@10 to 54.6%, and yielded a statistically significant 5% lift in requests per user in production A/B testing.
Scale-up networking has emerged as a critical architectural component for AI factories supporting trillion-parameter models, mixture-of-experts architectures, and disaggregated serving. NVIDIA presents its sixth-generation NVLink scale-up fabric and NVLink 6 Switch to address latency-sensitive all-to-all GPU communication and offload collective operations. In simulation benchmarks on models like DeepSeek-R1, Qwen 235B, and a simulated 2T parameter model, NVLink achieved up to 2.3X higher decode throughput than off-the-shelf Ethernet in a 72-accelerator domain. The architecture integrates NVLink-C2C to supply 1.8 TB/s of coherent CPU-GPU bandwidth in the Vera Rubin NVL72 platform, as well as NVLink Fusion for integrating semi-custom silicon XPUs. Through co-designed hardware, SHARP in-network compute, and NCCL libraries, the platform optimizes token throughput, factory uptime, and deployment economics.
NVIDIA Omniverse libraries, included in NVIDIA Agent Toolkit, provide modular components for incorporating physical AI features into current tools and services. As part of this stack, the ovrtx prerelease library offers a lightweight C and Python SDK for real-time RTX sensor simulation and visualization using OpenUSD scenes. Host applications retain architectural and workflow control by stepping the renderer directly from their internal loops. Integration relies on ovstage for shared OpenUSD data management alongside RenderProducts and RenderVars to configure camera, lidar, radar, and semantic segmentation outputs. PTC Onshape Render Studio is exploring ovrtx to embed RTX rendering directly into cloud-native CAD and PDM workflows.
AI transparency is presented as the practice of documenting and disclosing an AI system’s data, model behavior, decision-making processes, and accountability so affected stakeholders can evaluate its outputs. The guide distinguishes transparency from explainability, which addresses a specific prediction, and interpretability, which concerns direct access to a model’s internal logic. It recommends maintaining model architecture and version history, algorithms and hyperparameters, training-data provenance, and explainability tooling in a central registry, alongside model cards and data sheets. It also calls for subgroup performance metrics, recurring audits, visible AI disclosures, human-review paths, and incident playbooks covering notification, logs, rollback, remediation, and outcome tracking. The stated goal is durable governance that supports trust, bias detection, regulatory documentation, and accountability across high-stakes and generative-AI deployments.
Together AI and Y Combinator are partnering to provide YC portfolio companies with a dedicated GPU cluster, addressing compute access as a growing bottleneck for AI-native startups. The service is designed for both inference and training, letting founders reserve, provision, and manage GPUs through Together’s self-service portal rather than making long-term commitments or routing usage through YC. Startups can run short-term sprints at long-term rates, use capacity from single-node workloads to larger deployments, and receive GPUs in minutes, while the cluster is already running at full utilization and companies can plan needs months ahead. Together says its research on attention mechanisms and the Mamba architecture targets faster inference and improved unit economics. The partners plan to expand the cluster and support startups as their requirements grow.
Netflix established an in-house serving platform to run large language model inference directly inside existing production environments alongside traditional machine learning models. Built upon NVIDIA Triton Inference Server and vLLM, the unified architecture handles member-scale routing, candidate generation, feature fetching, and model execution over gRPC and OpenAI-compatible HTTP endpoints. The team shifted their primary engine from TensorRT-LLM to vLLM to support custom architectures, simpler debugging, and non-trivial constraint logic. Addressing production obstacles required patching Triton's frontend for guided decoding, pinning dependent library versions, rewriting logits processors in C++ for vLLM V1, and handling state machine resets during engine preemptions. The resulting platform unifies deployment flows while preserving operational stability across evolving model schemas.
Swiggy’s pLTV effort addresses early-stage lifetime-value prediction for Food Delivery and Instamart, where sparse signals, zero-inflated outcomes, long tails, and trial-driven acquisition make conversion models unsuitable. It encodes 350+ pre-order signals across 12 groups in a simple multilayer perceptron with three shared hidden layers, task-specific heads, a custom MAE loss, and Adam optimization. A four-head variant added order-count predictions for each vertical, producing marginally better performance with 135K rather than 363K parameters. Evaluation emphasized ranking through Spearman correlation and decile diagonal coverage, reaching above 0.75 Spearman and 70–80% diagonal coverage. In production bidding experiments, the in-house model delivered better retention and higher gross order value per acquired user at no incremental platform cost, while a third-party platform acquired more users per dollar.
Scientific workflows generate data across instruments, teams, and partner networks, but siloed systems can strip away metadata, lineage, and experimental context. Luma, Dotmatics’ scientific intelligence platform, captures outputs continuously and harmonizes them into structured, FAIR-compliant records, while Databricks supplies scalable storage, governance, and enterprise data and AI infrastructure. Running natively on Databricks, Luma preserves a continuous digital thread across experiment design, acquisition, analysis, reporting, and sharing, with Delta Sharing supporting governed exchange with collaborators. Chromatography illustrates the approach: connectivity across vendor systems and Virscidian’s Analytical Studio automate LC/MS processing while retaining context and adding dashboards, registration, and management tools. In one pharmaceutical deployment, Luma connected about 1,500 of more than 5,000 instruments across four LC/MS vendors, enabling cross-vendor performance trending, unified purity analysis, and a foundation for AI and machine learning.
The best engineering reads, once a week.
