Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Start Customizing NVIDIA Nemotron 3 Nano with Prime Intellect Lab in Minutes
Adapting open models to specialized tasks often requires significant infrastructure, technical expertise, and GPU resources. This tutorial demonstrates how developers can customize NVIDIA Nemotron 3 Nano using Prime Intellect Lab for hosted reinforcement learning with verifiable rewards. The workflow establishes a baseline evaluation on a Python Math task, trains a LoRA adapter using a TOML configuration, and reevaluates performance. Across 32 held-out problems capped at five assistant turns, the model improves its accuracy from 21.9% to 90.6% for less than five dollars in compute spend. The same hosted training approach applies to larger models in the family, including Nemotron 3 Super and Nemotron 3 Ultra.
Why A Frontier Data Agent Outperforms General Coding Agents in Quality and Cost
An evaluation compared Genie Code with three widely used coding agents from major AI labs on 401 real internal data tasks, using a shared 20-minute wall-clock budget and independent grading. Genie Code achieved 76.6% accuracy at an estimated $0.55 per task, outperforming rivals at 55.9–72.1% accuracy and $0.91–$1.16 per task. Its advantage is attributed to semantic search across catalog and workspace assets, persistent memory of tables and business logic, and enterprise-context understanding, which reduce exploratory tool use. On discovery-heavy tasks, general-purpose agents often wandered through large workspaces or timed out on inefficient scans, while Genie Code averaged 8.3 tool calls per task. The benchmark found no accuracy-cost trade-off, although the authors note that Genie Ontology was disabled and expect it to strengthen the advantage.
Connect Amazon S3 data to Databricks with Delegated IAM Permissions
Databricks introduces a simpler flow for connecting Amazon S3 to Unity Catalog through an external location, addressing the manual work previously required to establish governed read and write access. Instead of writing 140-line IAM trust policies, configuring bucket permissions, deploying CloudFormation templates, and switching between consoles, users specify a bucket and access level, then verify permissions in AWS. AWS IAM temporary delegation lets Databricks provision a least-privilege IAM role, configure its cross-account trust policy, register the external location, and enable Auto Loader and File Events. The delegated authorization is time-bounded and expires after setup; provisioning actions are logged in AWS CloudTrail, and users without sufficient permissions can request access from an AWS administrator in the flow. The result is a single-session setup intended to reduce configuration errors while supporting governed S3 access for ingestion, pipelines, analytics, and LTAP workloads.
How we built the new Table of Contents feature
Medium built a new Table of Contents feature after an existing Chrome extension demonstrated demand, while its approach of inserting a static list into post content created revision, maintenance, navigation, and reading-experience problems. The shipped implementation derives a nested list from the rendered post body at runtime: H2 and H3 become top-level entries, H4 headings nest beneath the latest top-level heading, and lead title, subtitle, and kicker are excluded. It appears as a right-margin indicator that scrolls with the reader, highlights the current heading, supports smooth scrolling, and uses a sliding window for posts with many headings. Full-width images receive a translucent background, one-heading posts hide the feature, and mobile needs a separate solution because desktop indicators overlap content; the feature took a couple of days, mostly for UX testing and design iteration.
Introducing AI spend controls with Unity AI Gateway
Unity AI Gateway now offers AI Spend Controls, extending existing cost visibility with proactive budget alerts across models and workloads. The feature supports budgets at user, use-case, workspace, and account levels, plus shared and per-user thresholds that can trigger email alerts or enforce hard caps by stopping requests after a limit is exceeded. Configuration starts in account settings under Usage and Budgets, where administrators select Unity AI Gateway, optionally scope workspaces and resource tags, and define monthly limits and recipients. Budget status and trends are available in the Cost section, while customizable Cost Analytics dashboards use Unity Catalog system tables to attribute DBU and model-provider costs by identity, workspace, endpoint, tags, model, provider, and request tags. The release positions Databricks budgets, Unity AI Gateway, and Unity Catalog as a combined governance layer for controlling AI access, usage, and spend.
From 57 bugs to 1, thanks to Seer
Software teams frequently defer low-priority bugs to the backlog because manual triage, investigation, and code fixes consume valuable engineering time. To resolve 56 out of 57 backlog bugs, Dan Mindru utilized Sentry telemetry combined with Seer and the hosted Sentry Model Context Protocol server. Sentry captures context including traces, logs, and session replays, allowing Seer to assess fixability scores and generate pull requests with unit tests for isolated issues. For more intricate problems, developers query production breadcrumbs and telemetry directly in their agentic editors while supplying missing historical business logic. Consequently, automated tool calling and agent workflows drastically lowered the economic cost of addressing routine production bugs.
Observability for every Supabase project with Grafana Cloud
Supabase now offers a one-click Grafana Cloud integration from its Dashboard, with the setup available on every plan, including the free plan. The setup provisions authentication, metric scraping, and a pre-built observability dashboard that tracks more than 200 metrics through Supabase’s Metrics API. Those metrics cover CPU and memory pressure, disk I/O, network traffic, connections and pooler behavior, replication lag, checkpoints, database size, and query statistics, while the same open-source dashboard definitions support self-hosted Grafana, the existing Grafana Cloud integration, and the new setup. The release is focused on metrics and is designed to accommodate logs without architectural changes; support for log drains to Grafana Cloud is already in progress. Users can start with Grafana Cloud’s free tier without a credit card on either side.
Introducing Claude Opus 4.7
Claude Opus 4.7 is generally available as an upgrade to Opus 4.6, with stated gains in advanced software engineering, difficult long-running tasks, instruction following, self-verification, and higher-resolution vision. The announcement reports benchmark and tester results including a 13% resolution increase on a 93-task coding benchmark, a 0.715 tie for top score across six research-agent modules, and 90.9% on BigLaw Bench at high effort. It also describes safeguards that detect and block prohibited or high-risk cybersecurity requests, while legitimate security professionals can apply to the Cyber Verification Program. The model is offered across Claude products, its API, Amazon Bedrock, Vertex AI, and Microsoft Foundry at $5 per million input tokens and $25 per million output tokens. New controls include xhigh effort, task budgets, Claude Code’s /ultrareview, and auto mode, while migration may increase token counts because of tokenizer and reasoning changes.
Introducing Claude Opus 4.5
Claude Opus 4.5 is available today across Claude apps, the API, and all three major cloud platforms, with the model identifier claude-opus-4-5-20251101 and pricing of $5/$25 per million tokens. The announcement positions it as a state-of-the-art model for coding, agents, computer use, deep research, spreadsheets, and slides, while reporting stronger vision, reasoning, and mathematics than predecessors. On the Claude API, an effort parameter lets developers trade off speed and spend against capability; at medium effort, Opus 4.5 matched Sonnet 4.5’s best SWE-bench Verified score using 76% fewer output tokens, while highest effort exceeded it by 4.3 percentage points using 48% fewer tokens. Additional platform changes include context compaction, advanced tool use, memory, and improved multi-agent coordination. Product updates add Claude Code’s editable Plan Mode and desktop sessions, automatic conversation summarization, and expanded Claude for Chrome and Excel access.
The production platform for open-weight AI inference
The update presents a production inference platform for open-weight, licensed closed-weight, and fine-tuned models, aimed at giving teams control over performance, cost, quality, and functionality without building an inference stack from scratch. It supports uploading weights or adapters from Hugging Face, S3, or local machines, uses tested deployment profiles for hardware, quantization, tensor parallelism, speculative decoding, and serving engine choices, and lets operators tune placement and autoscaling signals. Endpoints can keep a stable interface while multiple deployments receive traffic through canary, blue-green, rolling, A/B, or shadow testing, with threshold-based automatic rollback. The rebuilt caching and distribution layer proactively warms shared weights; internal testing reports roughly 4× faster warm starts across several frontier models, while a closed beta adds full-weight and LoRA reinforcement learning and supervised fine-tuning with deployable checkpoints.
Bringing Nunchaku 4-bit Diffusion Inference to Diffusers
Modern diffusion transformers running in BF16 often require 20–30 GB of VRAM, limiting access on consumer GPUs; the integration brings Nunchaku Lite checkpoints into Diffusers to reduce memory and denoising latency. SVDQuant handles activation outliers by moving them into weights, preserving a small 16-bit low-rank branch while quantizing residual weights to 4-bit, enabling W4A4 compute in transformer attention and MLP projections. Nunchaku Lite patches standard nn.Linear modules at runtime, loads Hub-hosted CUDA kernels through the kernels package, and exposes ordinary from_pretrained() loading without local compilation; AWQ W4A16 handles modulation and normalization projections. On an RTX 5090, the cited NVFP4 checkpoint generates 1024×1024 images in about 1.7 seconds at roughly 12 GB peak memory versus about 24 GB for BF16, while the bare-bones Nunchaku Lite path delivers around 30% speedup; torch.compile raises reported end-to-end speedup from 1.35x to 1.8x.
Simplify AI agent orchestration with Lakebase Postgres
CLA and Databricks built a production document-processing application for auditing that reduces extraction time from hours to minutes without compromising quality, using Databricks-native services including Lakebase Postgres, Databricks Apps, Lakeflow Jobs, MLflow, and Unity Catalog Volumes. Lakebase serves as the orchestration layer’s single source of truth for tasks and execution attempts, coordinating long-running work, retries, leases, priorities, rate limits, costs, and status visibility. The queue uses Postgres patterns including FOR UPDATE SKIP LOCKED, priority and FIFO ordering, expiring leases for crash recovery, and database-backed concurrency controls. Databricks Jobs process PDFs through intelligent document processing and vision/LLM calls, while MLflow Tracing records execution and cost details and dashboard updates combine fast Postgres data with slower billing queries. In production, this architecture avoids external brokers and schedulers while providing durable task management, real-time visibility, and per-task cost attribution.
Treating Pricing Changes Like Code Deploys
Thumbtack manages marketplace pricing by updating hundreds of thousands of bid settings stored in DynamoDB across service offerings. Previously executed through individual custom Go scripts, these pricing rollouts lacked automated runtime state tracking, dynamic database throughput pacing, and durable records for reliable rollbacks. To eliminate manual operational risk, engineering introduced a shared deployment-like engine that accepts declarative target states and executes pure Go transformation functions without direct I/O. The engine enforces idempotency, conducts dry runs with validation reports to S3, coordinates atomic DynamoDB updates with audit logging, and dynamically manages write capacity. This platform standardizes pricing updates into reviewable, resumable, and reversible operations that safely skip already written rows and check for production drift during rollbacks.
Building a serverless AI assistant at Pelago: concept to care in two weeks
Pelago engineered a serverless, event-driven AI assistant on AWS within two weeks to provide substance use recovery coaches with contextual response suggestions. Operating under strict Protected Health Information constraints, the system maintains clinical safety through human-in-the-loop review and keeps data confined to an Amazon VPC. Incoming messages flow via AWS AppSync and Amazon DynamoDB to an Amazon SNS topic, which fans out to independent AWS Lambda functions. The asynchronous Chat Assistant Lambda queries history, invokes Amazon Bedrock, and persists suggestions into Amazon RDS MySQL in under 10 seconds. When coaches access a conversation via Amazon API Gateway, pre-generated suggestions load in under 100 milliseconds.
Apache Arrow Cut Snowflake Fetch Memory Growth by Up to 79%
An ML training run was exhausting worker memory before training could start because a cold Snowflake fetch peaked at 88.8 GiB RSS, versus 17.6 GiB when loading local Parquet. The shared Snowflake-to-pandas path created Python objects and SQLAlchemy Row proxies before pandas built the DataFrame, with temporary representations requiring roughly three times the final DataFrame size at peak. An Apache Arrow path for supported reads reduced median peak process RSS growth by 60–79% across three query shapes, while peak Python allocations fell by 69–87%. The narrow and wide tests also ran faster, but aggregate timing was mixed, so elapsed time was not the headline claim. Existing jobs adopted the default without query rewrites, with normalization preserving the pandas contract and SQLAlchemy retained for unsupported reads; in one workflow, lower fetch peaks doubled the training-data window on the same cluster size.
Make Long-Running NVIDIA TensorRT Engine Builds Observable and Cancelable in Python or C++
Long-running NVIDIA TensorRT engine builds can freeze workflows and waste GPU hours when they lack visibility or abort mechanisms. Developers can resolve this by subclassing the IProgressMonitor API in Python or C++ and attaching it to the builder configuration. The monitor tracks hierarchical build phases through start, step completion, and finish callbacks, requiring thread synchronization because TensorRT calls monitor methods across internal builder threads. Returning false from the step completion callback requests early build cancellation, which unwinds active phases in reverse order at the next step boundary. Beyond terminal rendering with ANSI escape codes, teams can stream these progress events to IDE protocols, HTTP service endpoints, or agent tool-call traces.
Securing Infrastructure at Scale: Introducing Pinterest’s Resource Provisioner Pipeline (RPP)
Managing Infrastructure as Code across distributed repositories presents significant security and operational challenges. Pinterest developed the Resource Provisioner Pipeline (RPP), a proprietary Terraform execution engine, to securely deploy and manage AWS infrastructure across hundreds of workspaces. The pipeline utilizes centralized GitHub composite actions triggered by pull request events and enforces dual human approvals. To maintain least privilege, RPP uses GitHub OpenID Connect authentication to assume a central identity before validating repository paths and down-scoping to specific team execution roles. Automated checks include backend state validation, linting, plan generation, Semgrep static analysis, and optional LocalStack testing before deliberate comment-driven deployments execute.
Building multi-Region resiliency for AWS CloudFormation custom resource deployment
AWS CloudFormation custom resources extend template provisioning capabilities but lack built-in multi-Region coordination, risking duplicate executions and unhandled failures during regional outages. To solve this limitation, an active-active architecture leverages Amazon Simple Notification Service topics to fan out lifecycle events across primary and secondary infrastructure Regions. The primary Region processes events immediately, whereas the secondary Region processes incoming messages after a deliberate delay implemented via Amazon Simple Queue Service. Coordination relies on Amazon DynamoDB Global Tables with conditional writes to manage distributed locking and enforce idempotency. When failures occur in the primary Region, Amazon CloudWatch alarms signal Amazon Application Recovery Controller to automate failover without manual intervention.
Architecting offline-first generative AI applications for edge deployments using AWS services
Industrial edge environments often face significant unplanned downtime yet lack reliable cloud connectivity to run large-scale artificial intelligence models. To address this limitation, an offline-first reference architecture shifts generative inference to edge devices while leveraging cloud services for preparation and lifecycle management. The workflow uses Amazon Bedrock with Amazon Nova Pro to create labeled dataset pairs, Amazon SageMaker AI Pipelines to fine-tune a compact model, and AWS IoT Greengrass to orchestrate edge deployments. Local execution relies on Ollama, Strands Agents, and a CPU-based ChromaDB retrieval pipeline that consumes zero GPU VRAM. In evaluations across three foundation model judges, the fine-tuned gpt-oss-20b model combined with retrieval-augmented generation consistently outperformed the base model configuration across accuracy, completeness, and relevance metrics.
Automate custom PII detection at scale with Amazon Macie and Step Functions
Regulated organizations ingesting large data volumes must detect and classify standard and domain-specific personally identifiable information to satisfy compliance mandates like GDPR, HIPAA, CCPA, and PCI DSS. To eliminate manual inspection, an event-driven architecture orchestrates Amazon Macie scans for files landing in Amazon S3. Amazon EventBridge captures upload events and invokes AWS Step Functions, which manages Lambda functions to stage objects, trigger Macie classification jobs with custom regex identifiers, and poll for results. The pipeline separates data across a three-bucket pattern representing raw, staged, and scanned states to prevent mixing unprocessed data with validated assets. Finally, the workflow exports timestamped compliance reports in CSV and JSON formats while alerting security teams via Amazon SNS on high-severity findings.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.