Loading…
Topic
64 posts about Model Evaluation. Every summary links to the original.
See the idea
A benchmark can support two different verdicts: pass@1 on the first attempt, or pass@k with retries measured as cost per solved task. This comparison shows which reading matches your workload.
Compare assistive coding tools with agentic development environments to see where human control, automation, verification, and evaluation belong in the workflow.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Swiggy’s pLTV effort addresses early-stage lifetime-value prediction for Food Delivery and Instamart, where sparse signals, zero-inflated outcomes, long tails, and trial-driven acquisition make conversion models unsuitable. It encodes 350+ pre-order signals across 12 groups in a simple multilayer perceptron with three shared hidden layers, task-specific heads, a custom MAE loss, and Adam optimization. A four-head variant added order-count predictions for each vertical, producing marginally better performance with 135K rather than 363K parameters. Evaluation emphasized ranking through Spearman correlation and decile diagonal coverage, reaching above 0.75 Spearman and 70–80% diagonal coverage. In production bidding experiments, the in-house model delivered better retention and higher gross order value per acquired user at no incremental platform cost, while a third-party platform acquired more users per dollar.
Anthropic’s Frontier Red Team reports on national-security risks from frontier AI models, drawing on work across four releases over the past year. It finds early-warning signs of rapid progress in dual-use capabilities: Claude reached undergraduate-level performance on some cybersecurity evaluations and exceeded expert baselines in selected biology tasks, while current models remain below thresholds for substantially elevated national-security risk. The team compares model generations on CTF and Cybench tests, realistic roughly 50-host cyber ranges, virology troubleshooting, LabBench, and controlled biosecurity studies, including expert red-teaming. Models still fail autonomously in realistic network environments and make critical mistakes in weaponization planning, although researcher-built tools enabled an attack resembling a known large-scale theft of personally identifiable information; the team is expanding automated evaluations and pursuing government partnerships and stronger safeguards.
Frontier threats red teaming applies adversarial testing to evaluate whether AI models could create national-security risks, using biology as a test project. Anthropic worked with biosecurity experts to define threat models, probe models for more than 150 hours, learn jailbreak techniques, and build quantitative, repeatable evaluations in a secure interface without public-deployment safety monitoring. The project found that frontier models can sometimes produce sophisticated, accurate, detailed biological knowledge, with capability increasing alongside model size and potentially with tool access; the authors assess unmitigated risks as possibly near-term. Training changes and classifier-based filters reduced harmful outputs or access to chained expert information, and these mitigations were deployed in the public-facing frontier model. Future work will test LLM speedups over search engines, examine newer and smaller adapted models, and expand collaboration, disclosure, and independent evaluation.
Claude’s Safeguards team describes a lifecycle approach to reducing misuse while keeping the model useful, combining policy, training, evaluation, real-time enforcement, and monitoring. The team uses a Unified Harm Framework spanning physical, psychological, economic, societal, and individual autonomy harms, and policy vulnerability testing with external experts to shape policies and defenses. Before release, Claude undergoes safety, risk, and bias evaluations, including capability uplift testing in high-risk domains; findings can lead to new detection methods or guardrails. After deployment, prompted or fine-tuned Claude classifiers monitor violations for response steering and account actions, while human review and CSAM hash matching support enforcement. Ongoing analysis uses privacy-preserving topic clusters, hierarchical summarization, and threat intelligence to identify aggregate misuse and novel attacks.
Claude 3.7 Sonnet introduces extended thinking, letting users toggle deeper reasoning and developers set a thinking budget. The same model spends additional computation on sequential reasoning, while agent training supports iterative function calls, computer use, and open-ended tasks. The post examines visible thought processes, noting benefits for trust and research but also limitations around faithfulness, safety, and jailbreak risk; it therefore treats the feature as a research preview. Reported examples include stronger OSWorld performance, playing Pokémon Red through tens of thousands of interactions and progressing through three Gym Leaders, plus logarithmic math gains as thinking tokens increase. Safety work includes encrypted thoughts for some harmful content and computer-use prompt-injection mitigations that raise prevention from 74% to 88%.
The quality-flywheel skill addresses the gap between prompt tweaks that improve a few examples and evidence that an agent improved without regressions. A coding agent orchestrates dataset preparation, inference, trace grading, failure analysis, and optimization, using Google's adaptive AutoRaters or custom metrics while keeping the optimizer and evaluator independent. In a travel-concierge example, the revision_honored rubric found that 21% of cases ignored mid-conversation trip changes, exceeding the stated action threshold. A separate software-bug-assistant cycle raised tool-disclosure responses from 0% to 96% across 15 cases, while production traces and online monitors extend the same loop beyond synthetic scenarios. Human approval remains required, and adaptive scores are treated as directional signals rather than absolute grades.
ParallelKernelBench (PKB) measures whether language models can replace PyTorch + NCCL implementations with fast CUDA kernels for multi-GPU workloads, where communication can account for over 20% of inference latency. Its 87 problems come from real codebases and span tensor, context, data, expert, sequence, and FSDP/ZeRO parallelism, with hardware topology supplied and symmetric memory used for direct GPU communication over NVLink. In zero-shot evaluation, GPT-5.5 solved 28 problems and produced 22 faster-than-baseline solutions; sampling three attempts raised those figures to 36 and 27, but the best fast 1@3 result was only 31%. Models performed best on familiar collective primitives, tensor-parallel GEMMs, and Ulysses-style context parallelism, while failures often involved rank coordination, data partitioning, collective ordering, and underuse of TMA or NVLS.
Proactive coding agents need evaluation beyond task completion because existing public benchmarks such as SWE-Bench do not measure goal-oriented insight policies: deciding what matters, supporting evidence, and whether to interrupt or stay silent. To create preliminary ground truth, researchers clustered 705 bugs and 1,178 CLs from internal Google codebases using temporal proximity and semantic similarity, reverted each codebase to its pre-fix state, and gave the agent up to three exploration rounds. An LLM scored predicted insights from 1 to 5 against clustered bug targets, while success used average top score and Hit@K. With one exploration round, the agent averaged 4.5 out of 5; increasing the budget from two to three rounds raised Hit@5 from 33% to 57%, although the authors describe the findings as preliminary and are expanding the evaluation to public GitHub data and richer context streams.
The experiment compares Kimi K2.7 Code and Claude Fable 5 across 12 landing-page prompts covering B2B SaaS, a rooftop speakeasy, developer tools, and other categories. Kimi cost about 94% less overall, averaging roughly 16 times cheaper than Fable, while producing similar-quality output, especially when given visual references. The team first established a prompt-only baseline, then used a custom design MCP server to provide screenshots, UI elements, and other references directly to multimodal Kimi; this improved hierarchy, typography, composition, readability, loading behavior, and image handling. GPT-5.5 scored screenshots and source code on a 0–100 rubric, and although Fable scored higher, the relatively small quality gap led the authors to consider Kimi’s lower cost a reasonable trade-off for iterative landing-page generation.
Uber describes a company-wide Responsible AI program designed to govern models developed across its many platforms and teams as AI becomes central to products and services. Its five pillars are visibility, explainability, governance, education, and adoption, implemented through a centralized Model Catalog, automated metadata, feature attribution, lifecycle controls, and employee learning resources. Model Cards provide searchable model descriptions, metrics, deployment details, and documentation, while Michelangelo integrates global, local, and deep-learning explanations using PFI, SHAP-based methods, TreeSHAP, and integrated gradients. Governance is shifted left into product and engineering reviews, with completion prompts before deployment, and adoption work brings existing models into scope through lifecycle-aware, technology-agnostic, context-sensitive controls. The program gives engineers and oversight teams shared visibility into model behavior and risk while aiming to support continued AI growth without losing accountability and trust.
Anthropic describes a dialogue and research effort on how perspectives from religious, philosophical, cultural, and other communities can inform the development of frontier AI systems. The initial discussions included scholars, clergy, philosophers, and ethicists from more than 15 religious and cross-cultural groups, with broader engagement planned. The work focuses on moral formation, including how training shapes an AI system’s character, values, and behavior, while recognizing that alignment and deployment raise questions beyond technical work. In one experiment, Claude could call a tool during a task to receive a brief reminder of its ethical commitments; it used the tool before consequential actions, and the setup produced markedly lower rates of misaligned behavior on several internal alignment evaluations. Anthropic says it is still separating the effects of the reminder from pausing to reflect and plans further conversations and results.
Etsy’s search and recommendation systems need product representations that capture distinguishing details—such as materials, craft, and visual style—beyond categories and titles, despite noisy listing data, varied inventory, and latency constraints. The approach fine-tunes the open-source Qwen3-VL-8B on query, engaged-listing, and non-engaged-listing triplets, generating candidate summaries and scoring them with a frozen two-tower dense retriever. Group Relative Policy Optimization (GRPO) uses the query–engaged versus query–non-engaged similarity margin as a joint reward, encouraging grounded summaries that separate alternatives rather than generic keyword stuffing. Human review found specific, accurate details beyond titles, while adding the summaries to Etsy’s Semantic Relevance model improved macro F1 by 8.7% relative to description n-grams. Online production experiments are planned.
Evaluating an AI Agent for customer service requires more than measuring accuracy, resolution rates, containment, or benchmark performance on curated datasets. A credible proof of concept should reproduce real support conditions, including multi-turn, vague, sensitive, multilingual, rephrased, and multi-source queries, while testing recovery, task tracking, and human handoffs. It should also assess the customer experience: natural, on-brand tone, trust, graceful uncertainty, and seamless transitions can differ even when quantitative scores match. Finally, teams should examine whether conversations can be reviewed for gaps, how quickly knowledge and behavior can be updated, and how the vendor supports ongoing improvement. The resulting POC demonstrates not only that the Agent works now, but that it can support long-term operational success after launch.
CX Score was built to give support leaders a defensible view of customer experience across conversations, addressing whether an automated metric can be trusted. The team grounded its criteria in how experienced support professionals assess strong, mixed, and poor interactions, then created a dataset of thousands of conversations reviewed by experts, with two reviewers where possible and disagreement resolution. It operates as a production system using specialized prompt engineering, support-focused model configuration, neutral defaults for ambiguous evidence, and multi-stage evaluation across LLM models. Precision, recall, F1, regression, and bias checks set a required F1 above 0.8, while field testing with real customers examined varied tickets, languages, agent types, and conversation lengths. The stated conclusion is that CX Score is designed to be inspected, explained, and defended rather than treated as a black-box output.
Etsy’s Search Relevance team addresses a limitation of engagement-based search signals: popular listings can attract clicks even when they are not the best match for a query. It introduces semantic relevance, measured through a framework combining human-curated golden labels, an LLM annotator, cascaded models, and production integrations, with categories for relevant, partially relevant, and irrelevant query-listing pairs. The data pipeline uses sampled search-log pairs, dual human annotation, an evolving guideline, few-shot chain-of-thought prompting with o3 in LangGraph, and self-consistency sampling; a Qwen 3 VL 4B teacher then labels data at scale before distilling into a BERT-based two-tower student. The student supports real-time use with less than 10ms additional latency, while the framework evaluates and enhances search relevance alongside engagement signals.
Etsy’s seller-created inventory spans more than 100 million items and thousands of categories, making it difficult to capture attributes such as color, material, or niche properties without global SKUs. Because sellers often provide descriptions, titles, and photos instead of complete structured fields, Etsy uses LLMs to distill unstructured listing data into attributes for search filters, comparisons, and other experiences. The pipeline combines seller data, expert-selected few-shot examples, taxonomy logic, and category rules; parallel prompts are routed through LiteLLM, then responses are validated with Pydantic dataclasses and monitored with tracing, metrics, alerts, and performance checks. Human-reviewed silver labels support evaluation, while target categories increased complete attribute coverage from 31% to 91%, and inferred attributes improved search-filter engagement and overall post-click conversion.
Anthropic describes an ongoing partnership with the US Center for AI Standards and Innovation (CAISI) and UK AI Security Institute (AISI) to test and improve AI safeguards during model development. The organizations evaluated multiple iterations of Constitutional Classifiers on models including Claude Opus 4 and 4.1, using pre-deployment prototypes, varied system configurations, documentation, and classifier scores for red-teaming. Their testing exposed prompt-injection bypasses, universal jailbreaks, cipher-based and input/output obfuscation attacks, and automated attack refinement that produced an effective universal jailbreak. Anthropic patched specific vulnerabilities and restructured safeguard architecture, improved detection and filtering, and used findings to strengthen evaluation, deployment monitoring, and response practices. The collaboration also showed that sustained access, real-time data, daily technical communication, and complementary government, expert, and bug-bounty testing can reveal both common exploits and subtle attack vectors.
Teams evaluating AI solutions can waste time, money, and trust when they lack technical fluency, choose unsuitable vendors, underestimate internal requirements, or expect immediate perfection. The post introduces Retrieval-Augmented Generation (RAG), vector search, agentic AI, and MCP as concepts that help teams distinguish static or rules-based tools from systems that retrieve current information, understand intent, take actions, and operate within governed boundaries. It argues that building an AI Agent in-house requires substantial work across embeddings, retrieval, prompt chaining, governance, security, maintenance, and model upgrades. Vendor evaluation should consider ongoing R&D, product roadmaps, adaptability, and maintenance, while implementation depends on structured content, documentation, process visibility, and APIs. Fin is cited as typically resolving over 76% of support questions out of the box, with resolution expected to improve through ongoing optimization and measurement.
Daniel Botha argues that conventional benchmarks reveal little about how models behave in long-context, interactive situations or whether they deliver the tone a product requires. He presents games as a more observable evaluation setting, because success signals are unambiguous and gameplay exposes strategic reasoning, long-term planning, and adaptation with opponents or collaborators. The post focuses on a fork of a16z-infra’s AI Town, a pixel-art simulation inspired by Generative Agents: Interactive Simulacra of Human Behavior, where characters remember conversations, maintain relationships, react to events, and stay in character. A single deploy script runs the project on Fly Machines with scale to zero, cost and performance optimizations, and support for OpenAI-compatible services, Together.ai models, and custom embedding models configured through secrets.
The post compares Intercom’s Fin AI Agent with Zendesk’s AI agent, focusing on answer quality, complexity, and resolution performance. In a test using 48 fictional bed-and-breakfast help articles and 200 questions, both systems answered the same dataset, while four AI models judged responses and an Elo rating system selected winners. It reports that Fin produced better answers in 80% of cases, answered 96% of multi-source questions versus Zendesk’s 78%, and delivered responses averaging 120 words versus 50. The analysis also found no statistical difference in hallucination levels and estimated that Fin was 66% more likely to provide a complete resolution when both agents answered. The post notes limitations including simulated data, AI judging, and differing product features, then describes Fin’s integrations and 99¢-per-resolution pricing.
The best engineering reads, once a week.
