Works on my machine: how we use AI to reproduce reported bugs

Summary

Sentry maintains 159 open-source SDK packages across multiple languages, leading to high triage volume and significant boilerplate setup overhead when reproducing reported bugs. To streamline this workflow, the team developed a Claude skill that parses GitHub issue URLs to extract language, framework, and SDK versions. The skill generates a reproduction branch, sets up minimal boilerplate using ecosystem package managers like uv or npm, attempts execution, and opens a pull request with explicit run instructions. By allowing the model to exit early when encountering overly complex environments, the tool reliably prepares reproducible playgrounds without stumbling on edge cases. This workflow saved several hours in testing scenarios like Django HTTPS proxy bugs while avoiding excessive bot noise through scoped human interaction.

Context

Sentry maintains 159 published open-source SDK packages across various languages and frameworks. Triage and bug reproduction often stall due to tedious boilerplate setup, including configuring exact language versions, framework releases, legacy libraries, and complex environments like HTTPS proxies. Additionally, engineers lacked a maintained central repository for one-off reproduction applications.

Approach / What changed

Sentry built an iterative Claude skill that accepts a GitHub issue URL, parses version metadata, creates a targeted branch, and attempts a minimal reproduction using standard package tools such as uv, npm, or bundle. The skill uses the GitHub CLI to create PRs with execution instructions, leverages Claude tools like AskUserQuestion for backlinking, and includes an explicit bailout instruction if reproduction proves too complex.

Takeaways

• Providing an explicit early exit option in LLM prompts prevents agents from stumbling through overly complex tasks and encourages clear explanations of reproduction challenges.
• Structuring Claude skill Markdown files into discrete steps, integrating CLI tools like gh, and defining explicit error-handling boundaries increases agent reliability.
• SDK teams avoided full automated triage execution on GitHub issues to prevent excessive bot noise and retain necessary human oversight.

Related reading

• Sentry · May 7, 2026

  Fixing JavaScript observability, one library at a time

  JavaScript application performance monitoring tools rely on monkey-patching via require-in-the-middle and import-in-the-middle, which breaks with ECMAScript Modules, bundlers, and non-Node runtimes. To solve this, an initiative is replacing monkey-patching with Node's built-in diagnostics_channel TracingChannel API across server-side JavaScript libraries. Under this pattern, libraries publish structured events while monitoring vendors subscribe to them with zero overhead when unlistened. The author leveraged Claude Code workflows alongside direct maintainer communication to draft proposals, implement code, and manage reviews across 44 target packages. Ten libraries including mysql2, node-redis, ioredis, and unjs modules have merged support, while active efforts continue on shared OpenTelemetry mapper registries and remaining ecosystem packages.

  Abdelrahman Awad

• Sentry · Jul 9, 2026

  When and what should I be logging?