# Works on my machine: how we use AI to reproduce reported bugs

[Sentry](https://yomu.fyi/company/sentry) · Neel Shah · Jun 8, 2026

**Type:** Problem & solution

## Summary

Sentry maintains 159 open-source SDK packages across multiple languages, leading to high triage volume and significant boilerplate setup overhead when reproducing reported bugs. To streamline this workflow, the team developed a Claude skill that parses GitHub issue URLs to extract language, framework, and SDK versions. The skill generates a reproduction branch, sets up minimal boilerplate using ecosystem package managers like uv or npm, attempts execution, and opens a pull request with explicit run instructions. By allowing the model to exit early when encountering overly complex environments, the tool reliably prepares reproducible playgrounds without stumbling on edge cases. This workflow saved several hours in testing scenarios like Django HTTPS proxy bugs while avoiding excessive bot noise through scoped human interaction.

## Context

Sentry maintains 159 published open-source SDK packages across various languages and frameworks. Triage and bug reproduction often stall due to tedious boilerplate setup, including configuring exact language versions, framework releases, legacy libraries, and complex environments like HTTPS proxies. Additionally, engineers lacked a maintained central repository for one-off reproduction applications.

## Approach / What changed

Sentry built an iterative Claude skill that accepts a GitHub issue URL, parses version metadata, creates a targeted branch, and attempts a minimal reproduction using standard package tools such as uv, npm, or bundle. The skill uses the GitHub CLI to create PRs with execution instructions, leverages Claude tools like AskUserQuestion for backlinking, and includes an explicit bailout instruction if reproduction proves too complex.

## Takeaways

- Providing an explicit early exit option in LLM prompts prevents agents from stumbling through overly complex tasks and encourages clear explanations of reproduction challenges.
- Structuring Claude skill Markdown files into discrete steps, integrating CLI tools like gh, and defining explicit error-handling boundaries increases agent reliability.
- SDK teams avoided full automated triage execution on GitHub issues to prevent excessive bot noise and retain necessary human oversight.

**Tags:** [Developer Experience](https://yomu.fyi/topic/developer-experience), [LLMs](https://yomu.fyi/topic/llm), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python), [Testing](https://yomu.fyi/topic/testing)

[Read original post](https://blog.sentry.io/ai-bug-reproduction)