# Promptimus: Improving already good LLM prompts with zero manual engineering

[Amazon](https://yomu.fyi/company/amazon) · Zhengyuan Shen, Yunfei Bai, Sullam Jeoung, Shuai Wang · May 14, 2026

**Type:** Problem & solution

## Summary

Enterprise applications rely on mature large language model prompts that incorporate intricate business logic and regulatory constraints, making further optimization and cross-model migration challenging. Promptimus automates prompt refinement through a four-step iterative loop that evaluates candidate prompts against decomposed metric checkpoints, diagnoses failure bottlenecks, and generates targeted improvement strategies. The system operates either via full prompt rewrites in standard mode or through surgical, programmatic find-and-replace edits in edit mode to preserve structured taxonomies and schemas. Tested across multimodal classification and code generation tasks, the framework achieves performance gains with small development datasets of 20 to 50 samples. Amazon is making the model-agnostic optimization system available through Amazon Bedrock to support enterprise generative-AI model migration without manual engineering.

## Context

Enterprise large language model applications rely on complex prompts encoding domain knowledge and regulatory rules that are difficult to refine without causing regressions. Furthermore, rapid model releases require recurring prompt migrations across providers, where prompts tuned for one model often underperform on another due to differing instruction-following behaviors.

## Approach / What changed

Promptimus executes an automated four-step optimization loop using user-defined Python metric functions, initial prompt templates, and small datasets. A metric analyzer decomposes evaluation functions into intermediate validation checkpoints to pinpoint failure bottlenecks. Depending on prompt size and structure, the system operates in standard mode for full rewrites or edit mode for targeted programmatic find-and-replace modifications across exact, fuzzy, and similarity match levels.

## Takeaways

- Promptimus decomposes user-defined Python metric functions into intermediate validation checkpoints to diagnose specific failure bottlenecks rather than relying on scalar scores.
- For large prompts containing intricate business logic, edit mode applies surgical find-and-replace modifications programmatically with a 97.3% matching success rate without making LLM calls for edit application.
- The framework operates effectively on small datasets of 20 to 50 samples, enabling automated prompt optimization and model migration across targets without requiring ground truth for all samples.

**Tags:** [AWS](https://yomu.fyi/topic/aws), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Python](https://yomu.fyi/topic/python)

[Read original post](https://www.amazon.science/blog/promptimus-improving-already-good-llm-prompts-with-zero-manual-engineering)