DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding

Summary

On the DeepSWE coding benchmark, GPT-5.6 Luna achieves 67.2% pass@1 versus DeepSeek-V4 Flash’s 53.3%, a 14-point gap. DeepSeek costs $0.10 per task, roughly six times less than Luna’s $0.61, delivering 532 solves per $100 versus Luna’s 110. A cascade that runs DeepSeek first and escalates only on failure solves 78.9% of tasks at $0.385 each, beating Luna’s solo accuracy and cutting cost by 37%. DeepSeek fails more gracefully (9% broken tests vs 15%) but is slower and dramatically weaker on JavaScript (35% vs 60%). Luna dominates seven of eight task domains, while DeepSeek leads only on query and config languages, illustrating how inexpensive models can combine with flagships for superior price-performance.

Takeaways

1. 1

   GPT-5.6 Luna leads DeepSWE pass@1 at 67.2% vs 53.3% for DeepSeek-V4 Flash, a 14-point gap sustained at every attempt count.

2. 2

   DeepSeek-V4 Flash costs $0.10 per task versus Luna’s $0.61, delivering 532 solves per $100 against Luna’s 110.

3. 3

   A cascade that runs DeepSeek first and escalates to Luna on failure solves 78.9% of tasks at $0.385 each, beating Luna alone in accuracy and cost.

FAQ