§ evals · storyline
DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding
DeepSeek-V4 Flash and GPT-5.6 Luna benchmarked on DeepSWE show Luna leading pass@1 by 14 points and DeepSeek delivering 4.8x the solves per dollar.
We ran 900 DeepSWE rollouts on DeepSeek-V4 Flash and GPT-5.6 Luna. Luna leads pass@1 by 14 points; DeepSeek delivers 4.8x the solves per dollar.
§ sources1 publication · timeline below