Changing the software harness around a coding model often affected cost more than success rate in a controlled study of Claude Code, OpenCode and mini-SWE-agent. A harness supplies the system prompt, tools and context management that let a model work inside a repository.
Researchers at Shandong University ran five models on SWE-bench Verified while holding the model constant. On a 447-task pool, the heaviest and lightest harnesses finished less than two percentage points apart for both tested models. Across repeated runs, outcomes changed on a median 13% of tasks; switching harnesses produced the same median flip rate, while switching models raised it to 22%.
OpenCode was the exception, trailing by as much as nine points. The study attributed roughly half that deficit to how it handled output limits, while shorter time limits and broken offline search behavior added uncontrolled differences. Those implementation details caution against interpreting every gap as a superior agent design.
Costs varied by up to threefold because system prompts and tool descriptions are resent at each step. The researchers estimated about 16,600 baseline tokens for Claude Code, roughly 7,000 for OpenCode and fewer than 1,000 for mini-SWE-agent. The work mainly uses one benchmark, so its conclusion is narrower than “harnesses never matter.” It recommends publishing reruns to separate real gains from ordinary variance.