Researchers have released Harbor Adapters, open-source infrastructure that translates more than 80 agent benchmarks into a common evaluation system. The adapters are intended to reduce the custom environment and integration work that makes broad comparisons of tool-using agents difficult.
The team validated ports through code review and parity experiments, then evaluated eight models across 54 benchmarks. Every model ran with a common Terminus-2 harness and with one of three native harnesses, allowing the analysis to separate model capability from some of the effects introduced by the surrounding execution software.
The release also includes Harbor-Index, a smaller collection of 82 difficult tasks drawn from 29 benchmarks. Tasks went through difficulty filtering, AI and human review, and an audit-and-fix loop. No tested model-and-harness combination exceeded a 30 percent pass rate; GPT-5.5 with Codex scored highest at 28.0 percent.
A compact index makes repeated testing more affordable, but it cannot capture every environment or failure mode represented by the full suite. Benchmark adapters can also introduce subtle behavior differences despite parity checks. Publishing the code, results and curation process gives teams a way to audit those choices instead of relying on a single opaque agent score.