LEGO-RL links coding-agent harnesses to policy training

A new arXiv paper addresses a technical mismatch in training coding agents with reinforcement learning.

Modern coding agents run inside harnesses that manage tools, repository context, execution feedback, and long-running workflows. Those same harnesses can create problems for policy-gradient training: crashes corrupt reward signals, reward hacking distorts outcomes, and rollout behavior may differ from what the trainer sees.

LEGO-RL aims to bridge that gap without rewriting the agent harness. It uses in-process LLM proxying to capture raw generation streams, recomputes log probabilities on the trainer side, and adds sandbox orchestration to make execution more reliable. The goal is to keep training faithful to the environment where the agent will actually be used.

The work is important because coding agents are moving from chat interfaces into tool-heavy development loops. Better training infrastructure could make them more robust, but the paper also shows that agent performance depends on plumbing, logging, and evaluation discipline, not only on the base model.