Reinforcement learning controller cuts power spikes in LLM training tests

A new arXiv paper examines the power behavior of reinforcement-learning post-training for language models and tests whether workload-aware control can reduce energy problems.

The researchers instrumented GRPO training at 7B, 14B, and 72B model scales on A100 GPUs, collecting more than 380,000 half-second power samples. They then trained a PPO meta-controller that adjusts the workload’s generation parameters based on measured power.

In a 7B trace, the controller reduced power-limit violations by 89.8% while increasing token output by 18.1% and energy efficiency by 26.2%. The same controller family did not transfer cleanly to live 72B training, where sharding reduced the authority of its actuator. A revised controller using generation concurrency showed stronger control in 72B rollout-generation tests.

The result is not a universal datacenter fix. It shows that AI training power management may need to understand the workload itself, not just impose static caps after hardware gets hot or overloaded.