AWS has published a reference deployment for routing coding-agent requests between models through Nvidia NeMo Switchyard, LiteLLM and Amazon Bedrock. Its central metric is cost per solved task: total spending across attempts divided by the number of runs whose tests pass.
That measure captures costs hidden by token pricing. Coding agents may need different numbers of turns, repeatedly send conversation history, retry failed work or never finish. A cheaper model per token can therefore cost more for a completed change.
The sample deploys LiteLLM and Switchyard on Amazon ECS with Fargate. Switchyard’s stage strategy reads recent tool activity and chooses between one capable model and one lower-cost model. A random strategy supports weighted traffic splits and comparisons. Neither uses an additional judge model. Bedrock calls stay on AWS networking through a private endpoint, while a load balancer, AWS WAF and a restricted path list limit access.
AWS labels the Switchyard integration experimental and the sample non-production. The walkthrough uses published component versions whose configuration may change, and it requires teams to test security, compliance and model availability in their own region. The useful contribution is the evaluation method: routing should be judged on whether an entire agent task succeeds, not on the apparent price of an isolated request.