AWS has introduced evaluation tools for a problem that ordinary answer scoring can miss: an AI agent may sound convincing even when it selected the wrong procedure or ignored required steps. Strands Evals and Amazon Bedrock AgentCore Evaluations can now inspect how agents use modular skills.
A skill bundles instructions, tools, reference material, workflow rules and guardrails for a specific task, such as contract redaction or invoice reconciliation. Agents load the relevant package when needed instead of carrying every business procedure in one large system prompt. That makes procedures easier to update, but creates new routing and compliance risks.
The Skill Selection Accuracy evaluator checks whether each invoked skill matched the task. Skill Instruction Following assigns a five-level rating based on evidence that prescribed steps were completed. Strands Evals also includes a deterministic check that confirms whether a named skill was loaded at all.
The tools work from recorded action trajectories or OpenTelemetry traces, not just the final response. Teams still need representative test cases and must interpret model-based scores carefully, but the added visibility can separate a routing error from a failure inside the selected procedure.