The post argues that agent evaluations should inspect tool-call trajectories rather than score only a final natural-language response.
In an initial 11-prompt suite, Google says an automated grader marked three prompts as failures, while raw logs showed the plugin had completed the required actions. The company also says telemetry must preserve assistant-turn structure: flattening concurrent tool calls can obscure cases where searches run before a skill's injected guidance reaches the context.
Google recommends running live, sandboxed-credential tests beside plan-only runs with credentials removed. It says roughly half of its most important findings came from the difference between those conditions: live runs exposed execution and permission behavior, while plan-only runs exposed planning and routing. The post further recommends expert-curated golden sets as regression baselines for the combined model, harness, and tools.
This is a Google Cloud engineering account of one plugin's evaluation process. It does not establish comparative performance metrics, independently validate the reported findings, or show that the practices will produce the same results for other agent systems. Source: https://cloud.google.com/blog/topics/developers-practitioners/four-eval-engineering-lessons