Warp described a scoring workflow for agent runs in Warp Factories. Runs across a team are tracked with agent traces, artifacts, and code output, and teams can score all runs or a sample such as frontend tasks. Warp shared an internal "redundant test" scorer it uses to judge the quality of tests its agents write, and suggested starting scorers for compliance, efficiency, and verbosity.
Warp is packaging agent evaluation as a closed loop in which graded conversations feed scheduled agents that propose changes to the agent setup itself. Scoring agents analyze individual agent conversations against user-defined rubrics, the scores are aggregated into trends, and self-improvement agents review failing grades to suggest changes. Warp showed an example agent skills pull request that cites evidence from previous scoring runs.
Teams running many coding agents need a way to detect regressions in agent output as they change prompts, skills, and tooling. Aggregated scores let teams see dips, such as the drop Warp showed in its own redundant test quality, and open failed runs for investigation. Rubric-based scorers turn subjective questions about agent quality into repeatable grades that can be tracked over time.
The thread does not report how accurate the scoring agents are or whether the suggested setup changes actually improved later scores. Scoring is part of Warp Factories in early access, with up to $10k in usage offered to qualified companies. The next evidence to look for is published data showing grade trends before and after self-improvement agents' suggestions were merged.