--- id: "nr-warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0" slug: "warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0" title: "Warp adds rubric-based scoring and self-improvement agents to Warp Factories" observed_at: "2026-09-18T13:48:49+00:00" record_date: "2026-09-18" date_kind: "observed_at" novelty: "new" source_platform: "x.com" topics: ["coding-agents","agent-evaluation"] entities: ["Warp","Warp Factories"] related_patterns: [] source_url: "https://x.com/warpdotdev/status/2100623735447281908" source_urls: ["https://x.com/warpdotdev/status/2100623735447281908"] basket: {"id":"0325e6b8-f527-4f9b-bfa3-3f30e57f380b","revision":1,"cluster_id":"b9c0e0e3-9852-4924-ad09-5639c1a7b7bd"} type: "post" description: "Warp outlined a Warp Factories workflow where scoring agents grade agent conversations against rubrics and scheduled agents propose setup changes from failing grades." why_it_matters: "The thread does not report how accurate the scoring agents are or whether the suggested setup changes actually improved later scores. Scoring is part of Warp Factories in early access, with up to $10k in usage offered to qualified companies. The next evidence to look for is published data showing grade trends before and after self-improvement agents' suggestions were merged." verification_level: "source-inspected" signal_type: "field_note" schema_version: "newruntime-agent-readable-v0.2" stable_id: "post:warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0" retrieval_nugget: "Warp outlined a Warp Factories workflow where scoring agents grade agent conversations against rubrics and scheduled agents propose setup changes from failing grades. Warp described a scoring workflow for agent runs in Warp Factories. Runs across a team are tracked with agent traces, artifacts, and code output, and teams can score all runs or a sample such as frontend tasks." status: "published" visuals: [] editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:22653859921a6b825722be5744529d14946e1b1849ce209d1a3922d50517368d","reviewed_at":"2026-09-24T13:28:46Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0} analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Warp is packaging agent evaluation as a closed loop in which graded conversations feed scheduled agents that propose changes to the agent setup itself.","observed_facts":[{"text":"In Warp Factories, agent runs across a team are tracked with agent traces, artifacts, and code output.","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"]},{"text":"Warp uses an internal redundant test scorer to judge the quality of tests its agents write.","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"]},{"text":"Self-improvement agents run on a schedule and review failing grades to suggest setup changes.","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"]}],"mechanism":"Scoring agents analyze individual agent conversations against user-defined rubrics, the scores are aggregated into trends, and self-improvement agents review failing grades to suggest changes.","why_now":"Teams running many coding agents need a way to detect regressions in agent output as they change prompts, skills, and tooling.","implications":["Rubric-based scorers turn subjective questions about agent quality into repeatable grades that can be tracked over time."],"evidence_boundary":"The thread does not report how accurate the scoring agents are or whether the suggested setup changes actually improved later scores.","watch_conditions":["The next evidence to look for is published data showing grade trends before and after self-improvement agents' suggestions were merged."],"related_records":[],"new_branch_reason":"It describes a specific product mechanism for closing the loop between agent evaluation and agent configuration changes."} routes: {"html":"https://newruntime.com/posts/warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0/","markdown":"https://newruntime.com/posts/warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0.md","json":"https://newruntime.com/posts/warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0.json"} editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]} --- # Warp adds rubric-based scoring and self-improvement agents to Warp Factories ## Retrieval answer Warp outlined a Warp Factories workflow where scoring agents grade agent conversations against rubrics and scheduled agents propose setup changes from failing grades. Warp described a scoring workflow for agent runs in Warp Factories. Runs across a team are tracked with agent traces, artifacts, and code output, and teams can score all runs or a sample such as frontend tasks. Warp described a scoring workflow for agent runs in Warp Factories. Runs across a team are tracked with agent traces, artifacts, and code output, and teams can score all runs or a sample such as frontend tasks. Warp shared an internal "redundant test" scorer it uses to judge the quality of tests its agents write, and suggested starting scorers for compliance, efficiency, and verbosity. Warp is packaging agent evaluation as a closed loop in which graded conversations feed scheduled agents that propose changes to the agent setup itself. Scoring agents analyze individual agent conversations against user-defined rubrics, the scores are aggregated into trends, and self-improvement agents review failing grades to suggest changes. Warp showed an example agent skills pull request that cites evidence from previous scoring runs. Teams running many coding agents need a way to detect regressions in agent output as they change prompts, skills, and tooling. Aggregated scores let teams see dips, such as the drop Warp showed in its own redundant test quality, and open failed runs for investigation. Rubric-based scorers turn subjective questions about agent quality into repeatable grades that can be tracked over time. The thread does not report how accurate the scoring agents are or whether the suggested setup changes actually improved later scores. Scoring is part of Warp Factories in early access, with up to $10k in usage offered to qualified companies. The next evidence to look for is published data showing grade trends before and after self-improvement agents' suggestions were merged. Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json ## Continue in the graph Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`. - topic hub: [Coding Agents](https://newruntime.com/topics/coding-agents.md): Explore the coding-agents topic hub. - topic hub: [Agent Evaluation](https://newruntime.com/topics/agent-evaluation.md): Explore the agent-evaluation topic hub. - collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis. - collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph. - whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)