{"id":"nr-warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0","slug":"warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0","title":"Warp adds rubric-based scoring and self-improvement agents to Warp Factories","observed_at":"2026-09-18T13:48:49+00:00","record_date":"2026-09-18","date_kind":"observed_at","novelty":"new","source_platform":"x.com","topics":["coding-agents","agent-evaluation"],"entities":["Warp","Warp Factories"],"related_patterns":[],"source_url":"https://x.com/warpdotdev/status/2100623735447281908","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"],"basket":{"id":"0325e6b8-f527-4f9b-bfa3-3f30e57f380b","revision":1,"cluster_id":"b9c0e0e3-9852-4924-ad09-5639c1a7b7bd"},"type":"post","description":"Warp outlined a Warp Factories workflow where scoring agents grade agent conversations against rubrics and scheduled agents propose setup changes from failing grades.","why_it_matters":"The thread does not report how accurate the scoring agents are or whether the suggested setup changes actually improved later scores. Scoring is part of Warp Factories in early access, with up to $10k in usage offered to qualified companies. The next evidence to look for is published data showing grade trends before and after self-improvement agents' suggestions were merged.","verification_level":"source-inspected","signal_type":"field_note","schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0","retrieval_nugget":"Warp outlined a Warp Factories workflow where scoring agents grade agent conversations against rubrics and scheduled agents propose setup changes from failing grades. Warp described a scoring workflow for agent runs in Warp Factories. Runs across a team are tracked with agent traces, artifacts, and code output, and teams can score all runs or a sample such as frontend tasks.","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:22653859921a6b825722be5744529d14946e1b1849ce209d1a3922d50517368d","reviewed_at":"2026-09-24T13:28:46Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Warp is packaging agent evaluation as a closed loop in which graded conversations feed scheduled agents that propose changes to the agent setup itself.","observed_facts":[{"text":"In Warp Factories, agent runs across a team are tracked with agent traces, artifacts, and code output.","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"]},{"text":"Warp uses an internal redundant test scorer to judge the quality of tests its agents write.","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"]},{"text":"Self-improvement agents run on a schedule and review failing grades to suggest setup changes.","source_urls":["https://x.com/warpdotdev/status/2100623735447281908"]}],"mechanism":"Scoring agents analyze individual agent conversations against user-defined rubrics, the scores are aggregated into trends, and self-improvement agents review failing grades to suggest changes.","why_now":"Teams running many coding agents need a way to detect regressions in agent output as they change prompts, skills, and tooling.","implications":["Rubric-based scorers turn subjective questions about agent quality into repeatable grades that can be tracked over time."],"evidence_boundary":"The thread does not report how accurate the scoring agents are or whether the suggested setup changes actually improved later scores.","watch_conditions":["The next evidence to look for is published data showing grade trends before and after self-improvement agents' suggestions were merged."],"related_records":[],"new_branch_reason":"It describes a specific product mechanism for closing the loop between agent evaluation and agent configuration changes."},"routes":{"html":"https://newruntime.com/posts/warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0/","markdown":"https://newruntime.com/posts/warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0.md","json":"https://newruntime.com/posts/warp-adds-rubric-based-scoring-and-self-improvement-agents-b9c0e0.json"},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"next_reads":[{"type":"topic","path":"/topics/coding-agents/","url":"https://newruntime.com/topics/coding-agents/","title":"Coding Agents","reason":"Explore the coding-agents topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/coding-agents.md","json_url":"https://newruntime.com/topics/coding-agents.json"},{"type":"topic","path":"/topics/agent-evaluation/","url":"https://newruntime.com/topics/agent-evaluation/","title":"Agent Evaluation","reason":"Explore the agent-evaluation topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/agent-evaluation.md","json_url":"https://newruntime.com/topics/agent-evaluation.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}