{"type":"post","id":"existing-evidence-shows-that-ai-can-perform-software-14555e","slug":"existing-evidence-shows-that-ai-can-perform-software-14555e","title":"Existing evidence shows that AI can perform software engineering tasks relevant to AI research , dataset creation , open-ended optimization of defined metrics ( autoresearch ), and more .","description":"Task-completion horizons measure the difficulty of benchmark tasks an agent may solve at a given reliability, not how long it can run autonomously.","retrieval_nugget":"Task-completion horizons measure the difficulty of benchmark tasks an agent may solve at a given reliability, not how long it can run autonomously.","published_at":"2026-10-10","updated_at":"2026-10-10","record_date":"2026-10-10","date_kind":"published_at","topics":[],"source_urls":["https://metr.org/time-horizons"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:existing-evidence-shows-that-ai-can-perform-software-14555e","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:812adfd7371fbbb0d03580585f7a89d32fc23781e889c27e3abfb851f243fc06","reviewed_at":"2026-10-10T09:04:10.120Z","source_evidence_count":1,"verified_claim_count":2,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":2,"implication_count":1,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Task-completion horizons measure the difficulty of benchmark tasks an agent may solve at a given reliability, not how long it can run autonomously.","observed_facts":[{"text":"METR defines the 50% horizon as the human-expert task duration at which a fitted estimate predicts success half the time.","source_urls":["https://metr.org/time-horizons"]},{"text":"The page reports a predicted 50% horizon of 17 hours for GPT-4 and cautions that measurements above 16 hours are unreliable with the current task suite.","source_urls":["https://metr.org/time-horizons"]}],"mechanism":"METR fits a logistic curve to predicted success against human-expert task duration, then reads the duration at the specified success probability.","why_now":"The public page presents a current comparison of frontier agents alongside the metric definition and caveats, making it usable as bounded capability evidence.","implications":["Builders can use the measure to reason about delegation of bounded tasks, but should not read it as a project-autonomy estimate."],"evidence_boundary":"The evidence covers METR’s largely self-contained software engineering, machine learning, and cybersecurity task suite; it does not establish performance on context-rich professional work or autonomous research end to end.","watch_conditions":["Watch whether future evaluations broaden task coverage and report reliable measurements beyond the current upper range."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied for this measurement page, so this is a distinct analytical record."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/existing-evidence-shows-that-ai-can-perform-software-14555e/","markdown":"https://newruntime.com/posts/existing-evidence-shows-that-ai-can-perform-software-14555e.md","json":"https://newruntime.com/posts/existing-evidence-shows-that-ai-can-perform-software-14555e.json"},"next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}