---
type: "post"
id: "open-models-have-crossed-a-threshold-analysis-8bf2ef"
slug: "open-models-have-crossed-a-threshold-analysis-8bf2ef"
title: "Open Models have crossed a threshold"
description: "**Thesis:** LangChain’s early results suggest open-weight models are becoming viable candidates for defined agent workloads, but parity remains task- and harness-specific."
retrieval_nugget: "**Thesis:** LangChain’s early results suggest open-weight models are becoming viable candidates for defined agent workloads, but parity remains task- and harness-specific."
published_at: "2026-10-13"
updated_at: "2026-10-13"
record_date: "2026-10-13"
date_kind: "published_at"
topics: ["open-models","claude"]
source_urls: ["https://www.langchain.com/blog/open-models-have-crossed-a-threshold"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:open-models-have-crossed-a-threshold-analysis-8bf2ef"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:69f54ee9fac3c40adb454296eaa5d70e73bde49f6246097fe7333302e56c6fdd","reviewed_at":"2026-10-13T17:07:38.187Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"LangChain’s early results suggest open-weight models are becoming viable candidates for defined agent workloads, but parity remains task- and harness-specific.","observed_facts":[{"text":"LangChain reports that GLM-5 solved 94 of 138 cases (64%) and MiniMax M2.7 solved 85 (57%) in its Deep Agents evaluation.","source_urls":["https://www.langchain.com/blog/open-models-have-crossed-a-threshold"]},{"text":"In the same table, Claude Opus 4.6 scored 100 of 138 (68%), while GPT-5.4 scored 91 (61%).","source_urls":["https://www.langchain.com/blog/open-models-have-crossed-a-threshold"]},{"text":"The reported suite includes file operations, tool use, retrieval, conversation, memory, summarization, and unit tests.","source_urls":["https://www.langchain.com/blog/open-models-have-crossed-a-threshold"]}],"mechanism":"The mechanism is a shared harness and grading scheme: models face the same task cases and success assertions, while the evaluation tracks correctness alongside solve rate and efficiency measures. That makes model choice testable against an agent workflow rather than inferred from broad benchmark reputation. It does not eliminate provider, inference, or harness effects.","why_now":"LangChain says the results are early and that its evaluation set is being maintained and expanded. Its post also describes the suite running across 52 models, giving teams a practical route to compare candidates as open-weight options improve.","implications":["Builders can treat open models as candidates for workload-specific routing when their own evaluation shows acceptable task quality and operating cost.","Published pricing and latency comparisons can guide which candidates to test, but do not replace workload-specific measurement."],"evidence_boundary":"This is LangChain’s own early evaluation of its Deep Agents harness, using stated provider configurations and a finite task set; it does not establish general parity across applications or independently reproduce production outcomes.","watch_conditions":["Watch whether the maintained evaluation results remain comparable as the task set grows and whether teams report similar quality and cost trade-offs on their own production workflows."],"related_records":[],"new_branch_reason":"This source supplies a specific, side-by-side agent-harness evaluation and cost framing; no prior New Runtime record was supplied for this event."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/open-models-have-crossed-a-threshold-analysis-8bf2ef/","markdown":"https://newruntime.com/posts/open-models-have-crossed-a-threshold-analysis-8bf2ef.md","json":"https://newruntime.com/posts/open-models-have-crossed-a-threshold-analysis-8bf2ef.json"}
---

# Open Models have crossed a threshold

## Retrieval answer

**Thesis:** LangChain’s early results suggest open-weight models are becoming viable candidates for defined agent workloads, but parity remains task- and harness-specific.

**Thesis:** LangChain’s early results suggest open-weight models are becoming viable candidates for defined agent workloads, but parity remains task- and harness-specific.

**Observed facts:** LangChain reports that GLM-5 solved 94 of 138 cases (64%) and MiniMax M2.7 solved 85 (57%) in its Deep Agents evaluation. In the same table, Claude Opus 4.6 scored 100 of 138 (68%), while GPT-5.4 scored 91 (61%). The reported suite includes file operations, tool use, retrieval, conversation, memory, summarization, and unit tests.

**Mechanism:** The mechanism is a shared harness and grading scheme: models face the same task cases and success assertions, while the evaluation tracks correctness alongside solve rate and efficiency measures. That makes model choice testable against an agent workflow rather than inferred from broad benchmark reputation. It does not eliminate provider, inference, or harness effects.

**Why now:** LangChain says the results are early and that its evaluation set is being maintained and expanded. Its post also describes the suite running across 52 models, giving teams a practical route to compare candidates as open-weight options improve.

**First implication:** Builders can treat open models as candidates for workload-specific routing when their own evaluation shows acceptable task quality and operating cost.

The company estimates that 10 million output tokens per day would cost about $250 with Claude Opus 4.6 versus about $12 with MiniMax M2.7. These are the post’s illustrative prices, not a universal total-cost comparison; they do not establish equivalent performance on any particular production workload.

**Evidence boundary:** This is LangChain’s own early evaluation of its Deep Agents harness, using stated provider configurations and a finite task set; it does not establish general parity across applications or independently reproduce production outcomes.

**First watch condition:** Watch whether the maintained evaluation results remain comparable as the task set grows and whether teams report similar quality and cost trade-offs on their own production workflows.

The distinct analytical record is warranted because this source supplies a specific, side-by-side agent-harness evaluation and cost framing; no prior New Runtime record was supplied for this event.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Open Models](https://newruntime.com/topics/open-models.md): Explore the open-models topic hub.
- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)