Rented intelligence → owned learning loop
Organizations are moving the durable learning asset out of one model provider and into user-owned traces, evaluations, corrections, and promotion rules.
Tectonic shift
Before: Prompts, corrections, memory, and successful behavior accumulate inside one model provider After: A user-owned evaluation and trace layer improves and compares replaceable models Current stage: emerging This New Runtime record is an evidence-linked retrieval unit. Use its canonical page, machine-readable representations, dates, scope, and public source URLs to verify the claim before reusing it.
Source ledger
Publishable sources attached to this record.
| # | Source | Role | Public status |
|---|---|---|---|
| 1 | devblogs.microsoft.comarticle | primary receipt | source_urls |
| 2 | learn.microsoft.comsource | supporting receipt | source_urls |
| 3 | docs.cloud.google.comdocs | supporting receipt | source_urls |
| 4 | opentelemetry.iosource | supporting receipt | source_urls |
| 5 | openai.comsource | supporting receipt | source_urls |
| 6 | privacy.claude.comsource | supporting receipt | source_urls |
| 7 | langchain.comsource | supporting receipt | source_urls |
| 8 | evaliqa.comsource | supporting receipt | source_urls |
| 9 | vercel.comsource | supporting receipt | source_urls |
Current state, as of Oct 9, 2026
Where this stands now
strengthening User‑owned trace and evaluation layers are now deployed in multiple commercial platforms, confirming the shift from a single provider‑centric learning asset to an organization‑controlled improvement loop.
Production tools such as LangSmith and Uber’s feature‑logging framework now capture session trajectories, online evaluations, and correction data outside the model provider. These artifacts are versioned and used to rank and promote alternative model configurations, creating a portable improvement loop.
The practice is no longer limited to prototypes; Microsoft Foundry, Google agent evaluation docs, and emerging open‑source conventions all describe trace‑based scoring and model‑agnostic optimization. Organizations can replace underlying models without discarding accumulated evaluation evidence.
For builders, the implication is that designing a durable trace schema and evaluation API is now a prerequisite for any enterprise‑grade agent system, and model‑selection logic can be externalized to a router that consumes these user‑owned assets.
What changed since the original text
- LangChain’s LangSmith launch adds chronological session views, online evaluations, and dataset creation from production runs, demonstrating concrete trace capture outside the provider. 1
- Uber’s feature‑logging framework and LangSmith’s closed‑loop suite illustrate organizations shifting durable learning assets to user‑controlled layers, strengthening the claim. 2
Open questions
- How will standardization of trace formats affect cross‑vendor model promotion?
- What governance models will emerge for organization‑owned promotion rules and cost optimization?
Watch next: The release of a cross‑vendor model‑agnostic promotion API or standard trace schema would further cement the owned learning loop.
- Weekly update: LangSmith launches session trajectories with online evaluations update, Sep 28, 2026
- Weekly update: User‑owned trace layers and closed‑loop agent tools emerge across platforms update, Sep 21, 2026
Rewritten automatically by the New Runtime knowledge desk from this record's weekly updates, releases and archive. The original text below is kept as written (Jun 16, 2026); where the two disagree, this section is newer.
Weekly knowledge desk
Weekly updates
3 reviewed updates to this shift, newest first. Every update is checked against its evidence before it changes the record.
IBM describes learned behavioral guidance that can improve an agent without changing model weights
IBM Research describes ALTK-Evolve as distilling guidance from an agent’s task attempts and appending it to the system prompt, without changing model weights. That is a concrete example of learning preserved as reusable guidance rather than only in model parameters, though the item does not establish a user-owned comparison layer.
LangSmith launches session trajectories with online evaluations
LangChain’s LangSmith launch presents agent-session behavior in a chronological view and adds online evaluations and dataset creation from production sessions. This is a concrete example of capturing traces and evaluations outside the model provider, though the item does not establish model portability or user-owned promotion rules.
User‑owned trace layers and closed‑loop agent tools emerge across platforms
Recent posts describe Uber’s feature‑logging framework that makes the online inference record the canonical source for future training, and LangSmith’s suite that turns agent operations into a closed improvement loop. Both illustrate organizations shifting the durable learning asset from a single model provider to user‑controlled evaluation, trace, and correction layers. The convergence of these independent efforts strengthens the claim that firms are moving toward owned learning loops.
Record state: stage accelerating, confidence high.
- Taming the ML Firehose: Scaling Feature Consistency post, Sep 24, 2026
- Taming the ML Firehose: Scaling Feature Consistency signal, Sep 24, 2026
- 1️⃣ LangSmith Engine v2 2️⃣ Managed Deep Agents v0.8 3️⃣ LangSmith Trajectories 4️⃣ LangSmith Fine-Tuning 5️⃣ Custom Apps Everything we announced at Interrupt NYC ⤵️ https://t.co/8cnORN5IbT post, Sep 26, 2026
Organizations are beginning to treat their accumulated evaluation and correction data as a durable product asset. Models can change while the criteria for good work, representative tasks, failure cases, accepted outputs, and promotion history continue to compound.
What is changing?
A portable harness is only the enabling layer. The more consequential change is ownership of the improvement loop around it.
That loop includes:
- factual traces of model inputs, outputs, tool calls, and outcomes;
- owner corrections, accepted results, retries, and escalation decisions;
- versioned tasks, rubrics, protected cases, and cost constraints;
- prompts, skills, tool contracts, and routing policies tested against the same evaluation set;
- an audit trail showing why one candidate configuration or model was promoted.
When those artifacts live outside a model product, active models can be compared on the organization’s real work rather than on a public leaderboard. Changing the model no longer means discarding the evidence used to improve the system.
Evidence
Microsoft Foundry now describes a closed learning loop with a swappable model, traces for every run, organization-defined rubrics, and optimization across instructions, skills, tools, and model choice. Its Agent Optimizer evaluates candidates against the same task set and ranks them by outcome and token cost.
Google’s agent evaluation documentation treats a trace as a factual, immutable record containing model inputs, responses, and tool calls, then uses those traces as the basis for scoring and iterative optimization. OpenTelemetry’s GenAI conventions make model, token, prompt, completion, tool call, and tool-result telemetry portable enough to observe across different agent stacks.
Boundary
This shift should not be justified by claiming that every provider trains on all customer data. OpenAI and Anthropic state that their business and API products do not use customer inputs or outputs for model training by default.
The broader dependency risk remains: prompts, memories, corrections, eval datasets, and successful workflow history can still become trapped inside one provider’s product even when that provider does not train on the data.
Counter-evidence
Provider-native optimizers can exploit undocumented model behavior and proprietary runtime features that a portable learning layer cannot reproduce. Model changes also alter planning, tool use, latency, and safety, so switching is never free.
Revision trigger
Revise this shift if cross-provider replay remains too lossy for meaningful comparison, or if provider-native learning systems consistently beat user-owned traces and evaluations on reliability, total cost, and migration risk.
September 2026 update
What changed
Readable production trajectories, an independent evaluation framework, and observable memory operations give operators raw material to retain and improve their own loop.
Current boundary
Owning logs is not learning. The loop needs quality gates, correction authority, privacy controls, and an explicit promotion decision.