---
schema_version: "newruntime-topic-hub-v0.2"
type: "topic_hub"
stable_id: "topic_hub:inference"
slug: "inference"
title: "Inference - New Runtime"
description: "A New Runtime topic hub collecting signals, patterns, field notes, and public sources about inference."
retrieval_nugget: "A New Runtime topic hub collecting signals, patterns, field notes, and public sources about inference. Inference is tracked here as an evidence-linked topic, not as a static glossary entry. The page connects raw observations to pattern hypotheses, longer analysis, and public sources. Use it as the canonical landing page before drilling into individual records."
answer: ["Inference is tracked here as an evidence-linked topic, not as a static glossary entry.","The page connects raw observations to pattern hypotheses, longer analysis, and public sources.","Use it as the canonical landing page before drilling into individual records."]
search_intents: ["inference","inference AI agents","inference software"]
status: "featured"
last_updated: "2026-08-03"
record_date: "2026-08-03"
date_kind: "last_updated"
counts: {"total":37,"signals":31,"patterns":0,"posts":6,"atlas":0,"sources":63}
routes: {"html":"https://newruntime.com/topics/inference/","markdown":"https://newruntime.com/topics/inference.md","json":"https://newruntime.com/topics/inference.json"}
source_urls: ["https://ai.google.dev/gemini-api/docs/gemini-3","https://aiechoes.substack.com/p/building-a-biomedical-graphrag-when","https://anthropic.com/news/claude-sonnet-4-6","https://app.klingai.com/global/release-notes/vaxrndo66h?type=dialog","https://arxiv.org/pdf/2501.07572","https://arxiv.org/pdf/2501.14249","https://blog.bytebytego.com/p/how-chatgpt-optimizes-its-agent-loop","https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4","https://blog.google/technology/google-labs/opal-expansion-160","https://blog.nilenso.com/blog/2025/11/04/a-short-lesson-in-simpler-prompts","https://builders.ramp.com/post/thompson-sampling-model-routing","https://cdn.openai.com/pdf/5e10f4ab-d6f7-442e-9508-59515c65e35d/browsecomp.pdf"]
top_sources: ["https://ai.google.dev/gemini-api/docs/gemini-3","https://aiechoes.substack.com/p/building-a-biomedical-graphrag-when","https://anthropic.com/news/claude-sonnet-4-6","https://app.klingai.com/global/release-notes/vaxrndo66h?type=dialog","https://arxiv.org/pdf/2501.07572","https://arxiv.org/pdf/2501.14249","https://blog.bytebytego.com/p/how-chatgpt-optimizes-its-agent-loop","https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4","https://blog.google/technology/google-labs/opal-expansion-160","https://blog.nilenso.com/blog/2025/11/04/a-short-lesson-in-simpler-prompts","https://builders.ramp.com/post/thompson-sampling-model-routing","https://cdn.openai.com/pdf/5e10f4ab-d6f7-442e-9508-59515c65e35d/browsecomp.pdf"]
---

# Inference - New Runtime

A New Runtime topic hub collecting signals, patterns, field notes, and public sources about inference.

## Retrieval answer

A New Runtime topic hub collecting signals, patterns, field notes, and public sources about inference. Inference is tracked here as an evidence-linked topic, not as a static glossary entry. The page connects raw observations to pattern hypotheses, longer analysis, and public sources. Use it as the canonical landing page before drilling into individual records.

## Patterns


## Field Notes

- [ChatGPT Cuts Repeated Work Across The Agent Stack](https://newruntime.com/posts/chatgpt-agent-loop-efficiency-stack/): ByteByteGo's OpenAI engineering walkthrough connects persistent sessions, stable prompt prefixes, deferred tools, delta tokenization, cache-aware routing, and split inference.
- [Run Three Tests Before Replacing LoRA With Full Fine-Tuning](https://newruntime.com/posts/fireworks-lora-fullft-three-test-protocol/): Fireworks shows how data coverage, optimization, and adapter capacity can create or close an apparent quality gap between LoRA and full fine-tuning.
- [GPT-5.6 Turns Efficiency Work Into API Economics](https://newruntime.com/posts/openai-gpt-5-6-price-performance-frontier/): OpenAI turned GPT-5.6 serving and kernel efficiency gains into lower Luna and Terra prices, plus a faster Sol mode for latency-sensitive API workloads.
- [Ramp Teaches Its Gateway To Route By Failure, Latency, And Cost](https://newruntime.com/posts/ramp-adaptive-llm-routing/): Ramp's internal LLM gateway uses failure-aware online learning to reorder model and service-tier candidates, cutting spend without relaxing request deadlines.
- [Vercel AI Gateway Adds Runtime Budget Controls](https://newruntime.com/posts/vercel-ai-gateway-operating-budget/): Vercel's July 31 AI Gateway releases combine team and project spend budgets, unified fast mode, Laguna S 2.1 capacity, and updated MCP support into a practical inference control layer.
- [OpenAI Shows Efficiency Is a Full-Stack Agent Problem](https://newruntime.com/posts/openai-gpt-5-6-efficiency-stack/): OpenAI's GPT-5.6 efficiency write-up connects model training, inference optimization, and the Codex/ChatGPT Work harness into one compounding cost-performance loop.

## Recent Raw Signals

- 2026-07-15: [Colibri Runs a 744B Model in 25 GB of Memory](https://newruntime.com/signals/colibri-runs-a-744b-model-in-25-gb-of-memory/)
- 2026-07-02: [An inference gateway makes model switching operational](https://newruntime.com/signals/inference-gateway-makes-model-switching-operational/)
- 2026-06-25: [Sakana Fugu Presents Multiple Agents as One Model Endpoint](https://newruntime.com/signals/sakana-fugu-presents-multiple-agents-as-one-model-endpoint/)
- 2026-05-22: [GitHub / yichuan-w/LEANN: Routable Model Components](https://newruntime.com/signals/github-yichuan-w-leann-routable-model-components/)
- 2026-04-30: [GitHub / chenglou/pretext: Executable Design Context](https://newruntime.com/signals/github-chenglou-pretext-executable-design-context/)
- 2026-04-24: [OpenAI: Routable Model Components](https://newruntime.com/signals/openai-routable-model-components/)
- 2026-04-04: [Google / Developers Tools: Routable Model Components](https://newruntime.com/signals/google-developers-tools-routable-model-components/)
- 2026-04-03: [Google Developers / Bring State Of Art Agentic Skills To: Ai-Native Operating Models](https://newruntime.com/signals/google-developers-bring-state-of-art-agentic-skills-to-ai-native-operating-models/)
- 2026-02-18: [Anthropic / Claude Sonnet: Routable Model Components](https://newruntime.com/signals/anthropic-claude-sonnet-routable-model-components/)
- 2026-02-04: [GitHub / cloudflare/moltworker: Agent Security Runtime Boundaries](https://newruntime.com/signals/github-cloudflare-moltworker-agent-security-runtime-boundaries/)
- 2026-02-04: [GitHub / tobi/qmd: Routable Model Components](https://newruntime.com/signals/github-tobi-qmd-routable-model-components/)
- 2026-02-03: [Simonwillison: Routable Model Components](https://newruntime.com/signals/simonwillison-routable-model-components/)
- 2026-02-02: [GitHub / openclaw/openclaw: Routable Model Components](https://newruntime.com/signals/github-openclaw-openclaw-routable-model-components/)
- 2026-01-31: [Google: Agent Security Runtime Boundaries](https://newruntime.com/signals/google-agent-security-runtime-boundaries/)
- 2026-01-28: [Docs: Agent Security Runtime Boundaries](https://newruntime.com/signals/docs-agent-security-runtime-boundaries/)
- 2026-01-20: [GitHub / camel-ai/camel: Routable Model Components](https://newruntime.com/signals/github-camel-ai-camel-routable-model-components/)

## Public Sources

- https://ai.google.dev/gemini-api/docs/gemini-3
- https://aiechoes.substack.com/p/building-a-biomedical-graphrag-when
- https://anthropic.com/news/claude-sonnet-4-6
- https://app.klingai.com/global/release-notes/vaxrndo66h?type=dialog
- https://arxiv.org/pdf/2501.07572
- https://arxiv.org/pdf/2501.14249
- https://blog.bytebytego.com/p/how-chatgpt-optimizes-its-agent-loop
- https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4
- https://blog.google/technology/google-labs/opal-expansion-160
- https://blog.nilenso.com/blog/2025/11/04/a-short-lesson-in-simpler-prompts
- https://builders.ramp.com/post/thompson-sampling-model-routing
- https://cdn.openai.com/pdf/5e10f4ab-d6f7-442e-9508-59515c65e35d/browsecomp.pdf