---
type: "post"
id: "cutting-our-inference-bill-by-68-by-being-8790d7"
slug: "cutting-our-inference-bill-by-68-by-being-8790d7"
title: "Cutting our inference bill by 68% by being obsessive about optimization"
description: "**Thesis:** Cloudflare’s reported 68% inference-cost reduction suggests that workload-aware request handling can materially change the economics of an existing AI feature."
retrieval_nugget: "**Thesis:** Cloudflare’s reported 68% inference-cost reduction suggests that workload-aware request handling can materially change the economics of an existing AI feature."
published_at: "2026-10-19"
updated_at: "2026-10-19"
record_date: "2026-10-19"
date_kind: "published_at"
topics: ["model-routing"]
source_urls: ["https://blog.cloudflare.com/obsessive-inference-optimization"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:cutting-our-inference-bill-by-68-by-being-8790d7"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:b7a5abda6b76bc2bcdc458a9ec23f6468ee1851e2e9c2435de71eddb2e1e0e23","reviewed_at":"2026-10-19T20:07:41.592Z","source_evidence_count":1,"verified_claim_count":2,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":2,"implication_count":1,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Cloudflare’s reported 68% inference-cost reduction suggests that workload-aware request handling can materially change the economics of an existing AI feature.","observed_facts":[{"text":"Cloudflare says its AI code reviewer’s cost per run doubled before the team investigated token usage.","source_urls":["https://blog.cloudflare.com/obsessive-inference-optimization"]},{"text":"Cloudflare reports a 68% inference-bill reduction and faster reviews after applying prompt caching and intelligent model routing.","source_urls":["https://blog.cloudflare.com/obsessive-inference-optimization"]}],"mechanism":"Prompt caching reuses context and intelligent model routing directs requests according to the workload; Cloudflare attributes the savings to these techniques.","why_now":"As teams put AI code review into recurring engineering workflows, a cost increase per run can compound with usage.","implications":["Teams should measure inference cost and latency per completed workflow, then test caching and routing against their own request patterns."],"evidence_boundary":"The public post reports Cloudflare’s own outcome and describes the optimization levers; it does not establish an independently validated comparison or a universal savings rate. The supplied evidence does not provide enough detail to quantify how much of the reduction came from caching versus routing.","watch_conditions":["Look for workload-level measurements that separate the contribution of prompt caching from intelligent model routing."],"related_records":[],"new_branch_reason":"No prior New Runtime record is supplied, so this starts a distinct analytical record for Cloudflare’s reported inference-cost optimization."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/cutting-our-inference-bill-by-68-by-being-8790d7/","markdown":"https://newruntime.com/posts/cutting-our-inference-bill-by-68-by-being-8790d7.md","json":"https://newruntime.com/posts/cutting-our-inference-bill-by-68-by-being-8790d7.json"}
related_patterns: ["agent-economics-moves-to-completed-work"]
---

# Cutting our inference bill by 68% by being obsessive about optimization

## Retrieval answer

**Thesis:** Cloudflare’s reported 68% inference-cost reduction suggests that workload-aware request handling can materially change the economics of an existing AI feature.

**Thesis:** Cloudflare’s reported 68% inference-cost reduction suggests that workload-aware request handling can materially change the economics of an existing AI feature.

**Mechanism:** Prompt caching reuses context and intelligent model routing directs requests according to the workload; Cloudflare attributes the savings to these techniques. Cloudflare says its AI code reviewer’s cost per run doubled, prompting an examination of token usage. The company attributes the subsequent reduction to prompt caching and intelligent model routing. In this account, savings come from changing how requests reuse context and select models, rather than from announcing a new model or eliminating the review workflow.

**Why now:** As teams put AI code review into recurring engineering workflows, a cost increase per run can compound with usage. Cloudflare reports a 68% lower inference bill and faster reviews after its optimization work. This makes the case relevant to teams already operating AI features, but it is a vendor account of one workload—not evidence that the same interventions will produce the same result elsewhere.

**Evidence boundary:** The public post reports Cloudflare’s own outcome and describes the optimization levers; it does not establish an independently validated comparison or a universal savings rate. The supplied evidence does not provide enough detail to quantify how much of the reduction came from caching versus routing.

**Implication:** Teams should measure inference cost and latency per completed workflow, then test caching and routing against their own request patterns. **Watch condition:** Look for workload-level measurements that separate the contribution of prompt caching from intelligent model routing.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Model Routing](https://newruntime.com/topics/model-routing.md): Explore the model-routing topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)