---
type: "post"
id: "tensorfold-1-2-9-adds-faster-decoding-long-02e136"
slug: "tensorfold-1-2-9-adds-faster-decoding-long-02e136"
title: "TensorFold 1.2.9 adds faster decoding, long-prompt improvements and API controls"
description: "TensorFold 1.2.9 adds an Ollama-compatible API, configurable server launches and workload-specific inference changes."
retrieval_nugget: "TensorFold 1.2.9 adds an Ollama-compatible API, configurable server launches and workload-specific inference changes. The release reports faster decoding or prompt processing in several model and hardware configurations, giving teams running the engine new integration options and concrete claims to test against their own workloads."
published_at: "2026-11-16"
updated_at: "2026-11-16"
record_date: "2026-11-16"
date_kind: "published_at"
topics: ["inference"]
entities: ["TensorFold","Nemotron","GLM-5.3-Flash","Qwen3.8-Flash-Next","Ollama"]
related_patterns: ["agent-economics-moves-to-completed-work"]
novelty_label: "new_fact"
what_changed: "TensorFold 1.2.9 adds faster decoding, improved long-prompt handling, an Ollama-compatible API, and fixes for an incorrect reply and a hang."
prior_refs: ["https://newruntime.com/posts/tensorfold-1-2-0-local-inference-engine-speeds-03faa0/"]
source_url: "https://github.com/ashhart/TensorFold/releases/tag/v1.2.9"
source_urls: ["https://github.com/ashhart/TensorFold/releases/tag/v1.2.9"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:tensorfold-1-2-9-adds-faster-decoding-long-02e136"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/tensorfold-1-2-9-adds-faster-decoding-long-02e136/","markdown":"https://newruntime.com/posts/tensorfold-1-2-9-adds-faster-decoding-long-02e136.md","json":"https://newruntime.com/posts/tensorfold-1-2-9-adds-faster-decoding-long-02e136.json"}
markdown_url: "https://newruntime.com/posts/tensorfold-1-2-9-adds-faster-decoding-long-02e136.md"
---

# TensorFold 1.2.9 adds faster decoding, long-prompt improvements and API controls

## Retrieval answer

TensorFold 1.2.9 adds an Ollama-compatible API, configurable server launches and workload-specific inference changes. The release reports faster decoding or prompt processing in several model and hardware configurations, giving teams running the engine new integration options and concrete claims to test against their own workloads.

## Summary
TensorFold 1.2.9 adds an Ollama-compatible API, configurable server launches and workload-specific inference changes. The release reports faster decoding or prompt processing in several model and hardware configurations, giving teams running the engine new integration options and concrete claims to test against their own workloads.

## What happened
TensorFold’s GitHub release page says version 1.2.9 was released on November 15. The release describes faster decoding and long-prompt processing, an Ollama-compatible API, and fixes for an incorrect reply and a hang. It says the listed speedups preserve replies, while separately identifying changes that alter request handling or defaults.

For Nemotron on Metal without an MTP head, concurrent streams share one round. On an M5 Ultra, the release reports decoding about 20% faster with two streams, 75% faster with four, 2.3–2.5 times faster with eight, and 3.2–3.5 times faster with 24. For GLM-5.3-Flash on CUDA, it reports that the first token arrives 0.6–13.7% sooner at tensor parallelism (TP)=3 for prompts from 131,072 to 2,000,000 tokens, and 6.7% sooner at TP=2 for a 1,000,000-token prompt. Shorter prompts at TP=2 remain within about 2% either way. After a 1,040,384-token prompt, decode rounds are reported as 5.05%, 9.36% and 11.36% shorter at one, four and eight streams, respectively. Qwen3.8-Flash-Next on HIP (gfx1151) decodes about 6% faster for one stream and 2.8–4.6% faster for four.

The release adds Ollama-compatible `/api/chat`, `/api/generate`, `/api/embed`, `/api/tags`, `/api/show`, `/api/ps` and `/api/version` endpoints, using the same engine paths as the OpenAI routes. `tensorfold serve --config FILE.toml` reads server flags from a file. Linux users can opt into a memory-pressure governor driven by PSI with `TENSORFOLD_MEMORY_PRESSURE=1`. On the API side, `/v1/models` now reports each model’s `input_types`; unsupported video requests return HTTP 400 rather than 500. The notes also describe logprobs support across tools, stop strings, reasoning and the Responses API, plus opt-in streams carrying committed token IDs.

Some behavior changes are explicitly scoped. CUDA GLM kernels and Flash Next batched sampled draws are enabled by default; `TF_FLASHNEXT_BATCH_DRAWS=0` restores per-stream draws. On one RDNA 3.5 GPU serving alone, Flash Next on HIP uses a 0.8 MTP confidence floor instead of 0.3; other HIP GPUs and families retain 0.3, and `--mtp-confidence` overrides it. The release also says a CUDA server now refuses unsupported GPUs by name at startup. It identifies an experimental Living Weights v2 controller and several groundwork items as not yet used by served models.

## What the archive adds
New Runtime’s earlier coverage, [TensorFold 1.2.0](https://newruntime.com/posts/tensorfold-1-2-0-local-inference-engine-speeds-03faa0/) (October 31, 2026), covered reported Qwen3.8-27B speedups on Apple silicon and MiniMax-M3.1-Flash support. Version 1.2.9 is a new release on the same project, not a restatement: its new facts include an Ollama-compatible API and separate, configuration-specific decoding and long-prompt claims.

## Trend signal
This release modestly supports [agent economics moves to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work/): serving efficiency matters to the cost and throughput of useful work, not just model-call counts. But TensorFold’s figures measure decoding and prompt timing, not cost per verified completed task. This is an operational signal to test, not evidence that the broader unit of economics has shifted.

## Why it matters
For teams already operating TensorFold, API compatibility can reduce the work of adapting clients built around Ollama-style endpoints. A checked-in TOML configuration can also make server settings easier to review and reproduce. Those are practical integration changes independent of whether a particular GPU realizes the fastest benchmark figures.

The performance numbers are most useful as hypotheses for workload tests. Stream count, prompt length, model, backend and GPU all matter in the release’s examples; a team serving many concurrent requests or very long contexts may see different gains from one handling short prompts. Before changing production capacity assumptions, compare latency and throughput on the actual serving setup, and check the release’s default changes for their model and hardware.

## Limits and uncertainty
The performance figures are TensorFold’s release claims; the release does not describe independent benchmarking or provide a common test protocol and full baseline details for every comparison. Results are tied to named models, backends, hardware, stream counts and prompt lengths, so they do not establish a general speedup across TensorFold deployments. The announcement also does not quantify cost savings or production impact. The release labels some work experimental or groundwork, and says certain additions are not yet used by any served model. Teams should validate compatibility and output behavior in their own environment before relying on the changes.

## Skill to build now
Create a repeatable upgrade test for the TensorFold servers you operate. Check the Ollama-compatible routes against the client calls your application actually makes, then run a fixed prompt set at the production context lengths and concurrent-stream counts. Record time to first token, decode throughput, error responses and output comparisons before and after the upgrade. Keep launch flags in a checked-in TOML file, and explicitly review defaults such as batched Flash Next draws or the HIP MTP confidence floor. This makes a release claim a deployment decision rather than a benchmark headline.

## Reader takeaway
TensorFold 1.2.9 offers a useful API integration path and several targeted performance changes, but the numbers are not universal promises. If you run the engine, test the Ollama-compatible endpoints and benchmark the release on your own model, hardware and concurrency profile before revising capacity plans.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Inference](https://newruntime.com/topics/inference.md): Explore the inference topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
