--- type: "post" id: "pli-4-0-0-pre-release-open-source-6d6a03" slug: "pli-4-0-0-pre-release-open-source-6d6a03" title: "PLI 4.0.0 pre-release: open-source LLM inference server for Apple Silicon splits Qwen3.8-27B across GPU, CPU and Neural Engine" description: "PLI 4.0.0 is a pre-release of an Apache-2.0 inference server for Apple Silicon that runs Qwen3.8-27B across GPU, CPU and Neural Engine." retrieval_nugget: "PLI 4.0.0 is a pre-release of an Apache-2.0 inference server for Apple Silicon that runs Qwen3.8-27B across GPU, CPU and Neural Engine. Its published measurements are promising on one Mac Studio, but the release explicitly limits validation to that machine and model family, so builders should treat the results as a testable setup, not a general performance claim." published_at: "2026-11-03" updated_at: "2026-11-03" record_date: "2026-11-03" date_kind: "published_at" topics: ["agent-economics"] entities: ["PLI","Qwen3.8-27B","DFlash2","TensorFold","Apple"] related_patterns: ["agent-economics-moves-to-completed-work"] novelty_label: "new_fact" what_changed: "PLI 4.0.0 adds another Apple Silicon inference server, validated on one M1 Ultra and Qwen3.8-27B; this is separate from TensorFold’s release." prior_refs: ["https://newruntime.com/posts/tensorfold-1-2-0-local-inference-engine-speeds-03faa0/"] source_url: "https://github.com/nicopingu/pli/releases/tag/v4.0.0" source_urls: ["https://github.com/nicopingu/pli/releases/tag/v4.0.0"] schema_version: "newruntime-agent-readable-v0.2" stable_id: "post:pli-4-0-0-pre-release-open-source-6d6a03" status: "published" visuals: [] editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]} routes: {"html":"https://newruntime.com/posts/pli-4-0-0-pre-release-open-source-6d6a03/","markdown":"https://newruntime.com/posts/pli-4-0-0-pre-release-open-source-6d6a03.md","json":"https://newruntime.com/posts/pli-4-0-0-pre-release-open-source-6d6a03.json"} markdown_url: "https://newruntime.com/posts/pli-4-0-0-pre-release-open-source-6d6a03.md" --- # PLI 4.0.0 pre-release: open-source LLM inference server for Apple Silicon splits Qwen3.8-27B across GPU, CPU and Neural Engine ## Retrieval answer PLI 4.0.0 is a pre-release of an Apache-2.0 inference server for Apple Silicon that runs Qwen3.8-27B across GPU, CPU and Neural Engine. Its published measurements are promising on one Mac Studio, but the release explicitly limits validation to that machine and model family, so builders should treat the results as a testable setup, not a general performance claim. ## Summary PLI 4.0.0 is a pre-release of an Apache-2.0 inference server for Apple Silicon that runs Qwen3.8-27B across GPU, CPU and Neural Engine. Its published measurements are promising on one Mac Studio, but the release explicitly limits validation to that machine and model family, so builders should treat the results as a testable setup, not a general performance claim. ## What happened Nicopingu published PLI 4.0.0 as a GitHub pre-release on November 3. PLI is an LLM inference server for Apple Silicon, forked from TensorFold 0.6.6 under Apache-2.0. It retains TensorFold’s OpenAI- and Anthropic-compatible API and exact speculative decoding. The release adds work aimed at running Qwen3.8-27B, a dense model in MLX 4-bit, across GPU, CPU and Apple’s Neural Engine, alongside PLI’s own prompt and decode kernels. A central change is how speculative decoding drafts candidate tokens. PLI can choose between DFlash2 and the model’s own MTP head per conversation; the notes say DFlash2 alone is used for one-turn questions. For those questions it also adds draft trees. Other listed changes include extended draft lists, faster draft kernels, an eight-row verify GEMM attributed to Kapitsa, and attention changes for short rows and small prompt chunks beyond 65,536 tokens of context. These are implementation details in the release notes, not independently verified claims about performance outside the listed benchmark. The Neural Engine path requantizes weights on the CPU and keeps the Neural Engine awake between layers, with units selected according to chunk size. The project notes that this depends on private Apple APIs and that a macOS update could break it; in that case PLI falls back to computing on the GPU. For agent turns, the release describes retaining replies for the next turn, checkpoints without copying the KV cache, checkpoints at fork points, writing evicted checkpoints to disk, and preserving control tokens in requests as text. The author says the release was validated on one machine: a Mac Studio M1 Ultra with 64 GB of memory running macOS 27, and one model family, Qwen3.8-27B. Other Macs and models may run but were not measured. In the public suite, PLI reports 294 prompt tokens per second and 53.2 generated tokens per second at a 1k prompt; at 32k, 285 and 49.2; at 64k, 245 and 40.8; and at 96k, 213 and 34.1. The comparison table includes Kapitsa 0.2.24, llama.cpp b11926 and mlx-lm 0.32.2. PLI reports 47.0 tokens per second for writing the first six answers in an agent session, versus 46.7, 24.3 and 20.9 respectively. The supplied material gives an install command and says `--profile m1-ultra` sets production settings, but specifies no price or broader support guarantee. ## What the archive adds This is a new runtime release in a known local-inference thread, not a restatement. [TensorFold 1.2.0](https://newruntime.com/posts/tensorfold-1-2-0-local-inference-engine-speeds-03faa0/) reported Qwen3.8-27B speedups on Apple silicon, while [oMLX 0.7.3](https://newruntime.com/posts/omlx-0-7-3-speeds-up-deepseek-v4-fd0577/) and [mlx-serve v26.11.1](https://newruntime.com/posts/mlx-serve-v26-11-1-adds-installable-model-3f3b35/) covered distinct runtime improvements on the same hardware family. PLI adds a particular CPU/GPU/Neural Engine split and a single-machine benchmark. ## Trend signal This supports the pattern [agent economics moves to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work) only weakly: PLI includes an agent-session throughput measure, but the release does not establish cost per verified completed task. More directly, it is a data point in the archive’s Apple-silicon local-inference coverage, without a supplied registered assumption that this release can be linked to. ## Why it matters Local inference performance is not determined only by model size or a single accelerator. PLI’s design makes scheduling across GPU, CPU and Neural Engine part of the runtime, and its speculative-decoding choices vary with conversation shape. For a team evaluating private or offline inference on Apple hardware, the release offers a reproducible benchmark suite and a concrete configuration to test against existing runtimes. The practical trade-off is portability and confidence. The Neural Engine path depends on private APIs, so an OS update may remove that acceleration path; and the headline comparisons come from one M1 Ultra, not a range of Macs. Teams should measure their own prompt lengths, turn patterns, model quality and fallback behavior before choosing a deployment architecture. The release makes experimentation possible, not a general claim that PLI is faster on every Apple Silicon system. ## Limits and uncertainty The evidence is the project’s own release note and benchmark table. It covers one Mac Studio M1 Ultra with 64 GB, macOS 27, and Qwen3.8-27B in MLX 4-bit; other Macs and model families are expressly unmeasured. The provided material does not describe independent reproduction, benchmark variance, output quality, energy use, or performance under concurrent production workloads. Neural Engine support uses private Apple APIs and may be affected by macOS updates. The measurements therefore establish a result for the stated setup, not a general ranking of inference servers. ## Reader takeaway PLI 4.0.0 is worth testing if you run Qwen3.8-27B locally on an Apple Silicon Mac and want to explore mixed GPU, CPU and Neural Engine inference. Its numbers are a useful starting point, not a portable promise: reproduce them on your hardware and account for the private-API fallback before relying on the Neural Engine path. Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json ## Continue in the graph Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`. - topic hub: [Agent Economics](https://newruntime.com/topics/agent-economics.md): Explore the agent-economics topic hub. - collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis. - collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites. - whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)