What happened
oMLX maintainer Jun Kim released version 0.8.0rc1 on November 12 as a release candidate and explicitly recommends oMLX 0.7.3 for users who need a stable version. The headline addition, Prefill Accelerator, distributes prompt prefill across the primary Mac and a helper machine. The helper runs the first layers of each prompt chunk; the Mac runs the remaining layers, handles decode, and keeps the KV cache. An automatic setting chooses how many layers the helper should take based on measured speed of both machines and their connection. If splitting the work would be slower, oMLX keeps the prompt on the Mac.
The helper can be a PC with NVIDIA GPUs, a DGX Spark, or a second Mac with a faster GPU. The release's benchmark table measures prefill on a 64K prompt with oQ4e models, an M3 Ultra with 512 GB of memory, and a CUDA helper connected over 2.5 GbE. For Qwen3.8 27B, prefill rises from 352 tokens per second on the M3 Ultra alone to 1,062 with a DGX Spark, a reported 202% increase, and 1,879 with two RTX A6000 GPUs, a 434% increase. Qwen3.8 Flash-Next rises from 1,370 to 2,715 with DGX Spark and 3,928 with two RTX A6000s. The release says decode speed stays about the same in these tests.
Setup requires enabling Distributed Inference in Global Settings, pairing the helper in the Cluster tab, and selecting it under Prefill Accelerator in the model settings. For an NVIDIA PC, the release provides a Docker command for running jundot/omlx-cuda-helper; it says no SSH or oMLX installation is needed on that PC. The requirements are NVIDIA driver 580 or newer and an Ampere-or-newer GPU or a GB10. The helper has not been tested on Windows. Both Macs must run the same oMLX version. The release also flags a build-specific issue: people who downloaded build 3727 immediately after release may need to download again if the Cluster tab cannot find a DGX Spark or NVIDIA PC.
The candidate also changes DeepSeek V4.1 performance under concurrent requests and clusters. With the DSpark drafter enabled, the release reports total decode increasing from 73.2 to 107.9 tokens per second for eight requests on an M3 Ultra, a 47% gain. It reports other gains for tensor-parallel and pipeline clusters, including estimated prefill improvement from 742 to 1,284 tokens per second on a 32K prompt for an M3 Ultra plus M5 Max. Separately, a new Balanced concurrent-request priority setting keeps running streams at 10 tokens per second or more and cuts a new prompt's time to first token by 26%, according to the release.
What the archive adds
New Runtime's earlier coverage of oMLX 0.7.3 on October 31 covered faster DeepSeek V4.1 prefill, batched decode, and lower memory use for concurrent long-context requests on Apple silicon. This is a new release with a new cross-machine prefill capability, rather than a restatement of that earlier performance update.
Trend signal
This supports agent economics moves to completed work only weakly: the release is framed around throughput and latency rather than cost per verified task. It does show infrastructure shifting attention from isolated model speed toward completing real workloads faster, but provides no task-completion or cost measurements.
Why it matters
For teams that run large models locally on Apple silicon, prefill can be a separate bottleneck from token generation. This release offers a way to add compute for processing long prompts without replacing the Mac that handles decode or relocating its KV cache. The benchmarked improvements are substantial on the named models and hardware, and the automatic split-selection logic aims to avoid offloading when the link or helper would make it counterproductive.
The trade-off is operational: a team must provision and pair another machine, meet the GPU and driver requirements, and validate performance on its own network, model family, and prompt lengths. Since decode stays roughly unchanged in the stated test, the benefit should be largest in workflows where prompt processing is material to latency, not workloads dominated by generating long answers. The release-candidate status also argues for testing before relying on it in production.
Limits and uncertainty
The performance figures are reported by the oMLX maintainer, not independently verified in the release. They cover particular quantized models, prompt lengths, hardware, and a 2.5 GbE connection; results on other networks, Mac generations, model families, or workloads may differ. The release does not quantify end-to-end user latency or power and operating costs for adding a helper machine. It says the NVIDIA helper has not been tested on Windows, and 0.8.0rc1 is explicitly a release candidate that may contain bugs.
Skill to build now
If you run long-prompt inference on an M-series Mac and have a compatible NVIDIA machine, benchmark the same prompts locally and with the helper before changing production routing. Record prefill tokens per second, time to first token, decode speed, and network conditions; the release's setup is specific enough to make that comparison actionable, while the automatic split may behave differently across links.
Reader takeaway
oMLX 0.8.0rc1 makes local inference on a Mac a multi-machine workload: compatible NVIDIA hardware or a faster second Mac can take prefill layers and substantially raise reported prompt-processing throughput. Treat the published gains as hardware-specific maintainer measurements, and test the release candidate on your actual prompts and network before depending on it.