What happened
oMLX’s 0.7.3 release, published October 31, updates the Apple-silicon inference server and its Ask AI app. The release notes describe improvements since release candidate 0.7.3rc1 and summarize earlier 0.7.3 changes as well. The notes provide macOS 26/27 and macOS 15 Sequoia download options. They do not state a price.
For DeepSeek V4.1 Flash, the release adds INT8 Activation Prefill in Model Settings and Engram prefetch. On M5 Ultra, INT8 Activation Prefill raises reported prefill throughput from 2,882 to 3,507 tokens per second on a 64K prompt, a 22% increase; on a 195K prompt, the figures are 2,476 and 3,200 tokens per second, a 29% increase. Engram prefetch on a 128K prompt raises the reported rate from 2,075 to 2,390 tokens per second, or 15%. The notes also say that batches larger than eight requests now remain on fused decode kernels. On Macs before M5, changes to DeepSeek V4.1’s HC projection and decoder skip produce a reported 3–4% prefill increase on an M3 Ultra.
A separate set of changes targets memory and decode when many requests run together. The batch KV cache is updated in place rather than copied. In one M3 Ultra test, eight Lightning MTP streams at 81K context using Qwen3.8-Flash-Next oQ4e reduced peak memory from 248.0 to 161.1 GiB (35%) and increased decode from 180.7 to 241.1 tokens per second (33%). Eight agent conversations at 160K context, with MTP off, reduced memory from 185.3 to 153.0 GiB and increased output from 36.4 to 42.4 tokens per second. Another example, eight Qwen3.8-27B oQ4e requests on an M2 Max, reports decode rising from 27.9 to 48.6 tokens per second. These are workload-specific measurements from the release notes.
The desktop Ask AI interface adds voice conversations, web search, link reading, and screen capture. Voice is enabled in App > Ask AI; users press Option-Shift-Space to speak, and answers play back with captions. The release also adds a /v1/realtime endpoint that follows a subset of the OpenAI Realtime API. The model manager can check for updates and fetch only files changed on Hugging Face rather than redownloading a full model. Other changes include added model support, quantization and kernel updates, API behavior changes, and bug fixes; the notes do not give a complete compatibility matrix for every feature.
What the archive adds
This is a new release in a recurring local-inference thread, rather than a restatement of an earlier oMLX release. The archive covered Kapitsa v0.2.15 and TensorFold 1.2.0, both related Apple-silicon inference updates. An earlier Telegram post described Uzu as an open inference runtime for apps on Apple Silicon; the new event is oMLX’s specific combination of batched-inference changes and Ask AI features.
Trend signal
No supplied SITE_PATTERNS thesis directly fits this release. Its measured focus is hardware-specific local inference throughput and memory, rather than a shift in agent economics, runtime security boundaries, or portable agent protocols.
Why now
The release notes explain the timing through the release itself, but do not tie 0.7.3 to a broader event or deadline. The practical prompt is the availability of new measurements and features for users already running oMLX on supported Macs.
Why it matters
For a team hosting local model workloads on Apple silicon, memory can determine how many long-running conversations fit on one machine. The in-place KV-cache update and batched weight reads address that constraint directly: the supplied examples show lower peak memory alongside higher decode rates for selected multi-request configurations. That makes the release worth evaluating for local prototypes or deployments where concurrent sessions are already constrained by unified memory.
The numbers should guide a benchmark, not a capacity plan by themselves. Teams need to reproduce their model, quantization, context length, request mix, and hardware before deciding whether the update changes deployment economics. Ask AI’s voice and browsing features also make a local interface more capable, but the release notes do not establish that its search or screen-reading behavior meets any particular security, privacy, or reliability requirement.
Limits and uncertainty
All performance figures are reported by the project and correspond to named models, hardware, context lengths, and settings. The source does not provide independent replication, test methodology beyond those configurations, power consumption, sustained-run behavior, or comparisons with cloud inference. It does not show that every Apple-silicon machine will see the same gains, nor that memory reductions translate directly into support for a particular number of production users. The release notes also do not specify a price, support window, or a full feature-by-feature OS and hardware compatibility matrix. Ask AI’s realtime endpoint follows only a subset of the OpenAI Realtime API, so API compatibility should not be assumed to be complete.
Skill to build now
Build a small local inference regression test before upgrading: replay representative prompts at the context lengths and concurrency levels your team uses, record peak memory and decode throughput, and compare the same model and quantization on your target Mac. Include one long-context multi-session case, since that is where the release’s memory claim is most relevant. Treat the published figures as comparison points, not expected outcomes.
Reader takeaway
oMLX 0.7.3 is most notable for measured reductions in memory use and faster batched inference on specific Apple-silicon workloads, with additional voice and browsing features in Ask AI. If local concurrency is a real bottleneck, benchmark the update on your own model and machine; the release’s results are useful evidence to test, not a promise that every setup will match them.