What happened
Artificial Analysis publishes a comparison of speech-to-text API providers using three measures: its Artificial Analysis Word Error Rate index, a speed factor, and estimated transcription price. The AA-WER v2 index measures the percentage of words transcribed incorrectly, where lower is better. Artificial Analysis says it combines AA-AgentTalk at a 50% weight, VoxPopuli-Cleaned-AA at 25%, and Earnings22-Cleaned-AA at 25%.
The publisher says the aggregate covers approximately eight hours of audio and is intended to test real-world speech across diverse accents, domain-specific language, and difficult channel and acoustic conditions. That methodology matters because a single clean-audio test can make a recognizer look better than it will in a customer-support recording, a meeting, or an agent conversation. The benchmark displays 40 of 64 models in its metric views, while the summary table covers the broader set of models.
Artificial Analysis reports Microsoft AI’s MAI-Transcribe-2 at 2.0% word error rate, a median speed factor of 313.3, and an estimated $1.67 per 1,000 minutes of audio. Speed factor means input-audio seconds transcribed for each second of processing time, so a larger number indicates faster processing. The price measure is normalized across providers with different billing models and includes billed reasoning tokens where those apply.
The same table frames the result against competing API choices. ElevenLabs Scribe v3 is reported at 1.9% error, a 74.3 speed factor, and $3.33 per 1,000 minutes. Microsoft’s MAI-Transcribe-1.5 is at 2.4%, 78.2, and $6.00. AssemblyAI Universal-3.5 Pro is at 3.0%, 103.0, and $3.50. Deepgram Nova-3 is faster at a 433.9 speed factor, but the table reports 5.2% error and $4.30 per 1,000 minutes. The figures do not yield one universal winner; they show the shape of the quality, throughput, and cost trade-off.
Artificial Analysis says its speed values are medians from benchmark trials in the preceding seven days, using audio of ten minutes in duration. It notes that speed can vary at other lengths, particularly below one minute. The page also describes chunking in the Earnings22 evaluation for models that cannot reliably handle full-length input because of time limits: roughly nine-minute chunks for several services and roughly 30-second chunks for some shorter-limit models. Those implementation constraints are relevant when translating a dashboard comparison into a production workflow.
What the archive adds
New Runtime’s earlier coverage of Microsoft’s MAI-Transcribe-2-Streaming and MAI-Voice models on October 3, 2026, described Microsoft’s speech stack for conversational agents. An October 21, 2026 note on Voice Standard Bench examined voice-model comparison by performance, speed, and cost. This is a new benchmark snapshot on that known selection story, not a new Microsoft launch.
Trend signal
This supports the conclusion in New Runtime’s earlier Voice Standard Bench note that voice-model selection is becoming a downstream pipeline decision rather than an accuracy-only choice. Error rate, speed, and estimated price sit together in the comparison. A team therefore needs to judge the model inside the full workflow: ingestion, transcription, correction, downstream extraction, and delivery to a user or agent.
Why it matters
Transcription is often treated as a commodity API call, but its operational consequences vary sharply by product. In an asynchronous archive pipeline, a modest increase in error rate may be acceptable if throughput and cost are materially better. In a voice agent, a processing bottleneck can make the interaction feel broken even if the final transcript is accurate. In regulated or specialist workflows, errors on names, quantities, identifiers, and domain vocabulary can cost more than the headline word-error figure suggests.
The benchmark gives teams a disciplined starting point for a shortlist. MAI-Transcribe-2’s reported combination of 2.0% error, 313.3 throughput, and $1.67 per 1,000 minutes makes it a credible candidate where batch speed and unit economics matter. But a candidate is not a deployment decision. Teams still need to account for their audio distribution, request sizes, latency expectations, and the cost of correcting errors downstream.
What happens next
- More speech-to-text teams will evaluate models as quality-throughput-cost portfolios rather than accuracy leaderboards. (6 months, 70%). Watch for: public benchmarks adding workload-specific latency and end-to-end cost views.
- MAI-Transcribe-2 will be tested against production audio by teams that need high-volume asynchronous transcription. (3 months, 60%). Watch for: independent reports on domain-specific error rates and billed usage.
Limits and uncertainty
AA-WER is an aggregate of three datasets, with AA-AgentTalk carrying half of the weight. It may not predict results on a particular team’s languages, accents, microphones, recording conditions, or specialized vocabulary. The speed figure is based on ten-minute audio and can change with shorter clips, while the benchmark itself uses chunking for services subject to input-duration limits. Estimated price is normalized across billing models and may differ from a customer’s actual invoice. The comparison also does not independently establish service reliability, streaming latency, regional availability, privacy handling, or performance under a specific production load.
Skill to build now
Create a small transcription evaluation set from representative, consented recordings: include short clips, long files, background noise, key names and numbers, and the languages or accents your product actually receives. Run two or three candidate APIs through the same pipeline. Record word errors that matter to the product, end-to-end completion time, chunking behavior, and billed cost. This week’s outcome should be a workload-specific decision table rather than a provider choice based on a single public score.
Reader takeaway
MAI-Transcribe-2 looks highly competitive in Artificial Analysis’s current comparison because it combines low reported error, high throughput, and low estimated cost. Use that result to decide what to test first. The production choice still belongs to the team that measures the audio, latency, error types, and billing behavior of its own workflow.