Measured English word error rates, including where it fails
Transcription apps tend to publish one impressive number. This page publishes the benchmark runs instead: five public English datasets, the same processing the app ships, and an accented-speech set where the default model scores 19.18%. If you are deciding whether a transcript can be trusted, start with the row that looks like your audio.
Measured 2026-10-22 on an Apple Silicon Mac mini with 32 GB of memory. English only.
Results by dataset
All figures are word error rate on the default English model, after the app's normal formatting step. Lower is better.
| Dataset | Clips | Word error rate | What it represents |
|---|---|---|---|
| FLEURS (English) | 194 | 5.93% | Read speech in a studio-like setting; the general-purpose headline number. |
| LibriSpeech test-clean | 188 | 1.68% | Audiobook narration with clean recording conditions; the best case. |
| LibriSpeech test-other | 198 | 3.23% | Audiobook narration with harder speakers and noisier recordings. |
| VoxPopuli (English) | 200 | 6.16% | European Parliament speech: formal, multi-speaker, real room acoustics. |
| EDACC (accented) | 164 scored | 19.18% | Accented English. This is the known weak spot, and the number to plan around. |
| Long audio | 2 | 1.26% | Only two clips, so treat this as indicative — not a guarantee for hour-long files. |
How these numbers were produced.
- The full app ran on each file, not a model in isolation: files went through the same processing and formatting a user gets, and the output was scored as-is.
- Scoring is word-level edit distance against the reference transcript, lowercased and stripped of punctuation.
- EDACC is scored on 164 clips. Twenty-six clips shorter than half a second are excluded from scoring but kept in the audio; clips with no detected speech count as empty transcripts rather than being dropped.
- Hardware was an Apple Silicon Mac mini with 32 GB of memory. Word error rate does not depend on the Mac; speed and memory do.
- No audio was deleted after the run; the evaluation sets stay in the repository.
What a batch costs your Mac.
- Speed once the app is warmed up: roughly 40 seconds of audio processed per second of compute.
- End-to-end, as a file is actually processed: about 4.5 to 8.5 seconds of audio per second of compute. Use that figure when estimating a batch.
- Memory while transcribing: about 1.9 GB per file in progress. Batch limits are set from that figure.
Measured on the same Mac mini. Your machine may be faster or slower, and long files use more memory than a short one does.
Where this benchmark stops.
- Accented English is the documented weakness: 19.18% on EDACC versus 5.93% on FLEURS.
- The long-audio result covers only two clips. It is not evidence about arbitrary hour-long recordings.
- No speaker labels are produced in this beta, so the benchmark says nothing about diarization quality.
- Word error rate is a text metric. It does not measure whether a quotation is usable without checking the audio.
- Only English is measured here. Other languages are not part of this beta.
Datasets, and where they come from.
Every number above comes from a public corpus. These are the originals, so the result can be checked against them rather than taken on trust.
- FLEURS — Google's multilingual speech benchmark; the English split is used here.
- LibriSpeech — Audiobook narration, test-clean and test-other splits.
- VoxPopuli — European Parliament recordings released by Meta AI research.
- EdAcc — The Edinburgh International Accents of English Corpus.
Benchmark questions.
What do these numbers refer to?
The default English setup that ships with the app — what you get after a normal install, not a special evaluation build.
Why publish a bad score?
Because accented English is common and 19.18% changes how you should use the output. A benchmark that only lists the best dataset is a marketing page, not a measurement.
How was word error rate calculated?
Word-level edit distance divided by the number of reference words, after lowercasing and removing punctuation. It counts insertions, deletions and substitutions.
Is the benchmark run on the same app I would download?
Yes. The evaluation ran the same build and the same processing a user gets, not a separate test setup.
What hardware was used?
An Apple Silicon Mac mini with 32 GB of memory. Accuracy figures do not depend on the Mac; the speed and memory figures do.
How long does transcription take?
With the app warmed up, roughly 40 seconds of audio per second of compute. End to end, allowing for per-file setup, expect about 4.5 to 8.5 seconds of audio per second of compute — a one-hour file takes roughly 7 to 13 minutes.
How much memory does it need?
About 1.9 GB while a file is being transcribed. Running several at once multiplies that, so a batch works through a limited number of files at a time.
Will accuracy improve?
The English transcription is benchmarked again whenever accuracy work lands. Numbers on this page are dated, and an improvement gets a fresh row rather than a silent edit.
Reserve the beta before it ships.
Beta users get free transcription for good. Leave your email to reserve the beta — no payment, no card. We'll send the signed build to the same inbox once it's ready. No newsletter, no drip sequence, no sharing your address.
- Confirm your email — one click.
- We finish signing the build.
- Install link and setup guide arrive in the same inbox.
Reservation received. Check your inbox.
We sent a confirmation link. Click it to confirm your address — your reservation is complete, and the beta build goes to the same inbox the moment it's signed.