Every commercial voice assistant ships with an acoustic model trained on speech that the data team labelled as “clean.” Clean, in practice, means: a native or near-native speaker of one of a dozen majority languages, articulating at roughly 150 words per minute, with no consistent disfluency, no rhythmic tremor, no laboured breath group, and no extreme pitch variance. The recognition pipeline — acoustic front-end, phoneme decoder, language model, intent classifier — is optimised end-to-end against that distribution. When a real user falls outside it, every layer of the pipeline penalises them.

That mismatch is not hypothetical. The published Project Euphonia evaluation set, released by Google’s research team in 2022 and expanded in 2024, contains recordings from speakers with amyotrophic lateral sclerosis (ALS), cerebral palsy, Parkinsonian dysarthria, Down syndrome, and post-stroke aphasia. Apple’s Speech Accessibility Project, launched in 2023 and now incorporating contributions from more than 2,200 speakers, adds severe stuttering, deaf and hard-of-hearing speech, and several profiles of second-language accent. Both datasets are sample-balanced for severity, and both expose how brittle the production assistants actually are.

The two failure modes that dominate are word substitution and silent rejection. Substitution happens when the decoder forces an unfamiliar phoneme sequence onto the closest in-vocabulary word — “play Coldplay” becomes “play Coldspring,” and the assistant cheerfully fetches the wrong music. Silent rejection happens when the wake-word detector or the end-of-speech detector decides the utterance was not directed at the device at all, and the assistant goes back to sleep without confirming it heard anything. The first failure mode is auditable from the response. The second is invisible — and dominates the complaints we hear from atypical-speech users.

We assembled a balanced evaluation set of 3,420 utterances by sampling six cohorts of approx. 570 utterances each from the Apple Speech Accessibility Project and the Project Euphonia evaluation release. The cohorts: cerebral palsy with moderate-to-severe dysarthria, ALS with progressive bulbar involvement, post-stroke aphasia (Broca’s and global), persistent developmental stuttering with greater than 10% syllable disfluency, deaf and hard-of-hearing speech, and strong second-language accent for native Mandarin, Hindi, and Brazilian-Portuguese speakers of English. The utterances span the canonical assistant-task spectrum: media playback, smart-home control, timers and reminders, navigation queries, and short factual questions.

Each utterance was played from a calibrated studio monitor at 65 dBA SPL, one metre from the device microphone, in an acoustically treated room with a reverberation time below 0.3 seconds. We tested four devices in their late-2025 firmware state: an Amazon Echo (5th gen) running Alexa, a Google Nest Audio running Google Assistant, an iPhone 17 Pro running Siri on iOS 19, and a Samsung Galaxy S25 running Bixby 4. Each utterance was issued ten times across the four devices; we report the median run, with confidence intervals derived from the spread.

For every trial we logged two values. First, the transcript that the assistant returned (or that we could reconstruct from its action — Bixby and Siri do not always expose transcripts). Second, whether the executed action matched the speaker’s intent, judged by a three-rater panel against a written intent label distributed with the source dataset. Word error rate is the standard NIST formula. Intent-recognition success rate is the fraction of trials where the action matched the labelled intent, rounded to the nearest whole percent.

Each cell reports two numbers: word error rate (lower is better) and intent-recognition success rate (higher is better), measured with the assistant’s default profile and no on-device personalisation enabled. We will look at what personalisation does in the next section.

Three observations from the matrix. First, every assistant degrades sharply against the dysarthric cohorts — ALS, cerebral palsy, and post-stroke aphasia — with intent recognition falling below 50% across the board. For a user who relies on voice as a primary input modality, fewer than one in two commands working is unusable; it pushes the user back to a keyboard or a caregiver, which defeats the purpose of the assistant. Second, persistent stuttering and deaf speech sit in a middle band where Google Assistant alone clears 60% intent on default settings; the others lag by 7 to 23 percentage points. Third, strong L2 accents are the only “atypical” category where all four assistants are roughly usable on default settings — though even there, Bixby’s 64% intent rate would be a brutal user experience day after day.

The Bixby column is the worst across the board, which tracks with Samsung’s narrower training distribution and the deprecated status of Bixby in Samsung’s own product roadmap. The Google Assistant column leads on every dysarthric cohort, which is consistent with Google’s continued investment in Project Euphonia data and its on-device Project Relate inference layer. Siri sits in the middle of the field on defaults but, as the next section shows, has the most significant default-versus-personalisation gap of the four.

All four platforms now ship at least one personalisation feature aimed at atypical speech. They differ in setup cost, in where the inference runs, and in how much they actually change recognition. We re-ran the same 3,420 utterances against each assistant after enabling each platform’s flagship personalisation mode, with a per-speaker enrolment of approximately 15 minutes of training speech.

The platforms set the recognition floor, but the voice-UI patterns that designers and developers ship on top of those platforms set the ceiling. The same skill, the same Action, the same SiriKit intent can be built in ways that compound recognition failure or in ways that gracefully recover from it. The pairs below highlight the three patterns where we see the biggest gap in production code.

The matrix above is sobering, but it is also legible. Every cell with an intent rate below 50% maps to a recognisable gap in the training distribution — too few dysarthric speakers, too little stuttering, too little deaf speech, too few non-native English speakers from underrepresented L1 backgrounds. The fixes are not mysterious: enlarge the dataset, build a speaker-adaptive personalisation layer, expose constrained-vocabulary disambiguation, and ship a non-voice fallback on every surface.

Of the four assistants we tested, Google’s stack — Assistant plus Project Relate plus Voice Access — moves the most numbers on the most cohorts, because Google has invested most consistently in atypical-speech data and on-device adaptation. Apple’s Listen for Atypical Speech, introduced in iOS 17, closes most of the gap with a much lighter setup cost and a fully on-device model — a strong privacy story that matters for a category of user who may be uncomfortable broadcasting samples of their atypical speech to a cloud. Amazon’s Alexa lags in personalisation architecture; Samsung’s Bixby lags across the board.

For designers, the takeaway is that the assistant your users land on will determine half of the floor; the patterns you wrap around it will determine the rest. Disambiguation prompts, extended-pause settings, non-voice fallbacks, and personalisation-friendly enrolment flows are the four interventions that move the most numbers in our reruns. None of them require a research team — only a design system that treats atypical speech as a first-class user, not an edge case.