Whisper Hallucinations on Non-English Audio: The 3 Parameters That Fixed Our Pipeline

First-hand production notes · Updated August 2026 · We transcribe Tagalog/Taglish livestream replays in production on Apple Silicon; this failure and fix are from a real 24.4-minute recording.

Local Whisper is effectively free and, for most audio, excellent. Then we fed it a real-world Philippine livestream replay — 24.4 minutes of Taglish speech over background music and hype shouting — and default settings fell apart in two distinct ways. If you transcribe any non-English or music-mixed audio, you will eventually hit both.

The failure, precisely

Running whisper-large-v3 locally (via mlx_whisper on a Mac Studio) with default settings:

Failure 1 — language auto-detection guessed wrong. Taglish code-switches between Tagalog and English; the detector heard the English fragments and confidently judged the whole file as English, mangling every Tagalog sentence after it.

Failure 2 — music segments became a repetition loop. In stretches with only background music, the model latched onto its own previous output and emitted dozens of consecutive hallucinated lines — literally "guys guys guys…" — a textbook conditioning loop.

The three-parameter fix

import mlx_whisper

result = mlx_whisper.transcribe(
    wav,
    path_or_hf_repo="mlx-community/whisper-large-v3-mlx",
    language="tl",                     # 1. Force the language. Never trust auto-detect on code-switched audio.
    condition_on_previous_text=False,  # 2. Kill the repetition loop in music-only segments.
    no_speech_threshold=0.5,           # 3a. Raise the bar for "this segment contains speech".
    compression_ratio_threshold=2.2,   # 3b. Reject degenerate, repetitive outputs.
)

Why each matters: forcing language removes the single highest-impact failure — a wrong detection poisons the entire file, and code-switched speech is exactly where detectors misfire. Disabling condition_on_previous_text stops the model from feeding its own hallucination back to itself, which is what turns one bad music segment into fifty. The two thresholds work as a pair of guardrails: one suppresses transcription of non-speech, the other discards output whose compression ratio betrays it as looped garbage.

When local Whisper is enough — and when to pay

With the fix, local transcription on hardware we already own is effectively free at production volume, and that is what we run daily. We would reach for a paid speech-to-text service in three cases: when accuracy is contractually critical and you want vendor-graded models with support; when you need built-in speaker diarization and word-level timestamps without assembling your own stack; or when you have no capable local hardware and the per-minute price beats renting GPU time. If that is your situation, ElevenLabs' speech-to-text tier is one of the options we would shortlist — the same three failure modes above are worth testing on any vendor before you commit real workloads.

FAQ

Why does Whisper repeat the same phrase over and over? Music-only or silent segments plus condition_on_previous_text=True (the default) create a feedback loop: the model conditions on its own previous hallucination. Disable it and add a compression-ratio threshold.

Should I let Whisper auto-detect language? Not on code-switched or accented audio. A wrong guess poisons the whole file; force the primary language explicitly.

Is local Whisper good enough for production? For our daily livestream transcription, yes — free and reliable once the guardrail parameters are set. Pay for a hosted STT service when you need diarization, SLAs, or have no local hardware.

Disclosure: some links are referral links; signing up through them may earn us a commission at no extra cost to you. The failure case and parameters come from our own production pipeline.