v0.2.282026-11-22
Nemotron streaming on Apple Silicon, safer API token handling, and a lock fix that unblocks Dependabot.
0.2.27 was not published to PyPI. Its publish job expired before approval, so this is the first PyPI release since 0.2.26 and it includes everything in 0.2.27 (local MLX translation, bounded WebSocket sessions, per-session decoder context, the October security refresh and its fixes).
Added
- Nemotron streaming ASR through MLX (nemotron-mlx-asr extra, --backend nemotron-mlx-asr), using mlx-audio's incremental frontend and cached encoder with the converted mlx-community/nemotron-3.5-asr-streaming-0.6b checkpoint. Sessions have independent caches and language prompts; model loading and decoding stay on one dedicated thread. (#426, thanks @clkao)
Fixed
- A non-ASCII API token now gets 401 (REST) or the 4401 close (WebSocket) instead of HTTP 500, and a non-ASCII configured token matches itself. (#479, fixes #478, thanks @goranai)
Maintenance and security
- Rejected requests log the client address (client=ADDRESS), including REST token failures and refused WebSocket upgrades, so repeated attempts can be traced or fed to fail2ban. The token is never logged. (#476, following discussion #463)
- Locked dependencies refreshed for November (Accelerate 1.16.0 in the Voxtral HF graph, Torch 2.11.0 for the cpu extra on macOS arm64), and the lock now covers only the platforms Torch ships wheels for (Linux x86_64 and aarch64, macOS arm64, Windows AMD64). This fixes the Dependabot update failures. Advisories still held by upstream pins (mostly qwen-asr==0.0.6, NeMo and Diart) are listed in docs/dependency-audit.md. PyPI installs are unaffected by the lock. (#480)
- Reproducible M5 backend evaluation recorded under benchmarks/evaluations/m5-20261107/. (#444)
New contributors
- @goranai made their first contribution in #479.
Full changelog: https://github.com/QuentinFuxa/WhisperLiveKit/compare/v0.2.27...v0.2.28
v0.2.272026-11-07
Local MLX translation, bounded WebSocket sessions, and a security refresh of the locked dependencies.
Added
- Local sentence translation on Apple Silicon through mlx-lm (mlx-llm-mt extra: Hunyuan-MT, TranslateGemma), with shared model weights and per-session languages and buffers. (#422, thanks @clkao)
- Simultaneous MLX translation with calibrated attention (AlignAtt commit policy), and the zh→en calibration for Hy-MT2 1.8B 8bit. (#423, #448, thanks @clkao)
- Per-session decoder context: a context query option on the native and Deepgram-compatible WebSockets, and the OpenAI-compatible REST prompt field now conditions the decoder. (#424)
- --max-concurrent-sessions (64 per worker by default) and --session-idle-timeout (60 s by default) for WebSocket sessions. Upgrades beyond the limit are refused with HTTP 403 before any decoder or FFmpeg process is created. (#465)
Fixed
- Sortformer inference no longer blocks the event loop. (#461, thanks @guhyun9454)
- SimulStreaming reads the no-speech probability at the start-of-transcript token once the context buffer holds text. (#468, fixes #467 reported by @AuCourDe)
- Closing a translation no longer waits behind the inference it needs to interrupt. (#445)
- Final ASR output is no longer lost at the end of a stream, and decoder and diarization failures are reported to clients. (#440, #441)
- Sessions requesting another language on non-Whisper backends no longer fail at construction. (#439)
- Streaming queues are bounded, FFmpeg processes are reaped on disconnect, and failed requests no longer leave collector tasks running. (#432, #435)
Maintenance and security
- Locked dependencies refreshed with the October security fixes (certifi, urllib3, filelock, fsspec, msgpack, Pillow, PyJWT, aiohttp and others), plus AnyIO and HTTPX2. Remaining blocked upgrades are listed in docs/dependency-audit.md. (#466, #472)
- Checkpoint loading is restricted to weights-only tensors. (#438)
- The qwen3-vllm-metal extra no longer pins the vllm-metal GitHub wheel, which was blocking Dependabot. Install vllm-metal with its official installer, then install the extra into that environment. (#473)
- Transport security documented for remote AlignAtt translation servers. (#469)
- Audio input and translation are separated from session orchestration. (#436)
- Benchmark reports are reproducible, keep failures, and pace audio without artificial EOF delays. (#437, #442, #443)
- CI, Docker and dependency updates. (#429, #430, #433, #457, #458, #470, #471)
New contributors
- @clkao made their first contribution in #422.
- @guhyun9454 made their first contribution in #461.
Full changelog: https://github.com/QuentinFuxa/WhisperLiveKit/compare/v0.2.26...v0.2.27
v0.2.262026-08-29
New ASR options, stronger diarization, and predictable Deepgram-compatible streaming.
Added
- NVIDIA Canary-1b-v2 backend (--backend canary) with native timestamps, 25 European languages, and optional automatic language detection. Install with the canary extra. (#387, thanks @wuxuedaifu)
- Non-real-time transcription responses can now include diarization, duration, and real word timestamps. (#396, thanks @felixmr1)
- Opt-in ASR call coalescing (--asr-coalesce-min-s) can trade update cadence for lower repeated encoder work. It remains disabled by default. (#391, thanks @Eoin-Houstoun)
- Configurable pause segmentation and a tested maximum-speaker cap for Sortformer. (#414, #415)
Fixed
- Deepgram-compatible streams now preserve stable deltas, words, timestamps, speakers, punctuation, and causal event ordering; raw linear16, endpointing, UtteranceEnd, errors, and CloseStream are handled consistently. (#419)
- Diarized transcripts are split at speaker turns instead of allowing a segment to straddle two speakers. (#407)
- LocalAgreement consumes its pending buffer at end of input instead of losing the final deferred audio. (#411)
Maintenance and security
- Added manual PyPI Trusted Publishing with a protected GitHub environment and minimal OIDC permissions. (#417)
- Hardened repository workflows and distribution builds, including a patched Setuptools floor. (#400, #416)
- Refreshed Python dependency profiles and constrained Docker dependency updates. (#402, #406, #408, #420)
- Added community governance and support templates. (#412)
New contributors
- @wuxuedaifu made their first contribution in #387.
- @Eoin-Houstoun made their first contribution in #391.
- @felixmr1 made their first contribution in #396.
Full changelog: https://github.com/QuentinFuxa/WhisperLiveKit/compare/v0.2.25...v0.2.26
v0.2.252026-08-01
A FunASR backend, and diarization unbroken for qwen3.
Added
- FunASR SenseVoiceSmall backend (--backend funasr): auto, Mandarin, Cantonese, English, Japanese and Korean on the LocalAgreement policy, with fail-closed timestamp validation and per-session language overrides. Install with the funasr extra. (#385, thanks @LauraGPT)
- Published Docker images now carry SBOM attestations, and the provenance mode is pinned explicitly. (#390, thanks @kobihikri)
Fixed
- qwen3 backends with --diarization silently stopped emitting updates as soon as the first line was diarized: the qwen3 package's own token type lacked the helpers the alignment needs. Tokens are now normalized to WhisperLiveKit's ASRToken at the factory boundary. (#389, thanks @OookTheLibrarian)
- Diarization span corruption in concatenatediarsegments.
- --language yue --buffer-trimming sentence crashed on a bare assert even though the sentence tokenizer supports Cantonese. (#392)
- Invalid per-session parameters (such as an unsupported ?language=) now produce a structured error and close code 4400 on the WebSocket endpoints, and HTTP 400 on the REST endpoint, instead of an unexplained handshake failure. (#393)
- wlk pull funasr and wlk rm funasr explain that funasr manages its own model cache instead of exiting 1 silently. (#394)
v0.2.242026-07-11
Streaming LLM translation, refreshed benchmarks and loud failures.
Added
- AlignAtt translation backend (--translation-backend alignatt): streaming LLM translation through an Alignatt4LLM sidecar. The model drafts ahead over the unstable ASR tail and commits only target words whose attention lands on committed source words, so translations are append-only and released the instant the ASR commits. See docs/translation-alignatt.md.
- Benchmark figures re-measured on an H100 with the current backends (the README scatter plots dated from March and misrepresented the qwen3 backends). Raw results in benchmarks/h100_scatter/.
Fixed
- mlx-whisper produced empty captions with torch >= 2.13 (#383, thanks @RobertBartelds-FleetEnergies).
- The server now fails loudly when the ASR backend produces nothing: warmup errors abort startup with the cause, and a watchdog logs an explicit error if audio flows but no text is ever produced.
v0.2.232026-07-09
Qwen3-ASR backends, latency work and bug fixes.
Added
- Qwen3-ASR backends: qwen3-streaming (PyTorch, CUDA/MPS/CPU), qwen3-vllm (CUDA), qwen3-vllm-metal (Apple Silicon), via the new qwen3-asr-causal package. The causal streaming mode uses 3x less compute, at constant cost per audio second (English only).
- Prebuilt Docker images on GHCR: ghcr.io/quentinfuxa/whisperlivekit.
- Optional API token auth (--api-token).
- Per-session language and translation target via WebSocket query params.
Fixed
- mode=full sessions no longer lose early lines after 5 minutes (#372).
- REST transcription of long files no longer times out silently (#374).
- qwen3-vllm word timestamps and CJK spacing (#375, thanks @samx81).
- Sortformer memory growth on long sessions (#349).
- Local Sortformer weights: --sortformer-model-path (#378, thanks @felixmr1).
v0.2.222026-06-04
Core
- NLLW language aliases : Normalize Whisper Chinese language codes at the NLLW boundary (zh -> zh-CN) for transcription and translation configs
- VAD event handling : Preserve short silence boundaries and avoid cumulative timestamp drift in VAC/VAD chunk processing
- CUDA cache cleanup : Keep per-step decoder cleanup lightweight; CUDA cache release stays on coarse segment boundaries
v0.2.202026-03-12
Core
- Qwen3-ASR Beta : New backend with ForcedAligner word-level timestamps, auto language detection, and Whisper-style model aliases
- DiffTracker : Opt-in snapshot-then-diff WebSocket protocol (?mode=diff) for bandwidth efficiency
- SessionASRProxy : Thread-safe per-session language override
0.2.192026-02-23
Voxtral backend & benchmarks
New: Voxtral backend
- voxtral-mlx: Native MLX backend for Apple Silicon. Runs at 0.18-0.32x real-time, handles 100+ languages with automatic language detection. No extra dependencies needed on macOS.
- voxtral (HF): HuggingFace transformers backend for Linux/GPU. Requires pip install transformers torch.
Benchmarks
New offline benchmark harness (testbackendoffline.py --benchmark) that runs all installed backends and computes WER, RTF, and timestamp accuracy against ground truth transcripts. Results exportable as JSON.
Full benchmark report in BENCHMARK.md with tables, charts, and recommendations for every backend/policy/model combination.
Bug fixes
- Fixed silence double-counting in the audio processor
- Fixed median calculation for even-length lists in timestamp accuracy
- Fixed RTF inflation in metrics collector (was using wall-clock time instead of ASR processing time)
0.2.182026-02-13
Bug Fixes
- Fix SimulStreaming VRAM leak : cross-attention tensors were accumulating unboundedly during the decoding loop (up to env 5GB on repetition loops). Now capped to a rolling window (#283, #275)
- Fix --direct-english-translation not working with LocalAgreement backends : task="translate" was never propagated to the ASR backend's transcribe_kargs, so whisper always ran in transcribe mode regardless of the flag. (#306)
- Fix modelcachedir ignored in SimulStreaming : download path was hardcoded to None. Also fixed task being passed as a boolean instead of a string ("translate"/"transcribe"). (#310)
- Fix NoneType concatenation in addtranslation** : crash when ts.text was None during translation overlay. (#296)
- Fix Segment().fromtokens() bad classmethod call** : was incorrectly called on an instance instead of the class. (#331)
- Fix token.hasended type narrowing : replaced token.issilence() (non-existent method) with proper isinstance(token, Silence) check. (#329)
Improvements
- Add probability field to ASRToken FasterWhisperASR now passes word-level probabilities through (#330, #313)
- Add insertaudiochunk to DiartDiarization, brings the interface in line with SortformerDiarization (#332)
- Initialize OpenaiApiASR.task : was previously referenced but never set, causing potential AttributeError.