Release v5.23.02026-11-13
Release v5.23.0
New Model additions
Inkling2
<img width="1200" height="675" alt="image" src="https://github.com/user-attachments/assets/9115e142-1be3-4f79-a51d-ea0e5c25b152" />
Inkling 2 Small is a multimodal model from Thinking Machines Lab that accepts text and image inputs and generates text outputs, and it is intended for use in English and other languages and across multiple coding languages. Its language backbone has 48 mixture-of-experts decoder layers, each routing tokens to 8 of 384 experts, for 594B total and 17B active parameters. It keeps the Inkling decoder (learned relative attention, short causal convolutions, mixture-of-experts layers) and adds block attention residuals, gated attention with a learned sink, and dynamic key/value convolutions. Images are encoded by a Perception Encoder-based vision transformer and projected into the language model embeddings, one embedding per 40x40 pixel patch.
Links: Documentation | Blog Post
* Add inkling2 (#50682) by @guarin in #50682
JEV
JEV is a decision head that scores single-token labels with a linear layer at the last token of the prompt, soft-capped like the logits of a language model head. JevForDecision is the model of the checkpoint and its head, and it reads the head files shipped next to the weights (head.safetensors and judge_config.json for JEV). The same head is also used by GEV and reads the 255 answer codes of pplx-decider and d3, and it can be run through the experimental decision pipeline or directly with decide.
Note: JEV and the decision pipeline are currently experimental. Their APIs, configuration, and behavior may change as development continues. They should be used with caution in production environments.
Links: Documentation
* Add Jev + decision core (#50054) by @ArthurZucker in #50054
LFM3
<img width="732" height="815" alt="image" src="https://github.com/user-attachments/assets/5a67fe26-2cee-4bbf-bc51-65e8f843b756" />
LFM3 is the third generation of Liquid Foundation Models from Liquid AI. Like LFM2, it is a hybrid architecture that combines gated short convolutions with grouped-query attention, with the attention layers split between full-attention and sliding-window layers. The attention layers apply RMSNorm to the query and key heads and gate their output with a per-head sigmoid. Every layer applies a sandwich norm to the operator and feed-forward outputs before the residual addition.
Links: Documentation
* [Model] LFM3 Family: LFM3-Text (#50322) by @paulpak58 in #50322
Dots 3 Note Preview
Dots 3 Note Preview is a mixture-of-experts causal language model that accepts native text, image, video, and audio inputs. It uses a shared vision encoder for images and videos and a Whisper-style audio encoder, and both encoders project their outputs into the language model's hidden space before autoregressive text generation. BF16 and FP8 inference are supported.
Links: Documentation
* Add dots3-note Preview model support (#47844) by @miraclezqc in #47844
LFM3-VL
LFM3-VL is a vision-language model family from Liquid AI, built on the LFM3 language model. It takes images and videos at their native resolution together with text. It combines a hybrid vision encoder (a linear patchifier, gated depthwise 2D convolution layers, and bidirectional grouped-query attention layers with 2D rotary embeddings) with a projector and an LFM3 language model. The projector merges each 2x2 block of patch features into one language token through a layer norm and a two-layer MLP.
Links: Documentation
* [Model] LFM3 Family: LFM3-VL (#50392) by @paulpak58 in #50392
IQuestCoder
IQuest-Coder-V1 is a family of code language models developed by IQuest Lab, available in 7B, 14B and 40B sizes with base, instruct and thinking variants. Its architecture is the same as Llama, so the Transformers auto classes load it with LlamaModel and LlamaForCausalLM, and only IQuestCoderConfig is specific to IQuestCoder. The looped IQuest-Coder-V1--Loop- checkpoints use a different architecture and aren't supported.
Links: Documentation
* Add IQuestCoder (#50579) by @hmellor in #50579
InternLM3
InternLM3 is an 8B instruction-tuned language model developed by Shanghai AI Laboratory (checkpoint: internlm/internlm3-8b-instruct). Its architecture is the same as Llama, so the auto classes load it with the Llama implementation (LlamaModel and LlamaForCausalLM), and only InternLM3Config is specific to InternLM3. The config names the attention biases qkvbias and the output projection and MLP biases bias. InternLM3Config loads these as attentionbias and mlp_bias and requires them to be equal, as they are in the released checkpoints.
Links: Documentation
* Add InternLM3 (#50578) by @hmellor in #50578
Breaking changes
🚨 Remove the eprouter expert-parallel style (#49615) by @IlyasMoutawwakil
🚨 Run MXFP4 GPT-OSS experts through epdispatch_experts (#49613) by @IlyasMoutawwakil
🚨 deepgemmmegamoe EP implementation for DeepGEMM Mega MoE (#49606) by @IlyasMoutawwakil
* 🚨 Slim down [audio] to torchaudio + soundfile + soxr (drop librosa and phonemizer) (#50610) by @eustlb
* 🚨 Deprecate Wav2Vec2ProcessorWithLM and pyctcdecode/kenlm LM decoding (#50408) by @eustlb
* 🚨 Exporters: ExportedModel, Hub save/load, one backend registry (#50029) by @IlyasMoutawwakil
* 🚨 Move processing defaults to class (#49089) by @guarin
:rotatinglight: [kernels] Bump version (#50307) by @vasqu
:rotatinglight: Finally drop rope deltas from ModelOutput! (#50153) by @zucchini-nlp
* 🚨 Safe-ize pi0 graphs (#50160) by @molbap
* 🚨 [serve][breaking] update attn for mac users (#49730) by @SunMarc
🚨 build and expand once the tp/epplan/fsdp_plan (#49527) by @3outeille
Exporters: export, share and run any model on its backend (#50029, #50032, #50309, #50311)
https://github.com/user-attachments/assets/2b17b2ab-b2b8-4e34-a983-4a7177330e4e
Every export now returns an ExportedModel. You call it like the original model (same kwargs, same output class), push it to the Hub, and load it back on the backend's own runtime. The same API covers Dynamo, ONNX Runtime, OpenVINO and ExecuTorch.
``python
from transformers import AutoModel, AutoTokenizer
from transformers.exporters import ExportedModel, OnnxConfig, OnnxExporter
model = AutoModel.from_pretrained("google/embeddinggemma-2")
tokenizer = AutoTokenizer.from_pretrained("google/embeddinggemma-2")
inputs = tokenizer(["Exporters are here!"], return_tensors="pt")
exported = OnnxExporter().export(model, inputs, OnnxConfig(dynamic=True, ort_fusions="cuda"))
exported.pushtohub("my-org/embeddinggemma-2-onnx")
exportedmodel = ExportedModel.frompretrained("my-org/embeddinggemma-2-onnx", device="cuda")
outputs = exportedmodel(inputs) # same output class as model(**inputs)
`
- Fused for the runtime: OnnxConfig(ort_fusions="cpu" |
Patch release v5.22.22026-11-06
Patch release v5.22.2
Some in between patch for security and distributed fixes. See the included commits :hugs:
- [Distributed] Set device_id only for NCCL (#50281) by @michaelbenayoun and @3outeille
- Harden our Jinja sandboxing (#50431) by @Rocketknight1
Full Changelog: https://github.com/huggingface/transformers/compare/v5.22.1...v5.22.2
Patch release v5.22.12026-10-30
Patch release v5.22.1
We just had to fix the documentation!
** [Fix] Remove WIP doc for Ml4 (https://github.com/huggingface/transformers/pull/50124) from @remi-or
v5.22.0: Mistral Large 4 (Le Chonk)2026-10-29
New Models
Mistral Large 4 (Le Chonk)
<img width="1329" height="780" alt="image" src="https://github.com/user-attachments/assets/1cff9ca7-19e3-4622-8db6-fb1cfa842579" />
Mistral Large 4 is Mistral's largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.
Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, it shows especially good performance on workloads such as cybersecurity, finance, and law, and it surpasses frontier closed models on visual grounding. It is natively fluent in 160+ languages, including every official language of the European Union.
Key Features
Mistral Large 4 includes the following architectural choices:
- ~1T total / ~52B active parameters: granular Mixture-of-Experts with 128 routed experts (4 active per token) and 1 shared expert.
- 1M context length, for long-context workloads.
- Multimodal input: accepts text and image input, with text output. Pixtral-style vision encoder trained to handle variable image sizes and aspect ratios.
- Instruct and Reasoning functionalities with function calls, reasoning effort configurable per request.
Mistral Large 4 offers the following capabilities:
- Reasoning Mode: three reasoning efforts — low, medium, high (default) — so the same model can answer a quick chat reply or work through a complex agentic run.
- Vision: analyzes images, charts, technical drawings, and documents, with state-of-the-art visual grounding among open weights.
- Multilingual: natively fluent in 160+ languages, including all official languages of the European Union.
- System Prompt: strong adherence and support for system prompts.
- Agentic: native function calling, JSON output, and strict JSON schema enforcement.
- Cybersecurity: state of the art among open-weight models for security work, while refusing malicious requests.
- Large Context Window: supports a 1M context window.
The model is released under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.
Recommended Settings
- Reasoning Effort:
- 'low' / 'medium' / 'high' — default is 'high'.
- Use reasoning_effort="high" for complex prompts and agentic usage, 'low' for fast chat replies and 'medium' for day to day usage.
- Sampling: for all reasoning efforts, we recommend the following sampling parameters:
- temperature=1.0
- top_p=0.95
- System Prompt: define a clear environment and use case, including guidance on how to effectively leverage tools in agentic systems.
- Tools: keep the set of tools well-defined and limit their number to the minimum required for the use case.
- Context: the checkpoint supports a 1M context window.
Breaking changes
* 🚨 Separate EP logic into its own file (#49679) by @3outeille
Quantization
FP8 quantized models can now be used for adapter training (such as LoRA with PEFT). Backward passes were added to the dense and grouped FP8 matmuls, and is_trainable is now True. Backward also dequantizes experts in chunks and skips autograd at inference, which cuts peak memory in the MoE layer backward from about 28.9 GiB to 2.8 GiB at 8k tokens. Separately, the AMD CI image now install
* FP8 backward: chunked expert dequant, skip autograd.Function at inference (#50117) by @ArthurZucker in [#50117]
* Train adapters on top of FP8 weights (#50031) by @qgallouedec in [#50031]
* [AMD CI] Install onnxscript, bitsandbytes and peft in the AMD image (#50046) by @Abdennacer-Badaoui in [#50046]
Bugfixes and improvements
* Processor unnecessary warnings (#49812) by @zucchini-nlp in [#49812]
* Fix boolean padding in ClapFeatureExtractor (#23648) (#49960) by @rudy-07 in [#49960]
* [Fix] Count cropping of an uninitialized layer as a no-op (#50104) by @remi-or in [#50104]
* [loading] Allow PermuteForRope to act on single tensors while renaming (#50092) by @Cyrilvallez in [#50092]
* Add Nemotron 3.5 Transcribe (#50084) by @eustlb in [#50084]
* [Nemotron3Diarization] add left context support (#49283) by @eustlb in [#49283]
* [Janus] Fix processor iterating a string prompt character by character (#50044) by @tarekziade in [#50044]
fix(config): forward token kwarg to getconfigdict in PreTrainedConfig.frompretrained (#50081) by @ydshieh in [#50081]
Fix ropescaling setter for configs that assign it during init (#50080) by @hmellor in [#50080]
[timesfm] Fix fp16 overflow on padval in forwardtransform (#50021) by @tarekziade in [#50021]
* Update dev version (#50068) by @albertvillanova in [#50068]
* Pin the shared Hub MIC fleet dataset (#49802) by @tarekziade in [#49802]
* [docs] NVIDIA/Intel/CPU (#49650) by @stevhliu in [#49650]
* Document how to handle non-persistent buffer initialization in custom models (#49880) by @Tialo in [#49880]
* Remove legacy benchmark/ and disable automatic benchmark runs (#50040) by @tarekziade in [#50040]
* Remove deprecation cycle for candidate generators (#50034) by @Cyrilvallez in [#50034]
* Fix test pil import leak (#49915) by @guarin in [#49915]
* fix: preserve NumPy integer label dtypes in default data collators (#49900) by @BlazeSR10hawk in [#49900]
* [ESMFold2] Don't build the MSA encoder for checkpoints without one (#49890) by @jimburtoft in [#49890]
[minimaxm3_vl] Gate the integration tests on total accelerator memory (#50025) by @tarekziade in [#50025]
* Fix: Add kwargs in LFM2.5 conv layers (#50022) by @younesbelkada in [#50022]
Forward coderevision to every pipeline loader (#49832) by @Rocketknight1 in [#49832]
* [moshi] Skip Moshiko integration tests whose checkpoint is gone (#50017) by @tarekziade in [#50017]
* Keep a capturing group's escapes when reversing a conversion pattern (#49772) by @IlyasMoutawwakil in [#49772]
* perf(ci): skip the lazy blob fetch in git blame on a partial clone (#50007) by @nrenvo in [#50007]
* fix(canary): set layerdrop=0.0 in test tester to prevent flaky test (#50012) by @ydshieh in [#50012]
* add a WA to Dockerfile for XPU (#50008) by @kaixuanliu in [#50008]
pipelines: defer eager model imports in texttoaudio and tokenclassification (#50001) by @ydshieh in [#50001]
[exporters] resolve @registerpatch paths lazily to avoid eager model imports (#50000) by @ydshieh in [#50000]
* [Fix] Remove useless kwargs (#49966) by @remi-or in [#49966]
* Fix model's config attr naming (#49898) by @zucchini-nlp in [#49898]
Significant community contributions
The following contributors have made significant changes to the library over the last release:
* @tarekziade
Fix initweights gaps in 7 audio models and enable testcaninitallmissingweights for them (#49386)
* Fix image backend equivalence tests for Pillow 13 (#50105)
* [Janus] Fix processor iterating a string prompt character by character (#50044)
[timesfm] Fix fp16 overflow on padval in forwardtransform (#50021)
* Pin the shared Hub MIC fleet dataset (#49802)
* Remove legacy benchmark/ and disable automatic benchmark runs (#50040)
[minimaxm3_vl] Gate the integration tests on total accelerator memory (#50025)
* [moshi] Skip Moshiko integration tests whose checkpoint is gone (#50017)
* @eustlb
* Add Nemotron 3.5 Transcribe (#50084)
* [Nemotron3Diarization] add left context support (#49283)
Patch release v5.21.12026-10-29
Patch release v5.21.1
Fix ropescaling setter for configs that assign it during init (#50080 )
We broke remote code loading in vllm, this fixes it sorry everyone!
Release: v5.21.02026-10-26
Release v5.21.0
New Model additions
MiniCPM-OCR
MiniCPM-OCR is a document-understanding model from OpenBMB's MiniCPM-V family. It keeps the MiniCPM-V 4.6 architecture, which pairs a SigLIP vision encoder and a window-attention merger with a Qwen3.5 language model. It is specialized for reading text-dense images such as scanned pages, screenshots, receipts and tables.
Links: Documentation | Paper
* Feat test (#49920) by @guarin in #49920
Flint
<img width="3200" height="1800" alt="flintbenchmarksnvfp4" src="https://github.com/user-attachments/assets/c45955fb-dfcc-4266-8223-b0d70efb9a5b" />
Flint is Recursive's mixture-of-experts language model. Compared to a standard SwiGLU MoE transformer such as Qwen3-MoE, it uses hybrid attention (sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers). It also has learned attention sinks, as in GPT-OSS, with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score and adds an ungated shared expert, and the first two layers are dense.
Links: Documentation
* Add Flint model support. (#49824) by @lovishmadaan in #49824
UltraVBert
UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view. The connector merges each 4 x 4 block of vision patches into one token, so a tile becomes 64 image tokens, and RMS-normalizes them to the scale of UltraBERT's token embeddings. The maximagetiles setting (27 by default) caps the number of tiles per page.
Links: Documentation
* Add UltraVBERT (#49685) by @athrael-soju in #49685
Gradium Phonon TTS
Gradium Phonon TTS is Gradium's family of on-device text-to-speech models, trained with the Pocket TTS recipe. A causal transformer reads a voice prompt and the text, then emits one continuous 32-dimensional audio latent per step at 12.5 Hz through a one-step flow head, and an end-of-speech head decides when the utterance ends. The latents are decoded to 24 kHz audio by a Pocket TTS audio tokenizer. Unlike Pocket TTS, voice conditioning uses 512-dimensional latents from a separate, unreleased speaker codec, so voices come as precomputed latents and cannot be cloned from a recording.
Links: Documentation
* Add GradiumPhononTTS (#49791) by @eustlb in #49791
Pocket TTS
Pocket TTS is Kyutai's 100M-parameter text-to-speech model, built as a Continuous Audio Language Model that runs faster than real time on CPU. A causal transformer emits one continuous audio latent per frame (12.5 Hz) through a one-step flow head, and an end-of-speech head decides when to stop. A Mimi-based audio tokenizer encodes voice prompts and decodes the generated latents to 24 kHz audio. Two flow heads are released, Lagrangian self-distillation (default) and drifting, and voices can be cloned from a recording, passed as precomputed latents, or loaded from cached KV states.
Links: Documentation | Paper. It is the only paper the documentation links, and it describes the architecture family Pocket TTS belongs to. The docs do not cite a separate Pocket TTS technical report.) | Blog Post
* add PocketTTS (#49589) by @eustlb in #49589
Pocket TTS Audio Tokenizer
The Pocket TTS audio tokenizer is the codec used by Pocket TTS. It has Mimi's architecture (SEANet convolutions plus a small causal transformer on each side), but its residual vector quantizer is replaced by a continuous 32-dimensional bottleneck, the VAE-GAN from Continuous Audio Language Models. It converts 24 kHz audio into latents at 12.5 Hz and decodes them back to audio. It serves as the audio_tokenizer of PocketTTSProcessor, encoding voice prompts and decoding generated latents, and its encoder and decoder can also be loaded on their own.
Links: Documentation | Paper)
* add PocketTTSAudioTokenizer (#49588) by @eustlb in #49588
Breaking changes
Fixed Gemma4 processor imports (the Gemma4, Gemma4 unified and EmbeddingGemma2 processors) so from transformers import * no longer fails in environments that have PIL but not torch; no user action is needed.
:rotatinglight: Fix gemma4 cross imports resulting in import failing without torch but with PIL (#49748) by @Sai-Suraj-27
Added an "fp8" backend to QuantizedCache, alongside quanto and hqq, that stores KV states in torch.float8e4m3fn with a single scale per tensor; enable it with cacheimplementation="quantized" and cache_config={"backend": "fp8"}.
* 🚨Add an FP8 backend to QuantizedCache (#48677) by @jiqing-feng
Tensor parallelism now has a dedicated embedding style, shards every model's input embeddings by default, and supports widths that aren't divisible by the TP size, so ColwiseParallel and RowwiseParallel now apply only to linear layers; custom TP plans that applied them to embeddings must switch to the new embedding style.
* 🚨 TP: give embeddings their own style, shard input embeddings by default, and shard widths that don't divide by the TP size (#49570) by @IlyasMoutawwakil
Every MoE TP and EP plan now shards its experts, so models no longer keep all experts on every rank, and a new testmoeparallelplansshard_experts test enforces this; custom or downstream MoE parallel plans must add expert sharding to pass it and to avoid wasted memory.
* 🚨 MoE: shard the experts in every TP/EP plan, and test for it (#49206) by @IlyasMoutawwakil
Cache
This release improves cache handling across the library. Flash Attention now works with static caches, BLOOM no longer crashes with static cache or compiled generate, chunked prefill supports cache continuation, and the Qwen4-Exp indexer's position history stays aligned with its cached keys. Linear-attention caches also gain new getters (isrecordingpast, getrecurrentstate, conv-state access) that stop exposing cache internals and prepare for paged-c
* [FA] Fix FA with static cache (#49998) by @Cyrilvallez in [#49998]
* [docs] CB cache behavior (#49025) by @stevhliu in [#49025]
* [CB] Reduce warmup time by not computing attention over full cache (#49853) by @remi-or in [#49853]
* Fix Qwen4-Exp indexer position cache lifecycle (#49840) by @taking-lying-flat in [#49840]
* [Cache] Linear attn cache: use new getters (#49375) by @remi-or in [#49375]
* [Cache] Get conv states (#49309) by @remi-or in [#49309]
[Cache] Add getrecurrent_state (#49306) by @remi-or in [#49306]
* [Cache] Ad
Patch release v5.20.22026-10-19
Patch release: v5.20.2
This patch notably solves an issue where some inputs could loose their original type inside prepareinputsfor_generation in generate, leading to problems with multimodal models. It also fixes some other issues listed on the below commits.
It contains the following commits:
- Fix inputs dict casting in generate (#49731) by @Cyrilvallez
- Fix zero expert gradients in bfloat16 with expert parallelism and epsize < fsdpsize (#49688) by @qgallouedec
- Declare the input modalities of the VLMs that report text only (#49677) by @qgallouedec
- Avoid a host-to-device copy in eager_mask (#49684) by @hmellor
Release v5.20.12026-10-16
Release v5.20.1
Special surprise including the new model Beam from Reflection AI :partying_face:
<img width="2912" height="1440" alt="GeminiGeneratedImage_ox4hjlox4hjlox4h" src="https://github.com/user-attachments/assets/bfc7ff16-4340-490c-925c-b1512e4ceb32" />
Beam is a token-efficient, frontier-scale model focused on coding and agentic use cases. It is built for teams that want to run a powerful model on their own hardware, infrastructure, or cloud, and customize it to their specific needs.
Links: Documentation
* Fix small bug in auto mappings (#49664) by @yonigozlan in #49664
Some real patch fixes
We broke loading models on older torch versions :cry: should be fixed now. But here is the full list of fixes
- Fix HunYuan-VL image assignment during batched generation (#49557) by @Arnav66692
- Fix DeepGEMM Mega MoE under expert parallelism(#49428) by @IlyasMoutawwakil
- Fix from_pretrained on torch 2.7: require torch>=2.8 for distributed checkpointing (#49652) by @naveecode
- Make DtensorShardOperation work on torch 2.6 and 2.7 (#49657) by @hazelxue
Full Changelog: https://github.com/huggingface/transformers/compare/v5.20.0...v5.20.1
Release v5.20.02026-10-15
Release v5.20.0
New Model additions
Muse Spark Open
<img width="2400" height="1296" alt="image" src="https://github.com/user-attachments/assets/bf34ea27-0edf-4ded-ba6f-905b04096126" />
Muse Spark Open is a multimodal mixture-of-experts model from Meta that takes text, images, audio, and video as input and generates text. It has 816B total parameters (42B active per token) and a context window of up to 1,048,576 tokens shared across modalities. It is trained for agentic coding tasks such as multi-file editing, debugging, and tool use. Checkpoints are available in BF16 and NVFP4 formats.
Links: Documentation | Blog Post
* Add Muse Spark Open (#49644) by @guarin in #49644
Ling 3.0
Ling 3.0 Flash is a hybrid Mixture-of-Experts language model from InclusionAI. Its decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer. The MLA blocks use interleaved rotary embeddings and a head-wise output gate, and the feed-forward blocks combine routed and shared experts. It is natively supported in Transformers as BailingHybridForCausalLM, so it loads without trustremotecode=True.
Links: Documentation
* [Model] Add native Ling 3.0 Flash support (#49446) by @icavan in #49446
Kolibri1
Kolibri 1 is Aleph Alpha's mixture-of-experts reasoning model, focused on German and English. Each of its 50 layers routes every token to 6 of 384 experts using sigmoid routing, and adds one shared expert. Four sliding-window attention layers alternate with one full-attention layer, and RoPE is applied only in the sliding-window layers.
Links: Documentation
* Add Kolibri 1 (#49463) by @lucamotz in #49463
MiniCPM-V 4.7
MiniCPM-V is a series of efficient multimodal large language models developed by OpenBMB. Like MiniCPM-V 4.6, the MiniCPM-V 4.7 architecture pairs a SigLIP vision encoder that has a window-attention merger with a Qwen3.5 language model backbone, and supports both 4x and 16x visual downsampling modes.
The main addition over 4.6 is canvas M-RoPE: instead of numbering visual tokens along a single 1-D sequence, the model lays every image out on a 2-D canvas and assigns each visual token a (temporal, height, width) position, so slices of the same image keep their spatial relationship and video frames keep their temporal order.
* Fix more stuff (#48979) by @zucchini-nlp in #48979
UnlimitedOcr
UnlimitedOcr is a single 3B-parameter OCR model from Baidu with a standard 32,768-token context length. It builds on DeepSeek OCR and replaces all decoder attention layers with Reference Sliding Window Attention (R-SWA). This keeps the KV cache constant during decoding, so the model can transcribe dozens of pages in a single forward pass. It supports "gundam" mode (640x640 tiles with dynamic cropping) and "base" mode (a single 1024x1024 view), and it can output region detections with bounding boxes.
Links: Documentation | Paper)
* Add Unlimited OCR (#46836) by @guarin in #46836
UltraBERT
UltraBERT is a family of bidirectional text encoders trained with a combination of masking ratios, pruning, and distillation for classification and retrieval tasks. The released encoders support sequences of up to 32,768 tokens. They use rotary position embeddings, RMSNorm, SwiGLU feed-forward layers, and alternating local and global attention (every third layer is global). Checkpoints include UltraBERT-150M and UltraBERT-380M for masked language modeling and fine-tuning, and UltraBERT-Embed-150M for text embeddings.
Links: Documentation | Paper
* Integrate the UltraBERT architecture (#49596) by @tomaarsen in #49596
MOSS-Transcribe-Diarize
MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, speaker diarization, and timestamps. It combines a Whisper-style encoder, a 4× frame merge step, a multi-modal projector, and a Qwen3 language model. It produces joint transcription and diarization in a single pass, with segments formatted as [start][S01]text[end]. It supports long-form audio through Whisper-window chunking in the processor, promptable transcription with hotword biasing, and 50+ languages.
Links: Documentation
* Add MOSS-Transcribe-Diarize model (#48386) by @itazap in #48386
Breaking changes
🚨 General breaking notice that inference with EP has regressions in speed; we are working on fixing this ASAP! 🚨
Deprecated checks and methods have been removed from cacheutils.py (affecting models such as minimaxm3_vl and pi0), so any code still calling those deprecated cache methods must switch to the current cache API.
* 🚨 Remove deprecated cache utils methods (#49605) by @guarin
Accidental imports from other models were removed from several model files (e.g. embedding_gemma2, ijepa, lightglue), and the modular converter now avoids adding such cross-model imports. If you imported these names through those model modules, import them from their original modules instead.
* 🚨 Remove imports from other models (#49473) by @guarin
The minimum kernels version is now 0.17.2, which allows use_fallback=KernelizeFallback.ALL so that a failed kernel load falls back to the original implementation instead of raising an error. Upgrade kernels to 0.17.2 or later.
* 🚨 Bump min kernels to 0.17.2 for better fallback (#49235) by @SunMarc
A new SubConfigSpec centralizes the initialization of sub-configs, so default sub-configs are collected from dataclass fields instead of by instantiating the class, and this touches the configuration utilities and many model configs. Custom or existing nested configs may need to be updated to declare their sub-configs through this mechanism.
:rotatinglight: SubConfigSpec for centralized init of subconfigs (#48282) by @zucchini-nlp
The LUKE and mLUKE tokenizers now treat entity special tokens such as <ent> as special tokens (not split, honored by skipspecialtokens, normalized) while keeping the same spacing as regular vocabulary tokens when decoding. Tokenization and decoding output involving these tokens may therefore differ from before, so check any code or saved outputs that depend on the old behavior.
* 🚨🚨🚨 luke fixe (#48408) by @itazap
Exporters
Transformers exporters now support quantization directly during export across Dynamo, ONNX, OpenVINO, and ExecuTorch. The new unified quantization API integrates PT2E, ONNX Runtime, and NNCF quantizers, enabling static and dynamic quantization as well as weight-only compression (INT8/INT4), with configurable calibration workflows. This makes it easier to export optimized models for efficient inference across different backends.
For more
Release v5.19.02026-10-06
Release v5.19.0
New Model additions
EmbeddingGemma2
<img width="2716" height="2308" alt="image" src="https://github.com/user-attachments/assets/84734e74-163d-4d12-b166-ffcf6749d563" />
EmbeddingGemma 2 is a multimodal embedding model from Google built on the Gemma 4 architecture. It encodes text, images, audio, and video, individually or combined in one input, into a shared 768-dimensional vector space for cross-modal retrieval, semantic similarity, clustering, and classification. It uses Matryoshka Representation Learning, so embeddings can be truncated to 512, 256, or 128 dimensions. It also offers configurable visual and video token budgets, and unused vision or audio towers can be disabled at load time to save memory.
Links: Documentation
* Smthn smthn (#49364) by @vasqu in #49364
Breaking changes
All MoE models whose routers compute logits now return them when outputrouterlogits=True, following the Qwen3-MoE pattern (a router_logits recorder on the base model, MoeModelOutputWithPast from the backbone, and a MoE causal LM output from the head), so code that relied on the previous outputs or their absence should read the router logits from these output classes.
* 🚨 Return router logits from every MoE model that computes them (#48920) by @qgallouedec
Owlv2ForObjectDetection.embedimagequery now selects the query box with the highest objectness score, as in the original OWLv2 notebook, instead of the OWL-ViT heuristic, so image-guided query embeddings and detections may differ from earlier releases.
* 🚨 Select the OWLv2 image query by objectness (#49200) by @qgallouedec
The "paged|" prefix for SDPA and flash attention implementations is deprecated, so users should set the regular attention implementation (e.g. sdpa or flashattention2) for continuous batching instead of paged|sdpa or paged|flashattention2.
* 🚨 Attention 🚨 Deprecate "paged|" prefix for SDPA and flash (#49112) by @remi-or
The regular flash and SDPA attention functions (flashattention.py, sdpaattention.py) now support continuous batching directly, and "paged|..." implementations for these are redirected to them, while eager still requires the "paged|eager" prefix.
* 🚨 Attention 🚨 Make regular attention support CB (#49101) by @remi-or
In continuous batching, the cache update for the index-based and block-table paths is now fused into a single call, which slightly changes the cache update function's behavior and affects any custom code that calls the separate update paths.
* 🚨 [CB] 🚨 Fuse update for index and block table path (#49088) by @remi-or
Continuous batching internals changed in preparation for removing "paged": max
* 🚨 [CB] 🚨 Little fixes before removing "paged" (#49069) by @remi-or
Parallelization
Expert parallelism gains a token-dispatch implementation, selected via the new epdispatchexperts plan rule and now the default for Qwen3 MoE and Mellum, which removes the requirement that EP size equal TP size. The Trainer was also adapted to work with expert parallelism, and the docs now note that PEFT adapters support tensor parallelism. A CI-related fix for pipeline-parallel chart2table inference was also included.
* [distributed]: adapt Trainer to work Expert Parallel (#48873) by @3outeille in [#48873]
* [distributed] Add expert-parallel token dispatch, default for Qwen3 MoE (#48865) by @3outeille in [#48865]
* Fix pp chart2table inference (#49225) by @zucchini-nlp in [#49225]
* [docs] TP for PEFT adapters (#49060) by @stevhliu in [#49060]
Cache
This release fixes quantized cache handling: generate no longer mutates the user's cacheconfig, and QuantizedLayer.reordercache is repaired. It also adds per-layer cache configuration, so DynamicCache and StaticCache initialize each layer from its own config (sliding window, attention chunk size, conv states, and attention head counts) to better support heterogeneous models. Separately, the deprecation cycle on mask and cache
* Fix quantized cache (#48700) by @jiqing-feng in [#48700]
[CI] checkbad_commit: use EFS cache to avoid Xet FUSE OOM (exit 137) (#49273) by @ydshieh in [#49273]
* Support per-layer cache configuration (#48178) by @eladsegal in [#48178]
* Remove deprecation cycle on mask and cache (#49231) by @Cyrilvallez in [#49231]
Bugfixes and improvements
* Revert "[CI] Temporarily disable AMD scheduled CI trigger" (#49271) (#49362) by @ydshieh in [#49362]
* ALM testx fIxing... (#49355) by @zucchini-nlp in [#49355]
* Fix DataCollatorForLanguageModeling ignoring seed=0 (#49349) by @akanyaani in [#49349]
* Fix explicit kernelization modes for evaluation models (#49339) by @DimensionSTP in [#49339]
fix(coherecompass): add video to input_modalities (#49320) (#49323) by @destopianpirate in [#49323]
* GPT-OSS: read the clamped SwiGLU's alpha and limit from the config (#49346) by @IlyasMoutawwakil in [#49346]
[distributed] Default MoE epplans to token dispatch (#49160) by @3outeille in [#49160]
* Move CI to Python 3.11, with the version set in .python-version (#49311) by @tarekziade in [#49311]
* Bump doc-builder main docs workflow pin (#49350) by @paulinebm in [#49350]
[distributed] decoupled tpplan and ep_plan (#48859) by @3outeille in [#48859]
* [distributed] Add dense and expert device mesh (#48857) by @3outeille in [#48857]
* Bump doc-builder workflow pin (#49342) by @paulinebm in [#49342]
* Allow registering new processing backends (#48984) by @zucchini-nlp in [#48984]
* Fix WatermarkDetector repeated-ngram counting (#49315) by @LE0-Lin in [#49315]
* Fix minicpm 4.6V video batching and CI (#49259) by @zucchini-nlp in [#49259]
remove colwisegatheroutput for lmhead for DeepseekV4 (#49314) by @3outeille in [#49314]
* Fix the labels docstring of CLIPSegForImageSegmentation (#49001) by @qgallouedec in [#49001]
* [CI] Temporarily disable AMD scheduled CI trigger (#49271) by @ydshieh in [#49271]
* Remove the unused pandas entry from setup.py deps (#49329) by @tarekziade in [#49329]
Fix runglue.py label ids becoming strings with pandas>=3 (#49326) by @tarekziade in [#49326]
* [Fix] Remove old test file with two ancient tests (#49280) by @remi-or in [#49280]
* Fix chunked prefill with multi-axis position ids (Qwen3-VL) (#49319) by @dacorvo in [#49319]
* Why override when you can fix in Mixin (#49105) by @zucchini-nlp in [#49105]
* Do RoPE with mul instead of matmul (#49265) by @Rocketknight1 in [#49265]
* Deprecated pipelines should redirect; removed pipelines should raise (#49308) by @LysandreJik in [#49308]
Enable model test on upcoming torchtpu backend (#49207) by @tengomucho in [#49207]
* Fix encoder repetition penalty source rows after batch expansion (#49262) by @gitedmond in [#49262]
* Preserve masked EOS scores in exponential decay length penalty (#49260) by @gitedmond in [#49260]
* [CI] Avoid fetching all artifacts when only specific ones are needed (#49282) by @ydshieh in [#49282]
Revert "[CI] ssh-runner: add optional cachetype input to switch between bucket and EFS runners (#49159)" (#49270) by @ydshieh in [#49270]
Add Trainer.lossisscaledforga to declare whether computeloss already scales for gradient accumulation (#49240) by @qgallouedec in [#49240]
* fix precomputed flash kwargs clashing (#47928) by @IlyasMoutawwakil in [#47928]
* Reapply modular examples (#49230) by @Cyrilvallez in [#49230]
* Remove some stuff that does not exist (#49232) by @Cyrilvallez in [#49232]
* Remove rotaries deprecation cycle (#49228) by @Cyrilvallez in [#49228]
* [GTE`] Add memory mixin (#49234) by @vasqu in [#49234]
* [CB] Make CB more device agnostic and add support for XPU (#49156) by @remi-or in [#49156]
* correct final metrics due to resumi