November Release + FP8 Training!

[shimmyshimmer]

We’re getting close to our final release of 2025! Thanks so much for sticking with us this year. We’ve got lots of new features so please update Unsloth to use the latest updates! 🦥

• Introducing FP8 Reinforcement Learning in Unsloth! Train on any and get 1.4x faster with 60% less VRAM performance: Read our Blog/Guide
• DeepSeek-OCR fine-tuning is here! We fine-tuned DeepSeek-OCR, improving its language understanding by 89%. Read our Blog • Free notebook
• Qwen3-VL models supported including GGUFs to run locally: Blogpost + fixes • GGUFs
• We analyzed RL training-inference mismatch for FP16 vs. BF16 and concluded that Unsloth does not have this issue: Analysis and Results
• We’ve partnered with Docker to let you run LLMs locally with zero setup. Docker GGUFs are now powered by Unsloth Dynamic.
  Example: docker model run hf.co/unsloth/gpt-oss-20b-GGUF:F16 Read guide
• Baidu ERNIE models are now supported. Notebooks coming soon.
• Unsloth now supports SGLang. Read our guide
• We wrote guides for LoRA Hot Swapping and vLLM Engine Arguments
• Run Kimi-K2-Thinking the most powerful open model locally. Kimi-K2 Guide
• Lots of bug fixes! See further below.

Tip

Update Unsloth via pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo
If you want PyTorch 2.9: pip install --upgrade unsloth unsloth_zoo

Bug Fixes and Enhancements

1. Supports trl>=0.25.0 and vllm>=0.11.2 and transformers>=4.57.1
2. Fixed gpt-oss GRPO, RL excessive re-compilations on torch>=2.9.0
3. Fixes Sleep mode and reduces memory usage by 5 to 15% further for RL, GRPO
4. Fix propagation of trust_remote_code = True
5. Fix Unsloth offloaded gradient checkpointing not offloading on 1st step - reduces VRAM by >20%
6. Add logits.detach() to GRPO to solve double backwards on some pathways
7. Add int64 kernels & fixed RoPE embeddings to allow super ultra long context training
8. Fixed 📓 OpenEnv gpt-oss RL notebook
9. DGX Spark docker image fixed

What's Changed

• Grpo gradient accumulation edits by @pluesclues in Grpo gradient accumulation edits #3390
• Nightly by @danielhanchen in Nightly #3532
• Handle TRL version compatibility in rl_replacements.py by @pluesclues in Handle TRL version compatibility in rl_replacements.py #3540
• Bug fixes by @danielhanchen in Bug fixes #3546
• Sleep trl patch by @Datta0 in Sleep trl patch #3517
• Detach logits before returning from function by @pluesclues in Detach logits before returning from function #3554
• Fix typos in comment by @mk0walsk in Fix typos in comment #3557
• Formatting & bug fixes by @danielhanchen in Formatting & bug fixes #3563
• DeepseekOCR: add trust_remote_code kwarg by @mmathew23 in DeepseekOCR: add trust_remote_code kwarg #3564
• pre-commit CI config by @djsaunde in pre-commit CI config #3565
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in [pre-commit.ci] pre-commit autoupdate #3576
• Resize rope embeddings for long sequence training by @mmathew23 in Resize rope embeddings for long sequence training #3586
• Patch in tiled mlp by @mmathew23 in Patch in tiled mlp #3584
• Support for out-of-source quantizers by @Giuseppe5 in Support for out-of-source quantizers #3534
• Fix: prevent rope_embedding AssertionError by checking kv_seq_len before reuse by @jarrycyx in Fix: prevent rope_embedding AssertionError by checking kv_seq_len before reuse #3578
• Extend TorchAOConfig to support mobile usecases by @metascroy in Extend TorchAOConfig to support mobile usecases #3587
• fix qwen3 vl gradient accumulation by @mmathew23 in fix qwen3 vl gradient accumulation #3598
• Do not force set beta to 0 for DAPO by @Datta0 in Do not force set beta to 0 for DAPO #3604
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in [pre-commit.ci] pre-commit autoupdate #3606
• Fix broken links and typo in README by @mk0walsk in Fix broken links and typo in README #3611
• remove pre-commit workflow (covered by pre-commit app) by @djsaunde in remove pre-commit workflow (covered by pre-commit app) #3618
• Add an int64 path for mlp kernels by @mmathew23 in Add an int64 path for mlp kernels #3614
• Remove grpo requirement bs=num_generations by @mmathew23 in Remove grpo requirement bs=num_generations #3609
• Enable FP8 + RL training for bf16 models by @andrewor14 in Enable FP8 + RL training for bf16 models #3440
• Fix/save torchao model loading logic by @rolandtannous in Fix/save torchao model loading logic #3621
• Fix LlamaModel_fast_forward signature to match HF Transformers (Support inputs_embeds) by @MercuryYen in Fix LlamaModel_fast_forward signature to match HF Transformers (Support inputs_embeds) #3623
• Add 128x128 PerBlock FP8 + RL by @andrewor14 in Add 128x128 PerBlock FP8 + RL #3629
• Add trust_remote_code parameter to tokenizer by @Etherll in Add trust_remote_code parameter to tokenizer #3631
• [intel] change windows to remove windows-triton for intel xpu by @leizhenyuan in [intel] change windows to remove windows-triton for intel xpu #3168

Unsloth Zoo Changes

• Bug fixes by @danielhanchen in Bug fixes unsloth-zoo#327
• Fix GRPO by @danielhanchen in Fix GRPO unsloth-zoo#328
• fix gpt oss memory calculation for intel device by @leizhenyuan in fix gpt oss memory calculation for intel device unsloth-zoo#330
• Bug fixes by @danielhanchen in Bug fixes unsloth-zoo#331
• Bug fixes by @danielhanchen in Bug fixes unsloth-zoo#332
• fixed unbound local error tokenizer-model from cache by @rolandtannous in fixed unbound local error tokenizer-model from cache unsloth-zoo#333
• Now it works on a uv venv by @kittawere in Now it works on a uv venv unsloth-zoo#336
• Gemma3n fix by @mmathew23 in Gemma3n fix unsloth-zoo#338
• [Intel] remove triton windows for intel by @leizhenyuan in [Intel] remove triton windows for intel unsloth-zoo#243
• FP8 training enhancements by @Datta0 in FP8 training enhancements  unsloth-zoo#337
• GRPO gradient accumulation steps update and DAPO support by @pluesclues in GRPO gradient accumulation steps update and DAPO support unsloth-zoo#308
• Fix/video collate by @mmathew23 in Fix/video collate unsloth-zoo#342
• Bug fixes by @danielhanchen in Bug fixes unsloth-zoo#344
• FP8, Standby and vLLM updates by @Datta0 in FP8, Standby and vLLM updates unsloth-zoo#340
• Put importance sampling into no grad by @pluesclues in Put importance sampling into no grad unsloth-zoo#343
• Detach hidden states to avoid gradient carry by @pluesclues in Detach hidden states to avoid gradient carry unsloth-zoo#345
• Bug fixes by @danielhanchen in Bug fixes unsloth-zoo#347
• MoE: Cast routing_weights dtype correctly by @mmathew23 in MoE: Cast routing_weights dtype correctly unsloth-zoo#349
• return local model in determine_base_model_source with any quantization by @noah1510 in return local model in determine_base_model_source with any quantization unsloth-zoo#334
• Enable FP8 + RL training by @andrewor14 in Enable FP8 + RL training unsloth-zoo#351
• Tiled MLP Implementation by @mmathew23 in Tiled MLP Implementation unsloth-zoo#350
• Fix gradient checkpointing layer caller kwargs by @mmathew23 in Fix gradient checkpointing layer caller kwargs unsloth-zoo#353
• vLLM weight scale FP8 and standby override by @Datta0 in vLLM weight scale FP8 and standby override unsloth-zoo#354
• Fix docstring removing regex to support empty parentheses by @noisycat3 in Fix docstring removing regex to support empty parentheses unsloth-zoo#360

Unsloth Notebooks Changes

• Feat/qwen3 vl by @Erland366 in Feat/qwen3 vl notebooks#119
• Feat/double footer fix by @Erland366 in Feat/double footer fix notebooks#121
• Add GGUF section for Qwen3-VL by @Etherll in Add GGUF section for Qwen3-VL notebooks#123
• Fix TypeError in unsloth_push_to_hub_gguf() when pushing GGUF model to Hugging Face by @samanta-sc in Fix TypeError in unsloth_push_to_hub_gguf() when pushing GGUF model to Hugging Face notebooks#125
• fix TorchAOConfig' object has no attribute 'base_config' error by @rolandtannous in fix TorchAOConfig' object has no attribute 'base_config' error notebooks#129
• Updated Dockerfile for DGX Spark by @sameersegal in Updated Dockerfile for DGX Spark notebooks#133
• gemma3-270m: reduce batch size for sample packing by @djsaunde in gemma3-270m: reduce batch size for sample packing notebooks#135
• fix dataset formatting and mapping for Magistral reasoning by @rolandtannous in fix dataset formatting and mapping for Magistral reasoning notebooks#136
• fix magistral inference by @rolandtannous in fix magistral inference notebooks#138

Full Changelog: October-2025...November-2025

What's Changed

• Grpo gradient accumulation edits by @pluesclues in Grpo gradient accumulation edits #3390
• Nightly by @danielhanchen in Nightly #3532
• Handle TRL version compatibility in rl_replacements.py by @pluesclues in Handle TRL version compatibility in rl_replacements.py #3540
• Bug fixes by @danielhanchen in Bug fixes #3546
• Sleep trl patch by @Datta0 in Sleep trl patch #3517
• Detach logits before returning from function by @pluesclues in Detach logits before returning from function #3554
• Fix typos in comment by @mk0walsk in Fix typos in comment #3557
• Formatting & bug fixes by @danielhanchen in Formatting & bug fixes #3563
• DeepseekOCR: add trust_remote_code kwarg by @mmathew23 in DeepseekOCR: add trust_remote_code kwarg #3564
• pre-commit CI config by @djsaunde in pre-commit CI config #3565
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in [pre-commit.ci] pre-commit autoupdate #3576
• Resize rope embeddings for long sequence training by @mmathew23 in Resize rope embeddings for long sequence training #3586
• Patch in tiled mlp by @mmathew23 in Patch in tiled mlp #3584
• Support for out-of-source quantizers by @Giuseppe5 in Support for out-of-source quantizers #3534
• Fix: prevent rope_embedding AssertionError by checking kv_seq_len before reuse by @jarrycyx in Fix: prevent rope_embedding AssertionError by checking kv_seq_len before reuse #3578
• Extend TorchAOConfig to support mobile usecases by @metascroy in Extend TorchAOConfig to support mobile usecases #3587
• fix qwen3 vl gradient accumulation by @mmathew23 in fix qwen3 vl gradient accumulation #3598
• Do not force set beta to 0 for DAPO by @Datta0 in Do not force set beta to 0 for DAPO #3604
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in [pre-commit.ci] pre-commit autoupdate #3606
• Fix broken links and typo in README by @mk0walsk in Fix broken links and typo in README #3611
• remove pre-commit workflow (covered by pre-commit app) by @djsaunde in remove pre-commit workflow (covered by pre-commit app) #3618
• Add an int64 path for mlp kernels by @mmathew23 in Add an int64 path for mlp kernels #3614
• Remove grpo requirement bs=num_generations by @mmathew23 in Remove grpo requirement bs=num_generations #3609
• Enable FP8 + RL training for bf16 models by @andrewor14 in Enable FP8 + RL training for bf16 models #3440
• Fix/save torchao model loading logic by @rolandtannous in Fix/save torchao model loading logic #3621
• Fix LlamaModel_fast_forward signature to match HF Transformers (Support inputs_embeds) by @MercuryYen in Fix LlamaModel_fast_forward signature to match HF Transformers (Support inputs_embeds) #3623
• Add 128x128 PerBlock FP8 + RL by @andrewor14 in Add 128x128 PerBlock FP8 + RL #3629
• Add trust_remote_code parameter to tokenizer by @Etherll in Add trust_remote_code parameter to tokenizer #3631
• [intel] change windows to remove windows-triton for intel xpu by @leizhenyuan in [intel] change windows to remove windows-triton for intel xpu #3168
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in [pre-commit.ci] pre-commit autoupdate #3634
• Float8 GRPO, RL by @danielhanchen in Float8 GRPO, RL #3640

New Contributors

• @mk0walsk made their first contribution in Fix typos in comment #3557
• @pre-commit-ci[bot] made their first contribution in [pre-commit.ci] pre-commit autoupdate #3576
• @Giuseppe5 made their first contribution in Support for out-of-source quantizers #3534
• @jarrycyx made their first contribution in Fix: prevent rope_embedding AssertionError by checking kv_seq_len before reuse #3578
• @MercuryYen made their first contribution in Fix LlamaModel_fast_forward signature to match HF Transformers (Support inputs_embeds) #3623

Full Changelog: October-2025...November-2025

---

This discussion was created from the release November Release + FP8 Training!.