Unsloth AI

You can now run FP8 reinforcement learning on consumer GPUs! ⚡ DeepSeek-R1 demonstrated the power of FP8 GRPO. Now you can try it at home on just a 5GB GPU with Unsloth AI. • Qwen3-14B FP8 GRPO works on 24GB VRAM. Qwen3-1.7B works on 5GB. • We collabed with PyTorch TorchAO to make Unsloth FP8 RL inference via vLLM ~1.4× faster than FP16 • Unsloth uses 60% less VRAM and enables 12× longer context vs. other implementations • Works on any NVIDIA GeForce RTX 40, 50 series and H100, B200 etc. GPUs ⭐ Blog: https://lnkd.in/gC7-fpx8 Qwen3-8B FP8 GRPO Colab notebook: https://lnkd.in/gn7rpUp6

We made a guide on how to deploy LLMs locally! In collab with SGLang, you'll learn to: • Deploy fine-tuned LLMs for large scale production • Serve GGUFs for fast inference locally • Benchmark inference speed • Use on the fly FP8 for 1.6x inference ⭐ Guide: https://lnkd.in/g-zYjfjm

The future of fine-tuning is open and scalable. With Unsloth AI , NVIDIA Blackwell GPUs, and NVIDIA DGX Spark, developers can fine-tune massive LLMs locally—then scale the same workflows seamlessly into NVIDIA DGX Cloud and NVIDIA Cloud Partners for enterprise workloads. This open source project and NVIDIA engineering collaboration are redefining what’s possible in efficient AI model customization. #AI #MachineLearning #DeepLearning #LLM #GenAI #OpenSource #NVIDIA #DGXSpark #Blackwell #DGXCloud #AIDevelopment #FineTuning #CloudComputing #AIEcosystem Lee D. Brian Carpenter Paul Abruzzo Dane Aconfora NVIDIA AI 👉 Learn more:

OMG this is insane!! GG Unsloth AI Now on to obligatory follow-up: 1 billion wen? 😅

Run IBM's new Granite-4.0 series of open models! Run the 'Micro' 3B model on 4GB RAM or 'Small' 32B on 40GB RAM. Granite-4.0 excels at agentic tasks, document summary/analysis, RAG, edge AI applications and more! Unsloth Dynamic GGUFs: https://lnkd.in/gAMYXCsq Granite-4.0 Guide: https://lnkd.in/g2zejNaW Thank you to the IBM team for providing Unsloth with Day Zero support!

We made Reinforcement Learning (RL) work for OpenAI gpt-oss! ✨ We also achieved the fastest inference for gpt-oss RL vs. all implementations! Train gpt-oss-20b with GRPO locally on just 15GB VRAM or use our free notebook. Unsloth gpt-oss RL uses -50% VRAM with 8× longer context with 0 accuracy loss vs. all setups. ⭐Blog & Findings: https://lnkd.in/d6dhnf7Q

We're teaming up with Mistral AI and NVIDIA for an Unsloth event on Tues, Oct 21 at Y Combinator's office! 🦥 Join us in San Francisco for a night of talks, Unsloth AI merch and more. Food & drinks provided. RSVP required! ⭐ lu.ma/unsloth-yc

We're excited to be doing our first ever AMA on r/LocalLlama tomorrow 10AM–1PM PST. 🦥 🦙 Ask any question about kernels, our dynamic GGUFs, bug fixes, RL, fine-tuning, tips & tricks or anything! Join us tomorrow, Wednesday, Sept 11th, 10AM–1PM PST.