A high-throughput and memory-efficient inference and serving engine for LLMs

Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek-R1, Qwen3, Gemma 3, TTS 2x faster with 70% less VRAM.

SGLang is a fast serving framework for large language models and vision language models.

🔥 MaxKB is an open-source platform for building enterprise-grade agents. MaxKB 是强大易用的开源企业级智能体平台。

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 500+ LLMs (Qwen3, Qwen3-MoE, Llama4, GLM4.5, InternLM3, DeepSeek-R1, ...) and 200+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, Llava, GLM4v, Phi4, ...) (AAAI 2025).

Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement learning for Qwen2.5, Qwen3, Llama, and more!

Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

Fully Open Framework for Democratized Multimodal Training

Qwen3 Fine-tuning: Medical R1 Style Chat

OSS RL environment + evals toolkit

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

Pytorch DTensor native training library for LLMs/VLMs with OOTB Hugging Face support

Deploy open-source LLMs on AWS in minutes — with OpenAI-compatible APIs and a powerful CLI/SDK toolkit.

Auto Thinking Mode switch for Qwen3 in Open webui

Smart proxy for LLM APIs that enables model-specific parameter control, automatic mode switching (like Qwen3's /think and /no_think), and <think> tag filtering. Perfect for using advanced models with apps that lack parameter customization.

监控nof1.ai Alpha Arena AI大模型加密货币交易行为的通知系统

Makes a improved prompts from a basic prompt

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling

记录量化LLM中的总结。