LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

MLX: An array framework for Apple silicon

Tensor library for machine learning

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba. Full multimodal LLM Android App:[MNN-LLM-Android](./apps/Android/MnnLlmChat/READ…

lightweight, standalone C++ inference engine for Google's Gemma models.

Diffusion model(SD,Flux,Wan,Qwen Image,...) inference in pure C/C++

⚠️DirectML is in maintenance mode ⚠️ DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tas…

General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous and optimized for…

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Low-bit LLM inference on CPU/NPU with lookup table

Suno AI's Bark model in C/C++ for fast text-to-speech generation

C++ implementation of Qwen-LM

Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just like Ollama - but purpose-built and deeply optimized for the AMD NPUs.

Swift library to work with llama and other large language models.

Vulkan中文教程 | 自撰 | 面向对象

LLM inference in C/C++