LLM inference in C/C++

Port of OpenAI's Whisper model in C/C++

Cross-platform, customizable ML solutions for live and streaming media.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

A Fast and Easy to use microframework for the web.

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

a distributed deep learning platform

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

A library of GPU kernels for sparse matrix operations.

Single-thread, end-to-end C++ implementation of the Bitnet (1.58-bit weight) model