An opinionated list of awesome Python frameworks, libraries, software and resources.

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthr…

Reverse engineered ChatGPT API

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.

🐢 Open-Source Evaluation & Testing library for LLM Agents

Bias Auditing & Fair ML Toolkit

DSPydantic: Auto-Optimize Your Prompts and Pydantic Models with DSPy

📸 Open-Source Evaluation & Testing for Computer Vision AI systems

Phare is a LLM benchmark that evaluates models across key AI security & safety dimensions

A lightweight library that orchestrates LLM completions and agents in parallel workflows

A practical tool for managing async pipelines

Giskard LLM utils