A Swift package that provides a drop-in replacement for Apple's Foundation Models framework with support for custom language model providers. All you need to do is change your import statement:
- import FoundationModels
+ import AnyLanguageModelimport AnyLanguageModel
struct WeatherTool: Tool {
let name = "getWeather"
let description = "Retrieve the latest weather information for a city"
@Generable
struct Arguments {
@Guide(description: "The city to fetch the weather for")
var city: String
}
func call(arguments: Arguments) async throws -> String {
"The weather in \(arguments.city) is sunny and 72°F / 23°C"
}
}
let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
Prompt("How's the weather in Cupertino?")
}
print(response.content)- Apple Foundation Models
- Core ML models
- MLX models
- llama.cpp (GGUF models)
- Ollama HTTP API
- Anthropic Messages API
- OpenAI Chat Completions API
- OpenAI Responses API
- Swift 6.1+
- iOS 17.0+ / macOS 14.0+ / visionOS 1.0+
Add this package to your Package.swift:
dependencies: [
.package(url: "https://github.com/mattt/AnyLanguageModel.git", from: "0.2.0")
]AnyLanguageModel uses Swift 6.1 traits to conditionally include heavy dependencies, allowing you to opt-in only to the language model backends you need. This results in smaller binary sizes and faster build times.
Available traits:
CoreML: Enables Core ML model support (depends onhuggingface/swift-transformers)MLX: Enables MLX model support (depends onml-explore/mlx-swift-examples)Llama: Enables llama.cpp support (requiresmattt/llama.swift)
By default, no traits are enabled. To enable specific traits, specify them in your package's dependencies:
// In your Package.swift
dependencies: [
.package(
url: "https://github.com/mattt/AnyLanguageModel.git",
branch: "main",
traits: ["CoreML", "MLX"] // Enable CoreML and MLX support
)
]Uses Apple's system language model (requires macOS 26 / iOS 26 / visionOS 26 or later).
let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model)
let response = try await session.respond {
Prompt("Explain quantum computing in one sentence")
}Run Core ML models
(requires CoreML trait):
let model = CoreMLLanguageModel(url: URL(fileURLWithPath: "path/to/model.mlmodelc"))
let session = LanguageModelSession(model: model)
let response = try await session.respond {
Prompt("Summarize this text")
}Enable the trait in Package.swift:
.package(
url: "https://github.com/mattt/AnyLanguageModel.git",
branch: "main",
traits: ["CoreML"]
)Run MLX models on Apple Silicon
(requires MLX trait):
let model = MLXLanguageModel(modelId: "mlx-community/Qwen3-0.6B-4bit")
let session = LanguageModelSession(model: model)
let response = try await session.respond {
Prompt("What is the capital of France?")
}Enable the trait in Package.swift:
.package(
url: "https://github.com/mattt/AnyLanguageModel.git",
branch: "main",
traits: ["MLX"]
)Run GGUF quantized models via llama.cpp
(requires Llama trait):
let model = LlamaLanguageModel(modelPath: "/path/to/model.gguf")
let session = LanguageModelSession(model: model)
let response = try await session.respond {
Prompt("Translate 'hello world' to Spanish")
}Enable the trait in Package.swift:
.package(
url: "https://github.com/mattt/AnyLanguageModel.git",
branch: "main",
traits: ["Llama"]
)Supports both Chat Completions and Responses APIs:
let model = OpenAILanguageModel(
apiKey: ProcessInfo.processInfo.environment["OPENAI_API_KEY"]!,
model: "gpt-4o-mini"
)
let session = LanguageModelSession(model: model)
let response = try await session.respond {
Prompt("Write a haiku about Swift")
}For OpenAI-compatible endpoints that use older Chat Completions API:
let model = OpenAILanguageModel(
baseURL: URL(string: "https://api.example.com")!,
apiKey: apiKey,
model: "gpt-4o-mini",
apiVariant: .chatCompletions
)Uses the Messages API with Claude models:
let model = AnthropicLanguageModel(
apiKey: ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]!,
model: "claude-sonnet-4-5-20250929"
)
let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
Prompt("What's the weather like in San Francisco?")
}Run models locally via Ollama's HTTP API:
// Default: connects to http://localhost:11434
let model = OllamaLanguageModel(model: "qwen3")
// Custom endpoint
let model = OllamaLanguageModel(
endpoint: URL(string: "http://remote-server:11434")!,
model: "llama3.2"
)
let session = LanguageModelSession(model: model)
let response = try await session.respond {
Prompt("Tell me a joke")
}First, pull the model: ollama pull qwen3:0.6b
Run the test suite to verify everything works correctly:
swift testTests for different language model backends have varying requirements:
- CoreML tests:
swift test --enable-trait CoreML+ENABLE_COREML_TESTS=1+HF_TOKEN(downloads model from HuggingFace) - MLX tests:
swift test --enable-trait MLX+ENABLE_MLX_TESTS=1+HF_TOKEN(uses pre-defined model) - Llama tests:
swift test --enable-trait Llama+LLAMA_MODEL_PATH(points to local GGUF file) - Anthropic tests:
ANTHROPIC_API_KEY(no traits needed) - OpenAI tests:
OPENAI_API_KEY(no traits needed) - Ollama tests: No setup needed (skips in CI)
Example setup for all backends:
# Environment variables
export ENABLE_COREML_TESTS=1
export ENABLE_MLX_TESTS=1
export HF_TOKEN=your_huggingface_token
export LLAMA_MODEL_PATH=/path/to/model.gguf
export ANTHROPIC_API_KEY=your_anthropic_key
export OPENAI_API_KEY=your_openai_key
# Run all tests with traits enabled
swift test --enable-trait CoreML --enable-trait MLX --enable-trait Llama