Production-ready Claude Code proxy server supporting Databricks, OpenRouter, Ollama & Azure. Features MCP integration, prompt caching & 60-80% token optimization savings.

claude-code claude-proxy anthropic-api databricks-llm openrouter-integration ollama-local llama-cpp azure-openai azure-anthropic mcp-server prompt-caching token-optimization ai-coding-assistant llm-proxy self-hosted-ai git-automation code-generation developer-tools ci-cd-automation llm-gateway cost-reduction multi-provider-llm

1. Why Lynkr?
2. Quick Start (3 minutes)
3. Overview
4. Supported AI Model Providers
5. Lynkr vs Native Claude Code
6. Core Capabilities
   • Repo Intelligence & Navigation
   • Git Workflow Enhancements
   • Diff & Change Management
   • Execution & Tooling
   • Workflow & Collaboration
   • UX, Monitoring, and Logs
7. Production-Ready Features for Enterprise Deployment
   • Reliability & Resilience
   • Observability & Monitoring
   • Security & Governance
8. Architecture
9. Getting Started: Installation & Setup Guide
10. Configuration Reference
11. Runtime Operations

• Launching the Proxy
• Connecting Claude Code CLI
• Using Ollama Models
• Hybrid Routing with Automatic Fallback
• Using Built-in Workspace Tools
• Working with Prompt Caching
• Integrating MCP Servers
• Health Checks & Monitoring
• Metrics & Observability

12. Manual Test Matrix
13. Troubleshooting
14. Roadmap & Known Gaps
15. Frequently Asked Questions (FAQ)
16. References & Further Reading
17. Community & Adoption
18. License

Claude Code CLI is locked to Anthropic's API, limiting your choice of LLM providers, increasing costs, and preventing local/offline usage.

Lynkr is a production-ready proxy server that unlocks Claude Code CLI's full potential:

• ✅ Any LLM Provider - Databricks, OpenRouter (100+ models), Ollama (local), Azure, OpenAI, llama.cpp
• ✅ 60-80% Cost Reduction - Built-in token optimization (5 optimization phases implemented)
• ✅ Zero Code Changes - Drop-in replacement for Anthropic backend
• ✅ Local & Offline - Run Claude Code with Ollama or llama.cpp (no internet required)
• ✅ Enterprise Features - Circuit breakers, load balancing, metrics, K8s-ready health checks
• ✅ MCP Integration - Automatically discover and orchestrate Model Context Protocol servers
• ✅ Privacy & Control - Self-hosted, open-source (Apache 2.0), no vendor lock-in

• 🔧 Developers who want flexibility and cost control
• 🏢 Enterprises needing self-hosted AI with observability
• 🔒 Privacy-focused teams requiring local model execution
• 💰 Cost-conscious projects seeking token optimization
• 🚀 DevOps teams wanting production-ready AI infrastructure

npm install -g lynkr

# Option A: Use local Ollama (free, offline)
export MODEL_PROVIDER=ollama
export OLLAMA_MODEL=llama3.1:8b

# Option B: Use Databricks (production)
export MODEL_PROVIDER=databricks
export DATABRICKS_API_BASE=https://your-workspace.databricks.net
export DATABRICKS_API_KEY=your-api-key

# Option C: Use OpenRouter (100+ models)
export MODEL_PROVIDER=openrouter
export OPENROUTER_API_KEY=your-api-key
export OPENROUTER_MODEL=anthropic/claude-3.5-sonnet

lynkr start
# Server running at http://localhost:8080

# Point Claude Code CLI to Lynkr
export ANTHROPIC_BASE_URL=http://localhost:8080
export ANTHROPIC_API_KEY=dummy  # Ignored by Lynkr, but required by CLI

# Start coding!
claude "Hello, world!"

Claude Code CLI now works with your chosen provider.

Next steps:

• 📖 Configuration Guide - Customize settings
• 🏭 Production Setup - Deploy to production
• 💰 Token Optimization - Enable 60-80% cost savings
• 🔧 MCP Integration - Add custom tools

This repository contains a Node.js service that emulates the Anthropic Claude Code backend so that the Claude Code CLI (or any compatible client) can operate against alternative model providers and custom tooling.

Key highlights:

• Production-ready architecture – 14 production hardening features including circuit breakers, load shedding, graceful shutdown, comprehensive metrics (Prometheus format), and Kubernetes-ready health checks. Minimal overhead (~7μs per request) with 140K req/sec throughput.
• Multi-provider support – Works with Databricks (default), Azure-hosted Anthropic endpoints, OpenRouter (100+ models), and local Ollama models; requests are normalized to each provider while returning Claude-flavored responses.
• Enterprise observability – Real-time metrics collection, structured logging with request ID correlation, latency percentiles (p50, p95, p99), token usage tracking, and cost attribution. Multiple export formats (JSON, Prometheus).
• Resilience & reliability – Exponential backoff with jitter for retries, circuit breaker protection against cascading failures, automatic load shedding during overload, and zero-downtime deployments via graceful shutdown.
• Workspace awareness – Local repo indexing, CLAUDE.md summaries, language-aware navigation, and Git helpers mirror core Claude Code workflows.
• Model Context Protocol (MCP) orchestration – Automatically discovers MCP manifests, launches JSON-RPC 2.0 servers, and re-exposes their tools inside the proxy.
• Prompt caching – Re-uses repeated prompts to reduce latency and token consumption, matching Claude's own cache semantics.
• Policy enforcement – Environment-driven guardrails control Git operations, test requirements, web fetch fallbacks, and sandboxing rules. Input validation and consistent error handling ensure API reliability.

The result is a production-ready, self-hosted alternative that stays close to Anthropic's ergonomics while providing enterprise-grade reliability, observability, and performance.

Compatibility note: Claude models hosted on Databricks work out of the box. Set MODEL_PROVIDER=openai to use OpenAI's API directly (GPT-4o, GPT-4o-mini, o1, etc.). Set MODEL_PROVIDER=azure-anthropic (and related credentials) to target the Azure-hosted Anthropic /anthropic/v1/messages endpoint. Set MODEL_PROVIDER=openrouter to access 100+ models through OpenRouter (GPT-4o, Claude, Gemini, etc.). Set MODEL_PROVIDER=ollama to use locally-running Ollama models (qwen2.5-coder, llama3, mistral, etc.).

Further documentation and usage notes are available on DeepWiki.

Lynkr supports multiple AI model providers, giving you flexibility in choosing the right model for your needs:

• Best: Claude Sonnet 4.5 (via Databricks or Azure Anthropic)
• Alternative: GPT-4o (via Azure OpenAI or OpenRouter)
• Budget: GPT-4o-mini (via Azure OpenAI) or Claude Haiku (via OpenRouter)

• Best: Claude Opus 4.5 (via Databricks or Azure Anthropic)
• Alternative: GPT-4o (via Azure OpenAI)
• Local: Qwen2.5-Coder 32B (via Ollama)

• Best: Claude Haiku (via OpenRouter or Azure Anthropic)
• Alternative: GPT-4o-mini (via Azure OpenAI)
• Local: Llama 3.1 8B (via Ollama)

• Cheapest Cloud: Amazon Nova models (via OpenRouter) - free tier available
• Cheapest Local: Ollama (any model) - completely free, runs on your hardware

When using MODEL_PROVIDER=azure-openai, you can deploy any of the models in azure ai foundry:

Note: Azure OpenAI deployment names are configurable via AZURE_OPENAI_DEPLOYMENT environment variable.

For tool calling support (required for Claude Code CLI functionality):

✅ Recommended:

• llama3.1:8b - Good balance of speed and capability
• llama3.2 - Latest Llama model
• qwen2.5:14b - Strong reasoning (larger model needed, 7b struggles with tools)
• mistral:7b-instruct - Fast and capable

❌ Not Recommended for Tools:

• qwen2.5-coder - Code-only, slow with tool calling
• codellama - Code-only, poor tool support

Lynkr supports intelligent hybrid routing for cost optimization:

# Use Ollama for simple tasks, fallback to cloud for complex ones
PREFER_OLLAMA=true
FALLBACK_ENABLED=true
FALLBACK_PROVIDER=databricks  # or azure-openai, openrouter, azure-anthropic

How it works:

• Requests with few/no tools → Ollama (free, local)
• Requests with many tools → Cloud provider (more capable)
• Ollama failures → Automatic fallback to cloud

Routing Logic:

• 0-2 tools: Ollama
• 3-15 tools: OpenRouter or Azure OpenAI (if configured)
• 16+ tools: Databricks or Azure Anthropic (most capable)

Feature Comparison for Developers and Enterprises

Scenario: 100,000 API requests/month, average 50k input tokens, 2k output tokens per request

For Developers:

• 🆓 Use free local models (Ollama) for development
• 🔧 Switch providers without code changes
• 🚀 Faster iteration with local models

For Enterprises:

• 💰 Massive cost savings (ROI: $77k-115k/year)
• 🏢 Self-hosted = data stays private
• 📊 Full observability and metrics
• 🛡️ Production-ready reliability features

For Privacy-Focused Teams:

• 🔒 Air-gapped deployments possible
• 🏠 All data stays on-premises
• 🔐 No third-party API calls required

Reduce your Claude Code costs by 60-80% in under 3 minutes:

1. ⭐ Star this repo to show support and stay updated
2. 📖 Follow the Quick Start Guide to install and configure Lynkr
3. 💬 Join the Discussion for community support
4. 🐛 Report Issues to help improve Lynkr

NEW: Lynkr now includes a comprehensive long-term memory system inspired by Google's Titans architecture, enabling persistent context across conversations and intelligent memory management.

Key Features:

• 🧠 Surprise-Based Memory Updates – Automatically extracts and stores only important, novel, or surprising information from conversations using a 5-factor heuristic scoring system (novelty, contradiction, specificity, emphasis, context switch).
• 🔍 FTS5 Semantic Search – Full-text search with Porter stemmer and keyword expansion for finding relevant memories.
• 📊 Multi-Signal Retrieval – Ranks memories using recency (30%), importance (40%), and relevance (30%) for optimal context injection.
• ⚡ Automatic Integration – Memories are extracted after each response and injected before model calls with zero latency overhead (<50ms retrieval, <100ms async extraction).
• 🎯 5 Memory Types – Tracks preferences, decisions, facts, entities, and relationships.
• 🛠️ Management Tools – memory_search, memory_add, memory_forget, memory_stats for explicit control.

Quick Start:

# Memory system is enabled by default - just use Lynkr!
# Test it:
# 1. Say: "I prefer Python for data processing"
# 2. Later ask: "What language should I use for data tasks?"
# → Model will remember your preference and recommend Python

Configuration:

MEMORY_ENABLED=true                  # Enable/disable (default: true)
MEMORY_RETRIEVAL_LIMIT=5             # Memories per request (default: 5)
MEMORY_SURPRISE_THRESHOLD=0.3        # Min score to store (default: 0.3)
MEMORY_MAX_AGE_DAYS=90              # Auto-prune age (default: 90)
MEMORY_MAX_COUNT=10000              # Max memories (default: 10000)

What Gets Remembered:

• ✅ User preferences ("I prefer X")
• ✅ Important decisions ("Decided to use Y")
• ✅ Project facts ("This app uses Z")
• ✅ New entities (first mentions of files, functions)
• ✅ Contradictions ("Actually, A not B")
• ❌ Greetings, confirmations, repeated info (filtered by surprise threshold)

Benefits:

• 🎯 Better context understanding across sessions
• 💾 Persistent knowledge stored in SQLite
• 🚀 Zero performance impact (<50ms retrieval, async extraction)
• 🔒 Privacy-preserving (all local, no external APIs)
• 📈 Scales efficiently (supports 10K+ memories)

See MEMORY_SYSTEM.md for complete documentation and QUICKSTART_MEMORY.md for usage examples.

• Fast indexer builds a lightweight SQLite catalog of files, symbols, references, and framework hints.
• CLAUDE.md summary highlights language mix, frameworks, lint configs, and dependency signals.
• Symbol search and reference lookups return definition sites and cross-file usage for supported languages (TypeScript/JavaScript/Python via Tree-sitter parsers) with heuristic fallbacks for others.
• Automatic invalidation ensures removed files disappear from search results after workspace_index_rebuild.

• Git status, diff, stage, commit, push, and pull tooling via src/tools/git.js.
• Policy flags such as POLICY_GIT_ALLOW_PUSH and POLICY_GIT_REQUIRE_TESTS enforce push restrictions or test gating.
• Diff review endpoints summarise changes and highlight risks, feeding the AI review surface.
• Release note generator composes summarized change logs for downstream publishing.

• Unified diff summaries with optional AI review (workspace_diff_review).
• Release note synthesis from Git history.
• Test harness integrates with git policies to ensure guarding before commit/push events.
• (Planned) Per-file threaded reviews and automated risk estimation (see Roadmap).

• Flexible tool execution modes: Configure where tools execute via TOOL_EXECUTION_MODE:
  • server (default) – Tools run on the proxy server where Lynkr is hosted
  • client/passthrough – Tools execute on the Claude Code CLI side, enabling local file operations and commands on the client machine
• Client-side tool execution – When in passthrough mode, the proxy returns Anthropic-formatted tool_use blocks to the CLI, which executes them locally and sends back tool_result blocks. This enables:
  • File operations on the CLI user's local filesystem
  • Local command execution in the user's environment
  • Access to local credentials and SSH keys
  • Integration with local development tools
• Tool execution pipeline sandboxes or runs tools in the host workspace based on policy (server mode).
• MCP sandbox orchestration (Docker runtime by default) optionally isolates external tools with mount and permission controls.
• Automated testing harness exposes workspace_test_run, workspace_test_history, and workspace_test_summary.
• Prompt caching reduces repeated token usage for iterative conversations.

• Lightweight task tracker (workspace_task_* tools) persists TODO items in SQLite.
• Session database (data/sessions.db) stores conversational transcripts for auditing.
• Policy web fallback fetches limited remote data when explicitly permitted.

• Pino-based structured logs with timestamps and severity.
• Request/response logging for Databricks interactions (visible in stdout).
• Session appenders log every user, assistant, and tool turn for reproducibility.
• Metrics directory ready for future Prometheus/StatsD integration.

Lynkr includes comprehensive production-hardened features designed for reliability, observability, and security in enterprise environments. These features add minimal performance overhead while providing robust operational capabilities for mission-critical AI deployments.

• Automatic retry logic for transient failures
• Configurable retry attempts (default: 3), initial delay (1s), and max delay (30s)
• Jitter prevents thundering herd problems during outages
• Intelligent retry logic distinguishes retryable errors (5xx, network timeouts) from permanent failures (4xx)

• Protects against cascading failures to external services (Databricks, Azure Anthropic)
• Three states: CLOSED (normal), OPEN (failing fast), HALF_OPEN (testing recovery)
• Configurable failure threshold (default: 5) and success threshold (default: 2)
• Per-provider circuit breaker instances with independent state tracking
• Automatic recovery attempts after timeout period (default: 60s)

• Proactive request rejection when system is overloaded
• Monitors heap usage (90% threshold), total memory (85% threshold), and active request count (1000 threshold)
• Returns HTTP 503 with Retry-After header during overload
• Cached overload state (1s cache) minimizes performance impact
• Graceful degradation prevents complete system failure

• SIGTERM/SIGINT signal handling for zero-downtime deployments
• Health check endpoints immediately return "not ready" during shutdown
• Connections drain with configurable timeout (default: 30s)
• Database connections and resources cleanly closed
• Kubernetes-friendly shutdown sequence

• Keep-alive connections reduce latency and connection overhead
• Configurable socket pools (50 max sockets, 10 free sockets)
• Separate HTTP and HTTPS agents with optimized settings
• Connection timeouts (60s) and keep-alive intervals (30s)

• High-performance in-memory metrics with minimal overhead (0.2ms per operation)
• Request counts, error rates, latency percentiles (p50, p95, p99)
• Token usage tracking (input/output tokens) and cost estimation
• Databricks API metrics (success/failure rates, retry counts)
• Circuit breaker state tracking per provider

• JSON endpoint (/metrics/observability): Human-readable metrics for dashboards
• Prometheus endpoint (/metrics/prometheus): Industry-standard format for Prometheus scraping
• Circuit breaker endpoint (/metrics/circuit-breakers): Real-time circuit breaker state

• Liveness probe (/health/live): Basic process health for Kubernetes
• Readiness probe (/health/ready): Comprehensive dependency checks
  • Database connectivity and responsiveness
  • Memory usage within acceptable limits
  • Shutdown state detection
• Returns detailed health status with per-dependency breakdown

• Request ID correlation across distributed systems (X-Request-ID header)
• Automatic request ID generation when not provided
• Structured JSON logs with request context (method, path, IP, user agent)
• Request/response timing and outcome logging
• Error context preservation for debugging

• Zero-dependency JSON schema-like validation
• Type checking (string, number, boolean, array, object)
• Range validation (min/max length, min/max value, array size limits)
• Enum validation and pattern matching
• Nested object validation with detailed error reporting
• Request body size limits and sanitization

• Consistent error response format across all endpoints
• Operational vs non-operational error classification
• 8 predefined error types (validation, authentication, authorization, not found, rate limit, external API, database, internal)
• User-friendly error messages (stack traces only in development)
• Request ID in all error responses for traceability

• Configurable filesystem path restrictions
• Command execution sandboxing (Docker runtime support)
• MCP tool isolation with permission controls
• Environment variable filtering and secrets protection

• Token budget tracking per session
• Configurable budget limits and enforcement policies
• Cost tracking and budget exhaustion handling
• Request-level cost attribution

┌────────────────────┐      ┌───────────────────────────────────────────┐
│ Claude Code CLI    │──HTTP│ Claude Code Proxy (Express API Gateway)   │
│ (or Claude client) │      │ ┌─────────────────────────────────────┐   │
└────────────────────┘      │ │ Production Middleware Stack         │   │
                            │ │ • Load Shedding (503 on overload)   │   │
                            │ │ • Request Logging (Request IDs)     │   │
                            │ │ • Metrics Collection (Prometheus)   │   │
                            │ │ • Input Validation (JSON schema)    │   │
                            │ │ • Error Handling (Consistent format)│   │
                            │ └─────────────────────────────────────┘   │
                            └──────────┬────────────────────────────────┘
                                       │
        ┌──────────────────────────────┼──────────────────────────────────┐
        │                              │                                  │
┌───────▼───────┐              ┌───────▼────────┐              ┌─────────▼────────┐
│ Orchestrator  │              │ Prompt Cache   │              │ Session Store    │
│ (agent loop)  │              │ (LRU + TTL)    │              │ (SQLite)         │
└───────┬───────┘              └────────────────┘              └──────────────────┘
        │
        │  ┌─────────────────────────────────────────────────────────────┐
        │  │ Health Checks & Metrics Endpoints                           │
        │  │ • /health/live - Kubernetes liveness probe                  │
        └──│ • /health/ready - Readiness with dependency checks          │
           │ • /metrics/observability - JSON metrics                     │
           │ • /metrics/prometheus - Prometheus format                   │
           │ • /metrics/circuit-breakers - Circuit breaker state         │
           └─────────────────────────────────────────────────────────────┘
                                       │
        ┌──────────────────────────────┼──────────────────────────────────┐
        │                              │                                  │
┌───────▼────────────────────────────┐ │  ┌──────────────────────────────▼──┐
│ Tool Registry & Policy Engine      │ │  │ Indexer / Repo Intelligence     │
│ (workspace, git, diff, MCP tools)  │ │  │ (SQLite catalog + CLAUDE.md)    │
└───────┬────────────────────────────┘ │  └─────────────────────────────────┘
        │                              │
        │                   ┌──────────▼─────────────────────┐
        │                   │ Observability & Resilience     │
        │                   │ • MetricsCollector (in-memory) │
        │                   │ • Circuit Breakers (per-provider)│
        │                   │ • Load Shedder (resource monitor)│
        │                   │ • Shutdown Manager (graceful)  │
        │                   └──────────┬─────────────────────┘
        │                              │
┌───────▼────────┐          ┌─────────▼──────────────────────┐      ┌──────────────┐
│ MCP Registry   │          │ Provider Adapters              │      │ Sandbox      │
│ (manifest ->   │──RPC─────│ • Databricks (circuit-breaker) │──┐   │ Runtime      │
│ JSON-RPC client│          │ • Azure Anthropic (retry logic)│  │   │ (Docker)     │
└────────────────┘          │ • OpenRouter (100+ models)     │  │   └──────────────┘
                            │ • Ollama (local models)        │  │
                            │ • HTTP Connection Pooling      │  │
                            │ • Exponential Backoff + Jitter │  │
                            └────────────┬───────────────────┘  │
                                         │                      │
                        ┌────────────────┼─────────────────┐    │
                        │                │                 │    │
              ┌─────────▼────────┐  ┌───▼────────┐  ┌─────▼─────────┐
              │ Databricks       │  │ Azure      │  │ OpenRouter API│
              │ Serving Endpoint │  │ Anthropic  │  │ (GPT-4o, etc.)│
              │ (REST)           │  │ /anthropic │  └───────────────┘
              └──────────────────┘  │ /v1/messages│  ┌──────────────┐
                                    └────────────┘  │ Ollama API   │───────┘
                                         │          │ (localhost)  │
                                ┌────────▼──────────│ qwen2.5-coder│
                                │ External MCP tools└──────────────┘
                                │ (GitHub, Jira)    │
                                └───────────────────┘

graph TB
    A[Claude Code CLI] -->|HTTP POST /v1/messages| B[Lynkr Proxy Server]
    B --> C{Middleware Stack}
    C -->|Load Shedding| D{Load OK?}
    D -->|Yes| E[Request Logging]
    D -->|No| Z1[503 Service Unavailable]
    E --> F[Metrics Collection]
    F --> G[Input Validation]
    G --> H[Orchestrator]

    H --> I{Check Prompt Cache}
    I -->|Cache Hit| J[Return Cached Response]
    I -->|Cache Miss| K{Determine Provider}

    K -->|Simple 0-2 tools| L[Ollama Local]
    K -->|Moderate 3-14 tools| M[OpenRouter / Azure]
    K -->|Complex 15+ tools| N[Databricks]

    L --> O[Circuit Breaker Check]
    M --> O
    N --> O

    O -->|Closed| P{Provider API}
    O -->|Open| Z2[Fallback Provider]

    P -->|Databricks| Q1[Databricks API]
    P -->|OpenRouter| Q2[OpenRouter API]
    P -->|Ollama| Q3[Ollama Local]
    P -->|Azure| Q4[Azure Anthropic API]

    Q1 --> R[Response Processing]
    Q2 --> R
    Q3 --> R
    Q4 --> R
    Z2 --> R

    R --> S[Format Conversion]
    S --> T[Cache Response]
    T --> U[Update Metrics]
    U --> V[Return to Client]
    J --> V

    style B fill:#4a90e2,stroke:#333,stroke-width:2px,color:#fff
    style H fill:#7b68ee,stroke:#333,stroke-width:2px,color:#fff
    style K fill:#f39c12,stroke:#333,stroke-width:2px
    style P fill:#2ecc71,stroke:#333,stroke-width:2px,color:#fff

Key Components:

• src/api/router.js – Express routes that accept Claude-compatible /v1/messages requests.
• src/api/middleware/* – Production middleware stack:
  • load-shedding.js – Proactive overload protection with resource monitoring
  • request-logging.js – Structured logging with request ID correlation
  • metrics.js – High-performance metrics collection middleware
  • validation.js – Zero-dependency input validation
  • error-handling.js – Consistent error response formatting
• src/api/health.js – Kubernetes-ready liveness and readiness probes
• src/orchestrator/index.js – Agent loop handling model invocation, tool execution, prompt caching, and policy enforcement.
• src/cache/prompt.js – LRU cache implementation with SHA-256 keying and TTL eviction.
• src/observability/metrics.js – In-memory metrics collector with Prometheus export
• src/clients/circuit-breaker.js – Circuit breaker implementation for external service protection
• src/clients/retry.js – Exponential backoff with jitter for transient failure handling
• src/server/shutdown.js – Graceful shutdown manager for zero-downtime deployments
• src/mcp/* – Manifest discovery, JSON-RPC 2.0 client, and dynamic tool registration for MCP servers.
• src/tools/* – Built-in workspace, git, diff, testing, task, and MCP bridging tools.
• src/indexer/index.js – File crawler and metadata extractor that persists into SQLite and regenerates CLAUDE.md.

• Node.js 18+ (required for the global fetch API).
• npm (bundled with Node).
• Databricks account with a Claude-compatible serving endpoint (e.g., databricks-claude-sonnet-4-5).
• Optional: Docker for MCP sandboxing and tool isolation.
• Optional: Claude Code CLI (latest release). Configure it to target the proxy URL instead of api.anthropic.com.

Lynkr offers multiple installation methods to fit your workflow:

curl -fsSL https://raw.githubusercontent.com/vishalveerareddy123/Lynkr/main/install.sh | bash

This will:

• Clone Lynkr to ~/.lynkr
• Install dependencies
• Create a default .env file
• Set up the lynkr command

Custom installation directory:

curl -fsSL https://raw.githubusercontent.com/vishalveerareddy123/Lynkr/main/install.sh | bash -s -- --dir /opt/lynkr

No Ollama needed - Just use Databricks APIs directly:

# Install Lynkr
npm install -g lynkr

# Configure Databricks credentials
export DATABRICKS_API_BASE=https://your-workspace.cloud.databricks.com
export DATABRICKS_API_KEY=dapi1234567890abcdef

# Start Lynkr
lynkr

That's it! Lynkr will use Databricks Claude models for all requests.

Or use a .env file:

MODEL_PROVIDER=databricks
DATABRICKS_API_BASE=https://your-workspace.cloud.databricks.com
DATABRICKS_API_KEY=dapi1234567890abcdef
PORT=8080

For 40% faster responses and 65% cost savings on simple requests:

# Install Lynkr
npm install -g lynkr

# Run setup wizard (installs Ollama + downloads model)
lynkr-setup

# Start Lynkr
lynkr

The lynkr-setup wizard will:

• ✅ Check if Ollama is installed (auto-installs if missing on macOS/Linux)
• ✅ Start Ollama service
• ✅ Download qwen2.5-coder model (~4.7GB)
• ✅ Create .env configuration file
• ✅ Guide you through Databricks credential setup

Note: On Windows, you'll need to manually install Ollama from https://ollama.ai/download, then run lynkr-setup.

For a complete bundled experience with Ollama included:

# Clone repository
git clone https://github.com/vishalveerareddy123/Lynkr.git
cd Lynkr

# Copy environment template
cp .env.example .env

# Edit .env with your Databricks credentials
nano .env

# Start both services (Lynkr + Ollama)
docker-compose up -d

# Pull model (first time only)
docker exec ollama ollama pull qwen2.5-coder:latest

# Verify it's running
curl http://localhost:8080/health

See DEPLOYMENT.md for advanced deployment options (Kubernetes, systemd, etc.).

brew tap vishalveerareddy123/lynkr
brew install vishalveerareddy123/lynkr/lynkr

# Configure Databricks (Ollama optional)
export DATABRICKS_API_BASE=https://your-workspace.cloud.databricks.com
export DATABRICKS_API_KEY=dapi1234567890abcdef

# Start Lynkr
lynkr

Optional: Install Ollama for hybrid routing:

brew install ollama
ollama serve
ollama pull qwen2.5-coder:latest

# Clone repository
git clone https://github.com/vishalveerareddy123/Lynkr.git
cd Lynkr

# Install dependencies
npm install

# Start server
npm start

After installation, configure Lynkr by creating a .env file or exporting environment variables:

# For Databricks-only setup (no Ollama)
MODEL_PROVIDER=databricks
DATABRICKS_API_BASE=https://<your-workspace>.cloud.databricks.com
DATABRICKS_API_KEY=<personal-access-token>
PORT=8080
WORKSPACE_ROOT=/path/to/your/repo
PROMPT_CACHE_ENABLED=true

For hybrid routing with Ollama + cloud fallback, see Hybrid Routing section below.

You can copy .env.example if you maintain one, or rely on shell exports.

Set MODEL_PROVIDER to select the upstream endpoint:

• MODEL_PROVIDER=databricks (default) – expects DATABRICKS_API_BASE, DATABRICKS_API_KEY, and optionally DATABRICKS_ENDPOINT_PATH.
• MODEL_PROVIDER=azure-anthropic – routes requests to Azure's /anthropic/v1/messages endpoint and uses the headers Azure expects.
• MODEL_PROVIDER=openrouter – connects to OpenRouter for access to 100+ models (GPT-4o, Claude, Gemini, Llama, etc.). Requires OPENROUTER_API_KEY.
• MODEL_PROVIDER=ollama – connects to a locally-running Ollama instance for models like qwen2.5-coder, llama3, mistral, etc.

Azure-hosted Anthropic configuration:

MODEL_PROVIDER=azure-anthropic
AZURE_ANTHROPIC_ENDPOINT=https://<resource-name>.services.ai.azure.com/anthropic/v1/messages
AZURE_ANTHROPIC_API_KEY=<azure-api-key>
AZURE_ANTHROPIC_VERSION=2023-06-01
PORT=8080
WORKSPACE_ROOT=/path/to/your/repo

Ollama configuration:

MODEL_PROVIDER=ollama
OLLAMA_ENDPOINT=http://localhost:11434  # default Ollama endpoint
OLLAMA_MODEL=qwen2.5-coder:latest       # model to use
OLLAMA_TIMEOUT_MS=120000                # request timeout
PORT=8080
WORKSPACE_ROOT=/path/to/your/repo

Before starting Lynkr with Ollama, ensure Ollama is running:

# Start Ollama (in a separate terminal)
ollama serve

# Pull your desired model
ollama pull qwen2.5-coder:latest
# Or: ollama pull llama3, mistral, etc.

# Verify model is available
ollama list

llama.cpp configuration:

llama.cpp provides maximum performance and flexibility for running GGUF models locally. It uses an OpenAI-compatible API, making integration seamless.

MODEL_PROVIDER=llamacpp
LLAMACPP_ENDPOINT=http://localhost:8080  # default llama.cpp server port
LLAMACPP_MODEL=qwen2.5-coder-7b          # model name (for logging)
LLAMACPP_TIMEOUT_MS=120000               # request timeout
PORT=8080
WORKSPACE_ROOT=/path/to/your/repo

Before starting Lynkr with llama.cpp, ensure llama-server is running:

# Download and build llama.cpp (if not already done)
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Download a GGUF model (e.g., from HuggingFace)
# Example: Qwen2.5-Coder-7B-Instruct
wget https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF/resolve/main/qwen2.5-coder-7b-instruct-q4_k_m.gguf

# Start llama-server
./llama-server -m qwen2.5-coder-7b-instruct-q4_k_m.gguf --port 8080

# Verify server is running
curl http://localhost:8080/health

Why llama.cpp over Ollama?

Choose llama.cpp when you need maximum performance, specific quantization options, or want to use GGUF models not available in Ollama.

OpenRouter configuration:

OpenRouter provides unified access to 100+ AI models through a single API, including GPT-4o, Claude, Gemini, Llama, Mixtral, and more. It offers competitive pricing, automatic fallbacks, and no need to manage multiple API keys.

MODEL_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-v1-...                                    # Get from https://openrouter.ai/keys
OPENROUTER_MODEL=openai/gpt-4o-mini                                # Model to use (see https://openrouter.ai/models)
OPENROUTER_ENDPOINT=https://openrouter.ai/api/v1/chat/completions  # API endpoint
PORT=8080
WORKSPACE_ROOT=/path/to/your/repo

Popular OpenRouter models:

• openai/gpt-4o-mini – Fast, affordable GPT-4o mini ($0.15/$0.60 per 1M tokens)
• anthropic/claude-3.5-sonnet – Claude 3.5 Sonnet for complex reasoning
• google/gemini-pro-1.5 – Google's Gemini Pro with large context
• meta-llama/llama-3.1-70b-instruct – Meta's open-source Llama 3.1

See https://openrouter.ai/models for the complete list with pricing.

OpenAI configuration:

OpenAI provides direct access to GPT-4o, GPT-4o-mini, o1, and other models through their official API. This is the simplest way to use OpenAI models without going through Azure or OpenRouter.

MODEL_PROVIDER=openai
OPENAI_API_KEY=sk-your-openai-api-key                    # Get from https://platform.openai.com/api-keys
OPENAI_MODEL=gpt-4o                                      # Model to use (default: gpt-4o)
PORT=8080
WORKSPACE_ROOT=/path/to/your/repo

Getting an OpenAI API key:

1. Visit https://platform.openai.com
2. Sign up or log in to your account
3. Go to https://platform.openai.com/api-keys
4. Create a new API key
5. Add credits to your account (pay-as-you-go)

OpenAI benefits:

• ✅ Direct API access – No intermediaries, lowest latency to OpenAI
• ✅ Full tool calling support – Excellent function calling compatible with Claude Code CLI
• ✅ Parallel tool calls – Execute multiple tools simultaneously for faster workflows
• ✅ Organization support – Use organization-level API keys for team billing
• ✅ Simple setup – Just one API key needed

Getting an OpenRouter API key:

1. Visit https://openrouter.ai
2. Sign in with GitHub, Google, or email
3. Go to https://openrouter.ai/keys
4. Create a new API key
5. Add credits to your account (pay-as-you-go, no subscription required)

OpenRouter benefits:

• ✅ 100+ models through one API (no need to manage multiple provider accounts)
• ✅ Automatic fallbacks if your primary model is unavailable
• ✅ Competitive pricing with volume discounts
• ✅ Full tool calling support (function calling compatible with Claude Code CLI)
• ✅ No monthly fees – pay only for what you use
• ✅ Rate limit pooling across models

See src/config/index.js for the full configuration matrix, including sandbox mounts, permissions, and MCP networking policies.

# global install
lynkr start

# local checkout
npm run dev    # development: auto-restarts on file changes
npm start      # production

Logs stream to stdout. The server listens on PORT and exposes /v1/messages in the Anthropic-compatible shape. If you installed via npm globally, lynkr start reads the same environment variables described above.

1. Install or upgrade Claude Code CLI.
2. Export the proxy endpoint:

   export ANTHROPIC_BASE_URL=http://localhost:8080
   export ANTHROPIC_API_KEY=dummy # not used, but Anthropic CLI requires it

3. Launch claude CLI within WORKSPACE_ROOT.
4. Invoke commands as normal; the CLI will route requests through the proxy.

Lynkr can connect to locally-running Ollama models for fast, offline AI assistance. This is ideal for development environments, air-gapped systems, or cost optimization.

Quick Start with Ollama:

# Terminal 1: Start Ollama
ollama serve

# Terminal 2: Pull and verify model
ollama pull qwen2.5-coder:latest
ollama list

# Terminal 3: Start Lynkr with Ollama
export MODEL_PROVIDER=ollama
export OLLAMA_ENDPOINT=http://localhost:11434
export OLLAMA_MODEL=qwen2.5-coder:latest
npm start

# Terminal 4: Connect Claude CLI
export ANTHROPIC_BASE_URL=http://localhost:8080
export ANTHROPIC_API_KEY=dummy
claude

Supported Ollama Models:

Lynkr works with any Ollama model. Popular choices:

• qwen2.5-coder:latest – Optimized for code generation (7B parameters, 4.7GB)
• llama3:latest – General-purpose conversational model (8B parameters, 4.7GB)
• mistral:latest – Fast, efficient model (7B parameters, 4.1GB)
• codellama:latest – Meta's code-focused model (7B-34B variants)

Ollama Health Check:

# Basic health check
curl http://localhost:8080/health/ready

# Deep health check (includes Ollama connectivity)
curl "http://localhost:8080/health/ready?deep=true" | jq .checks.ollama

Tool Calling Support:

Lynkr now supports native tool calling for compatible Ollama models:

• ✅ Supported models: llama3.1, llama3.2, qwen2.5, qwen2.5-coder, mistral, mistral-nemo, firefunction-v2
• ✅ Automatic detection: Lynkr detects tool-capable models and enables tools automatically
• ✅ Format conversion: Transparent conversion between Anthropic and Ollama tool formats
• ❌ Unsupported models: llama3, older models (tools are filtered out automatically)

Limitations:

• Tool choice parameter is not supported (Ollama always uses "auto" mode)
• Some advanced Claude features (extended thinking, prompt caching) are not available with Ollama

Lynkr supports intelligent 3-tier hybrid routing that automatically routes requests between Ollama (local/fast), OpenRouter (moderate complexity), and cloud providers (Databricks/Azure for heavy workloads) based on request complexity, with transparent fallback when any provider is unavailable.

Why Hybrid Routing?

• 🚀 40-87% faster for simple requests (local Ollama)
• 💰 65-100% cost savings for requests that stay on Ollama
• 🎯 Smart cost optimization – use affordable OpenRouter models for moderate complexity
• 🛡️ Automatic fallback ensures reliability when any provider fails
• 🔒 Privacy-preserving for simple queries (never leave your machine with Ollama)

Quick Start:

# Terminal 1: Start Ollama
ollama serve
ollama pull qwen2.5-coder:latest

# Terminal 2: Start Lynkr with 3-tier routing
export PREFER_OLLAMA=true
export OLLAMA_ENDPOINT=http://localhost:11434
export OLLAMA_MODEL=qwen2.5-coder:latest
export OPENROUTER_API_KEY=your_openrouter_key    # Mid-tier provider
export OPENROUTER_MODEL=openai/gpt-4o-mini       # Mid-tier model
export DATABRICKS_API_KEY=your_key               # Heavy workload provider
export DATABRICKS_API_BASE=your_base_url         # Heavy workload provider
npm start

# Terminal 3: Connect Claude CLI (works transparently)
export ANTHROPIC_BASE_URL=http://localhost:8080
export ANTHROPIC_API_KEY=dummy
claude

How It Works:

Lynkr intelligently routes each request based on complexity:

1. Simple requests (0-2 tools) → Try Ollama first

   • ✅ If Ollama succeeds: Fast, local, free response (100-500ms)
   • ❌ If Ollama fails: Automatic transparent fallback to OpenRouter or Databricks

2. Moderate requests (3-14 tools) → Route to OpenRouter

   • Uses affordable models like GPT-4o-mini ($0.15/1M input tokens)
   • Full tool calling support
   • ❌ If OpenRouter fails or not configured: Fallback to Databricks

3. Complex requests (15+ tools) → Route directly to Databricks

   • Heavy workloads get the most capable models
   • Enterprise features and reliability

4. Tool-incompatible models → Route directly to cloud

   • Requests requiring tools with non-tool-capable Ollama models skip Ollama

Configuration:

# Required
PREFER_OLLAMA=true                    # Enable hybrid routing mode

# Optional (with defaults)
FALLBACK_ENABLED=true                         # Enable automatic fallback (default: true)
OLLAMA_MAX_TOOLS_FOR_ROUTING=3                # Max tools to route to Ollama (default: 3)
OPENROUTER_MAX_TOOLS_FOR_ROUTING=15           # Max tools to route to OpenRouter (default: 15)
FALLBACK_PROVIDER=databricks                  # Final fallback provider (default: databricks)
OPENROUTER_API_KEY=your_key                   # Required for OpenRouter tier
OPENROUTER_MODEL=openai/gpt-4o-mini           # OpenRouter model (default: gpt-4o-mini)

Example Scenarios:

# Scenario 1: Simple code generation (no tools)
User: "Write a hello world function in Python"
→ Routes to Ollama (fast, local, free)
→ Response in ~300ms

# Scenario 2: Moderate workflow (3-14 tools)
User: "Search the codebase, read 5 files, and refactor them"
→ Routes to OpenRouter (moderate complexity)
→ Uses affordable GPT-4o-mini
→ Response in ~1500ms

# Scenario 3: Heavy workflow (15+ tools)
User: "Analyze 20 files, run tests, update documentation, commit changes"
→ Routes directly to Databricks (complex task needs most capable model)
→ Response in ~2500ms

# Scenario 4: Automatic fallback chain
User: "What is 2+2?"
→ Tries Ollama (connection refused)
→ Falls back to OpenRouter (if configured)
→ Falls back to Databricks (if OpenRouter unavailable)
→ User sees no error, just gets response

Circuit Breaker Protection:

After 5 consecutive Ollama failures, the circuit breaker opens:

• Subsequent requests skip Ollama entirely (fail-fast)
• Fallback happens in <100ms instead of waiting for timeout
• Circuit auto-recovers after 60 seconds

Monitoring:

Track routing performance via /metrics/observability:

curl http://localhost:8080/metrics/observability | jq '.routing, .fallback, .cost_savings'

Example output:

{
  "routing": {
    "by_provider": {"ollama": 100, "databricks": 20},
    "successes_by_provider": {"ollama": 85, "databricks": 20},
    "failures_by_provider": {"ollama": 15}
  },
  "fallback": {
    "attempts_total": 15,
    "successes_total": 13,
    "failures_total": 2,
    "success_rate": "86.67%",
    "reasons": {
      "circuit_breaker": 8,
      "timeout": 4,
      "service_unavailable": 3
    }
  },
  "cost_savings": {
    "ollama_savings_usd": "1.2345",
    "ollama_latency_ms": { "mean": 450, "p95": 1200 }
  }
}

Rollback:

Disable hybrid routing anytime:

# Option 1: Disable entirely (use static MODEL_PROVIDER)
export PREFER_OLLAMA=false
npm start

# Option 2: Ollama-only mode (no fallback)
export PREFER_OLLAMA=true
export FALLBACK_ENABLED=false
npm start

Performance Comparison:

You can call tools programmatically via HTTP:

curl http://localhost:8080/v1/messages \
  -H 'Content-Type: application/json' \
  -H 'x-session-id: manual-test' \
  -d '{
    "model": "claude-proxy",
    "messages": [{ "role": "user", "content": "Rebuild the workspace index." }],
    "tools": [{
      "name": "workspace_index_rebuild",
      "type": "function",
      "description": "Rebuild the repo index and project summary",
      "input_schema": { "type": "object" }
    }],
    "tool_choice": {
      "type": "function",
      "function": { "name": "workspace_index_rebuild" }
    }
  }'

Tool responses appear in the assistant content block with structured JSON.

Lynkr supports client-side tool execution, where tools execute on the Claude Code CLI machine instead of the proxy server. This enables local file operations, commands, and access to local resources.

Enable client-side execution:

# Set in .env or export before starting
export TOOL_EXECUTION_MODE=client
npm start

How it works:

1. Model generates tool calls – Databricks/OpenRouter/Ollama model returns tool calls
2. Proxy converts to Anthropic format – Tool calls converted to tool_use blocks
3. CLI executes tools locally – Claude Code CLI receives tool_use blocks and runs them on the user's machine
4. CLI sends results back – Tool results sent back to proxy in next request as tool_result blocks
5. Conversation continues – Proxy forwards the complete conversation (including tool results) back to the model

Example response in passthrough mode:

{
  "id": "msg_123",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "I'll create that file for you."
    },
    {
      "type": "tool_use",
      "id": "toolu_abc",
      "name": "Write",
      "input": {
        "file_path": "/tmp/test.txt",
        "content": "Hello World"
      }
    }
  ],
  "stop_reason": "tool_use"
}

Benefits:

• ✅ Tools execute on CLI user's local filesystem
• ✅ Access to local credentials, SSH keys, environment variables
• ✅ Integration with local development tools (git, npm, docker, etc.)
• ✅ Reduced network latency for file operations
• ✅ Server doesn't need filesystem access or permissions

Use cases:

• Remote proxy server, local CLI execution
• Multi-user environments where each user needs their own workspace
• Security-sensitive environments where server shouldn't access user files

Supported modes:

• TOOL_EXECUTION_MODE=server – Tools run on proxy server (default)
• TOOL_EXECUTION_MODE=client – Tools run on CLI side
• TOOL_EXECUTION_MODE=passthrough – Alias for client

• Set PROMPT_CACHE_ENABLED=true (default) to activate the cache.
• The cache retains up to PROMPT_CACHE_MAX_ENTRIES entries for PROMPT_CACHE_TTL_MS milliseconds.
• A cache hit skips the Databricks call; response metadata populates cache_read_input_tokens.
• Cache misses record cache_creation_input_tokens, indicating a fresh prompt was cached.
• Cache entries are invalidated automatically when they age out; no manual maintenance required.
• Disable caching temporarily by exporting PROMPT_CACHE_ENABLED=false and restarting the server.

1. Place MCP manifest JSON files under ~/.claude/mcp or configure MCP_MANIFEST_DIRS.
2. Each manifest should define the server command, arguments, and capabilities per the MCP spec.
3. Restart the proxy; manifests are loaded at boot. Registered tools appear with names mcp_<server>_<tool>.
4. Invoke tools via workspace_mcp_call or indirectly when the assistant selects them.
5. Sandbox settings (MCP_SANDBOX_*) control Docker runtime, mounts, environment passthrough, and permission prompts.

Lynkr exposes Kubernetes-ready health check endpoints for orchestrated deployments:

curl http://localhost:8080/health/live

Returns 200 OK with basic process health. Use this for Kubernetes liveness probes to detect crashed or frozen processes.

Kubernetes Configuration:

livenessProbe:
  httpGet:
    path: /health/live
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 3

curl http://localhost:8080/health/ready

Returns 200 OK when ready to serve traffic, or 503 Service Unavailable when:

• System is shutting down
• Database connections are unavailable
• Memory usage exceeds safe thresholds

Response Format:

{
  "status": "healthy",
  "timestamp": "2024-01-15T10:30:00.000Z",
  "checks": {
    "database": {
      "healthy": true,
      "latency": 12
    },
    "memory": {
      "healthy": true,
      "heapUsedPercent": 45.2,
      "totalUsedPercent": 52.1
    }
  }
}

Kubernetes Configuration:

readinessProbe:
  httpGet:
    path: /health/ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5
  timeoutSeconds: 3
  failureThreshold: 2

Lynkr collects comprehensive metrics with minimal performance overhead (7.1μs per request). Three endpoints provide different views:

curl http://localhost:8080/metrics/observability

Returns detailed metrics in JSON format:

{
  "requests": {
    "total": 15234,
    "errors": 127,
    "errorRate": 0.0083
  },
  "latency": {
    "p50": 125.3,
    "p95": 342.1,
    "p99": 521.8,
    "count": 15234
  },
  "tokens": {
    "input": 1523421,
    "output": 823456,
    "total": 2346877
  },
  "cost": {
    "total": 234.56,
    "currency": "USD"
  },
  "databricks": {
    "requests": 15234,
    "successes": 15107,
    "failures": 127,
    "successRate": 0.9917,
    "retries": 89
  }
}

curl http://localhost:8080/metrics/prometheus

Returns metrics in Prometheus text format for scraping:

# HELP http_requests_total Total number of HTTP requests
# TYPE http_requests_total counter
http_requests_total 15234

# HELP http_request_errors_total Total number of HTTP request errors
# TYPE http_request_errors_total counter
http_request_errors_total 127

# HELP http_request_duration_seconds HTTP request latency
# TYPE http_request_duration_seconds summary
http_request_duration_seconds{quantile="0.5"} 0.1253
http_request_duration_seconds{quantile="0.95"} 0.3421
http_request_duration_seconds{quantile="0.99"} 0.5218
http_request_duration_seconds_count 15234

Prometheus Configuration:

scrape_configs:
  - job_name: 'lynkr'
    static_configs:
      - targets: ['localhost:8080']
    metrics_path: '/metrics/prometheus'
    scrape_interval: 15s

curl http://localhost:8080/metrics/circuit-breakers

Returns real-time circuit breaker states:

{
  "databricks": {
    "state": "CLOSED",
    "failureCount": 2,
    "successCount": 1523,
    "lastFailure": null,
    "nextAttempt": null
  },
  "azure-anthropic": {
    "state": "OPEN",
    "failureCount": 5,
    "successCount": 823,
    "lastFailure": "2024-01-15T10:25:00.000Z",
    "nextAttempt": "2024-01-15T10:26:00.000Z"
  }
}

For visualization, import the included Grafana dashboard (monitoring/grafana-dashboard.json) or create custom panels:

• Request rate and error rate over time
• Latency percentiles (p50, p95, p99)
• Token usage and cost tracking
• Circuit breaker state transitions
• Memory and CPU usage correlation

A Dockerfile and docker-compose.yml are included for reproducible deployments.

cp .env.example .env        # populate with Databricks/Azure credentials, workspace path, etc.
docker compose up --build

The compose file exposes:

• Proxy HTTP API on 8080
• Optional SearxNG instance on 8888 (started automatically when WEB_SEARCH_ENDPOINT is the default)

Workspace files are mounted into the container (./:/workspace), and ./data is persisted for SQLite state. If you launch the proxy outside of this compose setup you must provide your own search backend and point WEB_SEARCH_ENDPOINT at it (for example, a self-hosted SearxNG instance). Without a reachable search service the web_search and web_fetch tools will return placeholder responses or fail.

docker build -t claude-code-proxy .
docker run --rm -p 8080:8080 -p 8888:8888 \
  -v "$(pwd)":/workspace \
  -v "$(pwd)/data":/app/data \
  --env-file .env \
  claude-code-proxy

Adjust port and volume mappings to suit your environment. Ensure the container has access to the target workspace and required credentials.

docker run --rm -p 8080:8080 \
  -v "$(pwd)":/workspace \
  -v "$(pwd)/data":/app/data \
  -e MODEL_PROVIDER=databricks \
  -e DATABRICKS_API_BASE=https://<workspace>.cloud.databricks.com \
  -e DATABRICKS_ENDPOINT_PATH=/serving-endpoints/<endpoint-name>/invocations \
  -e DATABRICKS_API_KEY=<personal-access-token> \
  -e WORKSPACE_ROOT=/workspace \
  -e PORT=8080 \
  claude-code-proxy

Use additional -e flags (or --env-file) to pass Azure Anthropic credentials or other configuration values as needed. Replace <workspace> and <endpoint-name> with your Databricks workspace host and the Serving Endpoint you want to target (e.g. /serving-endpoints/databricks-gpt-4o-mini/invocations) so you can choose any available model.

• Databricks – Mirrors Anthropic's hosted behaviour. Automatic policy web fallbacks (needsWebFallback) can trigger an extra web_fetch, and the upstream service executes dynamic pages on your behalf.
• OpenAI – Connects directly to OpenAI's API for GPT-4o, GPT-4o-mini, o1, and other models. Full tool calling support with parallel tool execution enabled by default. Messages and tools are automatically converted between Anthropic and OpenAI formats. Supports organization-level API keys. Best used when you want direct access to OpenAI's latest models with the simplest setup.
• Azure OpenAI – Connects to Azure-hosted OpenAI models. Similar to direct OpenAI but through Azure's infrastructure for enterprise compliance, data residency, and Azure billing integration.
• Azure Anthropic – Requests are normalised to Azure's payload shape. The proxy disables automatic web_fetch fallbacks to avoid duplicate tool executions; instead, the assistant surfaces a diagnostic message and you can trigger the tool manually if required.
• OpenRouter – Connects to OpenRouter's unified API for access to 100+ models. Full tool calling support with automatic format conversion between Anthropic and OpenAI formats. Messages are converted to OpenAI's format, tool calls are properly translated, and responses are converted back to Anthropic-compatible format. Best used for cost optimization, model flexibility, or when you want to experiment with different models without changing your codebase.
• Ollama – Connects to locally-running Ollama models. Tool support varies by model (llama3.1, qwen2.5, mistral support tools; llama3 and older models don't). System prompts are merged into the first user message. Response format is converted from Ollama's format to Anthropic-compatible content blocks. Best used for simple text generation tasks, offline development, or as a cost-effective development environment.
• llama.cpp – Connects to a local llama-server instance running GGUF models. Uses OpenAI-compatible API format (/v1/chat/completions), enabling full tool calling support with grammar-based generation. Provides maximum performance with optimized C++ inference, lower memory usage through quantization, and support for any GGUF model from HuggingFace. Best used when you need maximum performance, specific quantization options, or models not available in Ollama.
• In all cases, web_search and web_fetch run locally. They do not execute JavaScript, so pages that render data client-side (Google Finance, etc.) will return scaffolding only. Prefer JSON/CSV quote APIs (e.g. Yahoo chart API) when you need live financial data.

• path must be a non-empty string errors – Tool calls like fs_read require explicit paths. Verify the CLI sent a valid path argument.
• Agent loop exceeding limits – Increase POLICY_MAX_STEPS or fix misbehaving tool that loops.
• spawn npm test ENOENT – Configure WORKSPACE_TEST_COMMAND or ensure npm test exists in the workspace.
• MCP server not discovered – Confirm manifests live inside MCP_MANIFEST_DIRS and contain executable commands. Check logs for discovery errors.
• Prompt cache not activating – Ensure PROMPT_CACHE_ENABLED=true. Cache only stores tool-free completions; tool use requests bypass caching by design.
• Claude CLI prompts for missing tools – Verify tools array in the client request lists the functions you expect. The proxy only exposes registered handlers.
• Dynamic finance pages return stale data – web_fetch fetches static HTML only. Use an API endpoint (e.g. Yahoo Finance chart JSON) or the Databricks-hosted tooling if you need rendered values from heavily scripted pages.

• "No choices in OpenRouter response" errors – OpenRouter sometimes returns error responses (rate limits, model unavailable) with JSON but no choices array. As of the latest update, Lynkr gracefully handles these errors and returns proper error responses instead of crashing. Check logs for "OpenRouter response missing choices array" warnings to see the full error details.
• Multi-prompt behavior with certain models – Some OpenRouter models (particularly open-source models like openai/gpt-oss-120b) may be overly cautious and ask for confirmation multiple times before executing tools. This is model-specific behavior. Consider switching to:
  • anthropic/claude-3.5-sonnet – More decisive tool execution
  • openai/gpt-4o or openai/gpt-4o-mini – Better tool calling behavior
  • Use Databricks provider with Claude models for optimal tool execution
• Rate limit errors – OpenRouter applies per-model rate limits. If you hit limits frequently, check your OpenRouter dashboard for current usage and consider upgrading your plan or spreading requests across multiple models.

• 503 Service Unavailable errors during normal load – Check load shedding thresholds (LOAD_SHEDDING_*). Lower values may trigger too aggressively. Check /metrics/observability for memory usage patterns.
• Circuit breaker stuck in OPEN state – Check /metrics/circuit-breakers to see failure counts. Verify backend service (Databricks/Azure) is accessible. Circuit will automatically attempt recovery after CIRCUIT_BREAKER_TIMEOUT (default: 60s).
• "Circuit breaker is OPEN" errors – The circuit breaker detected too many failures and is protecting against cascading failures. Wait for timeout or fix the underlying issue. Check logs for root cause of failures.
  • Azure OpenAI specific: If using Azure OpenAI and seeing circuit breaker errors, verify your AZURE_OPENAI_ENDPOINT includes the full path (including /openai/deployments/YOUR-DEPLOYMENT/chat/completions). Missing endpoint variable or undefined returns can trigger circuit breaker protection.
• High latency after adding production features – This is unexpected; middleware adds only ~7μs overhead. Check /metrics/prometheus for actual latency distribution. Verify network latency to backend services.
• Health check endpoint returns 503 but service seems healthy – Check individual health check components in the response JSON. Database connectivity or memory issues may trigger this. Review logs for specific health check failures.
• Metrics endpoint shows incorrect data – Metrics are in-memory and reset on restart. For persistent metrics, configure Prometheus scraping. Check that METRICS_ENABLED=true.
• Request IDs not appearing in logs – Ensure REQUEST_LOGGING_ENABLED=true. Check that structured logging is configured correctly in src/logger.js.
• Validation errors on valid requests – Check request body against schemas in src/api/middleware/validation.js. Validation is strict by design. Review error details in 400 response.
• Graceful shutdown not working – Ensure process receives SIGTERM (not SIGKILL). Check GRACEFUL_SHUTDOWN_TIMEOUT is sufficient for your workload. Kubernetes needs proper terminationGracePeriodSeconds.
• Prometheus scraping fails – Verify /metrics/prometheus endpoint is accessible. Check Prometheus configuration targets and metrics_path. Ensure firewall rules allow scraping.

Run the included benchmarks to verify performance:

# Run comprehensive test suite
node comprehensive-test-suite.js

# Run performance benchmarks
node performance-benchmark.js

Expected results:

• Test pass rate: 100% (80/80 tests)
• Combined middleware overhead: <10μs per request
• Throughput: >100K requests/second

If performance is degraded:

1. Check /metrics/observability for latency patterns
2. Review memory usage (should be <200MB for typical workload)
3. Check circuit breaker states (stuck OPEN states add latency)
4. Verify backend API latency (primary bottleneck)
5. Review logs for retry patterns (excessive retries indicate backend issues)

Production Hardening (All 14 features implemented with 100% pass rate):

• ✅ Exponential backoff with jitter retry logic
• ✅ Circuit breaker pattern for external services
• ✅ Load shedding with resource monitoring
• ✅ Graceful shutdown for zero-downtime deployments
• ✅ HTTP connection pooling
• ✅ Comprehensive metrics collection (Prometheus format)
• ✅ Health check endpoints (Kubernetes-ready)
• ✅ Structured request logging with correlation IDs
• ✅ Consistent error handling with 8 error types
• ✅ Input validation (zero-dependency, JSON schema-like)
• ✅ Token budget enforcement
• ✅ Path allowlisting and sandboxing
• ✅ Rate limiting capabilities
• ✅ Safe command DSL

Latest Features (December 2025):

• ✅ Client-side tool execution (TOOL_EXECUTION_MODE=client/passthrough) – Tools can now execute on the Claude Code CLI side instead of the server, enabling local file operations, local commands, and access to local credentials
• ✅ OpenRouter error resilience – Graceful handling of malformed OpenRouter responses (missing choices array), preventing crashes during rate limits or service errors
• ✅ Enhanced format conversion – Improved Anthropic ↔ OpenRouter format conversion for tool calls, ensuring proper tool_use block generation and session consistency across providers

• Per-file diff comments & conversation threading – Planned to mirror Claude's review UX.
• Automated risk assessment tied to diffs – Future enhancement leveraging test outcomes and static analysis.
• Expanded language-server fidelity – Currently Tree-sitter-based; deeper AST integration or LSP bridging is a future goal.
• Claude Skills parity – Skills are not reproduced; designing a safe, declarative skill layer is an open area.
• Coverage dashboards & historical trends – Test summary tracks latest runs but no long-term history yet.
• Response caching – Redis-backed response cache for frequently repeated requests (Option 3, Feature 13).

export ANTHROPIC_BASE_URL=http://localhost:8080
export ANTHROPIC_API_KEY=dummy  # Required by CLI, but ignored by Lynkr
claude "Your prompt here"

export MODEL_PROVIDER=ollama
export OLLAMA_MODEL=llama3.1:8b  # or qwen2.5-coder, mistral, etc.
lynkr start

export MCP_MANIFEST_DIRS=/path/to/manifests:/another/path

export WORKSPACE_ROOT=/path/to/your/project
lynkr start

# 1. Build llama.cpp (or download pre-built binary)
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# 2. Download a GGUF model
wget https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF/resolve/main/qwen2.5-coder-7b-instruct-q4_k_m.gguf

# 3. Start the server
./llama-server -m qwen2.5-coder-7b-instruct-q4_k_m.gguf --port 8080

# 4. Configure Lynkr
export MODEL_PROVIDER=llamacpp
export LLAMACPP_ENDPOINT=http://localhost:8080
lynkr start

export MODEL_PROVIDER=openrouter
export OPENROUTER_API_KEY=sk-or-v1-...
export OPENROUTER_MODEL=openai/gpt-4o-mini
lynkr start

export MODEL_PROVIDER=openai
export OPENAI_API_KEY=sk-your-api-key
export OPENAI_MODEL=gpt-4o  # or gpt-4o-mini, o1-preview, etc.
lynkr start

• 💬 Ask on GitHub Discussions
• 🐛 Report an Issue
• 📚 Read Full Documentation

Agentic AI Systems:

• Zhang et al. (2024). Agentic Context Engineering. arXiv:2510.04618. arXiv

Long-Term Memory & RAG:

• Mohtashami & Jaggi (2023). Landmark Attention: Random-Access Infinite Context Length for Transformers. arXiv
• Google DeepMind (2024). Titans: Learning to Memorize at Test Time. arXiv

For BibTeX citations, see CITATIONS.bib.

• Claude Code CLI Documentation - Official Claude Code reference
• Model Context Protocol (MCP) Specification - MCP protocol documentation
• Databricks Foundation Models - Databricks LLM documentation
• Anthropic API Documentation - Claude API reference

• Ollama - Local LLM runtime for running open-source models
• OpenRouter - Multi-provider LLM API gateway (100+ models)
• llama.cpp - High-performance C++ LLM inference engine
• LiteLLM - Multi-provider LLM proxy (alternative approach)
• Awesome MCP Servers - Curated list of MCP server implementations

⭐ Star this repository to show your support and help others discover Lynkr!

• 🐛 Report Issues: GitHub Issues - Bug reports and feature requests
• 💬 Discussions: GitHub Discussions - Questions, ideas, and community help
• 📚 Documentation: DeepWiki - Comprehensive guides and examples
• 🔧 Contributing: CONTRIBUTING.md - How to contribute to Lynkr

Help spread the word about Lynkr:

• 🐦 Share on Twitter
• 💼 Share on LinkedIn
• 📰 Share on Hacker News
• 📱 Share on Reddit

• 💰 Massive cost savings - Save 60-80% on token costs with built-in optimization
• 🔓 Provider freedom - Choose from 7+ LLM providers (Databricks, OpenRouter, Ollama, Azure, llama.cpp)
• 🏠 Privacy & control - Self-hosted, open-source, no vendor lock-in
• 🚀 Production-ready - Enterprise features: circuit breakers, metrics, health checks
• 🛠️ Active development - Regular updates, responsive maintainers, growing community

Licensed under the Apache License, Version 2.0. See LICENSE for details.

If you find Lynkr useful, please ⭐ the repo — it helps more people discover it.