A high-performance proxy server that accelerates Nix dependency retrieval across your local network by caching and serving packages locally.

• Overview
• Problem & Solution
• Key Features
• How It Works
• Quick Start
• Installation
• Configuration
• High Availability Deployments
• Client Setup
• Troubleshooting
• Contributing

ncps acts as a local binary cache for Nix, fetching store paths from upstream caches (like cache.nixos.org) and storing them locally. This reduces download times and bandwidth usage, especially beneficial when multiple machines share the same dependencies.

When multiple machines running NixOS or Nix pull packages, they often download the same dependencies from remote caches, leading to:

• ❌ Redundant downloads - Each machine downloads identical files
• ❌ High bandwidth usage - Significant network traffic for large projects
• ❌ Slower build times - Network latency impacts development velocity

ncps solves these issues by acting as a centralized cache on your local network, dramatically reducing redundant downloads and improving build performance.

sequenceDiagram
    participant Client as Nix Client
    participant NCPS as ncps Server
    participant Cache as Cache Storage
    participant Upstream as Upstream Cache

    Client->>NCPS: Request store path
    NCPS->>Cache: Check cache

    alt Path exists in cache
        Cache-->>NCPS: Return cached path
        NCPS-->>Client: Serve cached path
    else Path not cached
        NCPS->>Upstream: Fetch from upstream
        Upstream-->>NCPS: Return store path
        NCPS->>Cache: Cache and sign path
        NCPS-->>Client: Serve downloaded path
    end

1. Request - Nix client requests a store path from ncps
2. Cache Check - ncps checks if the path exists in cache storage
3. Upstream Fetch - If not cached, fetches from configured upstream caches
4. Cache & Sign - Stores and signs the path with ncps private key
5. Serve - Delivers the path to the requesting client

NEW in v0.5.0: ncps now supports flexible storage backends!

graph TB
    NCPS[ncps Server]

    subgraph Database["Database Backends (Choose One)"]
        SQLite[(SQLite<br/>--cache-database-url sqlite:...)]
        Postgres[(PostgreSQL<br/>--cache-database-url postgresql:...)]
        MySQL[(MySQL/MariaDB<br/>--cache-database-url mysql:...)]
    end

    subgraph Storage["Storage Backends (Choose One)"]
        Local[Local Filesystem<br/>--cache-storage-local]
        S3[S3-Compatible Storage<br/>--cache-storage-s3-*]
    end

    subgraph S3Detail["S3-Compatible Options"]
        AWS[AWS S3]
        MinIO[MinIO]
        Other[Other S3-Compatible]
    end

    NCPS -.->|Option 1| SQLite
    NCPS -.->|Option 2| Postgres
    NCPS -.->|Option 3| MySQL
    NCPS -.->|Option A| Local
    NCPS -.->|Option B| S3
    S3 --> S3Detail

    style Database fill:#ffe1f5
    style Storage fill:#e1f5ff
    style S3Detail fill:#fff4e1
    style NCPS fill:#d4edda

Database Options:

• SQLite (default) - Embedded database for single-server deployments, no external dependencies
• PostgreSQL - Scalable relational database for production deployments
• MySQL/MariaDB - Popular open-source relational database for production deployments

Storage Options:

• Local Filesystem - Traditional file-based storage for single-server deployments
• S3-Compatible Storage - Scalable cloud storage supporting AWS S3, MinIO, and other S3-compatible services

Get ncps running quickly with Docker:

# Pull the images
docker pull alpine
docker pull kalbasit/ncps

# Create the storage volume
docker volume create ncps-storage
docker run --rm -v ncps-storage:/storage alpine /bin/sh -c \
  "mkdir -m 0755 -p /storage/var && mkdir -m 0700 -p /storage/var/ncps && mkdir -m 0700 -p /storage/var/ncps/db"

# Initialize database
docker run --rm -v ncps-storage:/storage kalbasit/ncps /bin/dbmate --url=sqlite:/storage/var/ncps/db/db.sqlite migrate up

# Start the server
docker run -d --name ncps -p 8501:8501 -v ncps-storage:/storage kalbasit/ncps \
  /bin/ncps serve \
  --cache-hostname=your-ncps-hostname \
  --cache-storage-local=/storage \
  --cache-database-url=sqlite:/storage/var/ncps/db/db.sqlite \
  --cache-upstream-url=https://cache.nixos.org \
  --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

# Pull the image
docker pull kalbasit/ncps

# Create volume for database only (cache data goes to S3)
docker volume create ncps-db
docker run --rm -v ncps-db:/db alpine mkdir -m 0700 -p /db

# Initialize database
docker run --rm -v ncps-db:/db kalbasit/ncps /bin/dbmate --url=sqlite:/db/db.sqlite migrate up

# Start the server with S3 storage
docker run -d --name ncps -p 8501:8501 -v ncps-db:/db kalbasit/ncps \
  /bin/ncps serve \
  --cache-hostname=your-ncps-hostname \
  --cache-storage-s3-bucket=my-ncps-cache \
  --cache-storage-s3-endpoint=s3.amazonaws.com \
  --cache-storage-s3-region=us-east-1 \
  --cache-storage-s3-access-key-id=YOUR_ACCESS_KEY \
  --cache-storage-s3-secret-access-key=YOUR_SECRET_KEY \
  --cache-database-url=sqlite:/db/db.sqlite \
  --cache-upstream-url=https://cache.nixos.org \
  --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

Your cache will be available at http://localhost:8501 and the public key at http://localhost:8501/pubkey.

docker pull kalbasit/ncps

docker volume create ncps-storage

docker run --rm -v ncps-storage:/storage alpine /bin/sh -c \
  "mkdir -m 0755 -p /storage/var && mkdir -m 0700 -p /storage/var/ncps && mkdir -m 0700 -p /storage/var/ncps/db"

docker run --rm -v ncps-storage:/storage kalbasit/ncps /bin/dbmate --url=sqlite:/storage/var/ncps/db/db.sqlite migrate up

docker run -d \
  --name ncps \
  -p 8501:8501 \
  -v ncps-storage:/storage \
  kalbasit/ncps \
  /bin/ncps serve \
  --cache-hostname=your-ncps-hostname \
  --cache-storage-local=/storage \
  --cache-database-url=sqlite:/storage/var/ncps/db/db.sqlite \
  --cache-upstream-url=https://cache.nixos.org \
  --cache-upstream-url=https://nix-community.cachix.org \
  --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= \
  --cache-upstream-public-key=nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=

services:
  create-directories:
    image: alpine:latest
    volumes:
      - ncps-storage:/storage
    command: >
      /bin/sh -c "
        mkdir -m 0755 -p /storage/var &&
        mkdir -m 0700 -p /storage/var/ncps &&
        mkdir -m 0700 -p /storage/var/ncps/db
      "
    restart: "no"

  migrate-database:
    image: kalbasit/ncps:latest
    depends_on:
      create-directories:
        condition: service_completed_successfully
    volumes:
      - ncps-storage:/storage
    command: >
      /bin/dbmate --url=sqlite:/storage/var/ncps/db/db.sqlite migrate up
    restart: "no"

  ncps:
    image: kalbasit/ncps:latest
    depends_on:
      migrate-database:
        condition: service_completed_successfully
    ports:
      - "8501:8501"
    volumes:
      - ncps-storage:/storage
    command: >
      /bin/ncps serve
      --cache-hostname=your-ncps-hostname
      --cache-storage-local=/storage
      --cache-database-url=sqlite:/storage/var/ncps/db/db.sqlite
      --cache-upstream-url=https://cache.nixos.org
      --cache-upstream-url=https://nix-community.cachix.org
      --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
      --cache-upstream-public-key=nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
    restart: unless-stopped

volumes:
  ncps-storage:

docker compose up -d

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ncps
  labels:
    app: ncps
    tier: proxy
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ncps
  labels:
    app: ncps
    tier: proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ncps
      tier: proxy
  template:
    metadata:
      labels:
        app: ncps
        tier: proxy
    spec:
      initContainers:
        - image: alpine:latest
          name: create-directories
          args:
            - /bin/sh
            - -c
            - "mkdir -m 0755 -p /storage/var && mkdir -m 0700 -p /storage/var/ncps && mkdir -m 0700 -p /storage/var/ncps/db"
          volumeMounts:
            - name: ncps-persistent-storage
              mountPath: /storage
        - image: kalbasit/ncps:latest # NOTE: It's recommended to use a tag here!
          name: migrate-database
          args:
            - /bin/dbmate
            - --url=sqlite:/storage/var/ncps/db/db.sqlite
            - migrate
            - up
          volumeMounts:
            - name: ncps-persistent-storage
              mountPath: /storage
      containers:
        - image: kalbasit/ncps:latest # NOTE: It's recommended to use a tag here!
          name: ncps
          args:
            - /bin/ncps
            - serve
            - --cache-hostname=ncps.yournetwork.local # TODO: Replace with your own hostname
            - --cache-storage-local=/storage
            - --cache-temp-path=/nar-temp-dir
            - --cache-database-url=sqlite:/storage/var/ncps/db/db.sqlite
            - --cache-upstream-url=https://cache.nixos.org
            - --cache-upstream-url=https://nix-community.cachix.org
            - --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
            - --cache-upstream-public-key=nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
          ports:
            - containerPort: 8501
              name: http-web
          volumeMounts:
            - name: ncps-persistent-storage
              mountPath: /storage
            - name: nar-temp-dir
              mountPath: /nar-temp-dir
      volumes:
        - name: ncps-persistent-storage
          persistentVolumeClaim:
            claimName: ncps
        - name: nar-temp-dir
          emptyDir:
            sizeLimit: 5Gi

apiVersion: v1
kind: Service
metadata:
  name: ncps
  labels:
    app: ncps
    tier: proxy
spec:
  type: ClusterIP
  ports:
    - name: http-web
      port: 8501
  selector:
    app: ncps
    tier: proxy

{
  services.ncps = {
    enable = true;
    cache.hostName = "your-ncps-hostname";
    upstream = {
      caches = [
        "https://cache.nixos.org"
        "https://nix-community.cachix.org"
      ];
      publicKeys = [
        "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
        "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
      ];
    };
  };
}

{
  services.ncps = {
    enable = true;
    cache = {
      hostName = "your-ncps-hostname";
      dataPath = "/path/to/ncps/data";
      tempPath = "/path/to/ncps/tmp"; # Introduced in NixOS 25.09
      databaseURL = "sqlite:/path/to/ncps/db/db.sqlite";
      maxSize = "50G";
      lru.schedule = "0 2 * * *"; # Clean up daily at 2 AM
      allowPutVerb = true;
      allowDeleteVerb = true;
    };
    server.addr = "0.0.0.0:8501";
    upstream = {
      caches = [
        "https://cache.nixos.org"
        "https://nix-community.cachix.org"
      ];
      publicKeys = [
        "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
        "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
      ];
    };
  };
}

go install github.com/kalbasit/ncps@latest

git clone https://github.com/kalbasit/ncps.git
cd ncps
go build .

All the flags can be set using the configuration file. See config.example.yaml for a complete reference including S3 storage configuration examples.

Note (1): Either --cache-storage-local OR S3 storage flags (see below) are required, but not both.

Use these options for S3-compatible storage instead of --cache-storage-local:

S3 Storage Examples:

ncps serve \
  --cache-hostname=ncps.example.com \
  --cache-storage-s3-bucket=my-ncps-cache \
  --cache-storage-s3-endpoint=https://s3.amazonaws.com \
  --cache-storage-s3-region=us-east-1 \
  --cache-storage-s3-access-key-id=AKIAIOSFODNN7EXAMPLE \
  --cache-storage-s3-secret-access-key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
  --cache-database-url=sqlite:/var/lib/ncps/db.sqlite \
  --cache-upstream-url=https://cache.nixos.org \
  --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

ncps serve \
  --cache-hostname=ncps.example.com \
  --cache-storage-s3-bucket=ncps-cache \
  --cache-storage-s3-endpoint=http://minio.example.com:9000 \
  --cache-storage-s3-access-key-id=minioadmin \
  --cache-storage-s3-secret-access-key=minioadmin \
  --cache-storage-s3-force-path-style \
  --cache-database-url=sqlite:/var/lib/ncps/db.sqlite \
  --cache-upstream-url=https://cache.nixos.org \
  --cache-upstream-public-key=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

Configure timeout values for upstream cache connections. Increase these values if you experience timeout errors with slow or remote upstream caches.

Common timeout values:

• 3s - Default, works for most local/fast upstreams
• 10s - Recommended for slow networks or distant upstreams
• 30s - For very slow connections (e.g., satellite, slow VPN)

Example: For slow upstream connections, increase timeouts:

ncps serve \
  --cache-hostname=ncps.example.com \
  --cache-storage-local=/var/lib/ncps \
  --cache-upstream-url=https://cache.nixos.org \
  --cache-upstream-dialer-timeout=10s \
  --cache-upstream-response-header-timeout=10s

NEW in distributed-lock branch: ncps now supports running multiple instances for high availability!

Running multiple ncps instances provides:

• ✅ Zero Downtime - Instance failures don't interrupt service
• ✅ Load Distribution - Requests spread across multiple servers
• ✅ Horizontal Scaling - Add instances to handle more traffic
• ✅ Geographic Distribution - Deploy instances closer to clients

To run ncps in high-availability mode, you need:

1. Shared Storage - All instances must access the same storage:
   • S3-compatible storage (AWS S3, MinIO, etc.), OR
   • Shared filesystem (NFS, GlusterFS, etc.) mounted on all instances
2. Shared Database - All instances must connect to the same PostgreSQL or MySQL database
   • SQLite is NOT supported in HA mode (doesn't support concurrent writes from multiple processes)
3. Redis Server - Provides distributed locking for coordination between instances

Option 1: Using S3-compatible storage

# Start Redis for distributed locking
redis-server

# Configure and start instance 1
docker run -d \
  --name ncps-1 \
  -p 8501:8501 \
  kalbasit/ncps:latest serve \
  --cache-hostname=cache.example.com \
  --cache-database-url=postgresql://user:pass@postgres:5432/ncps \
  --cache-storage-s3-bucket=ncps-cache \
  --cache-storage-s3-endpoint=https://s3.amazonaws.com \
  --cache-storage-s3-region=us-east-1 \
  --cache-redis-addrs=redis:6379

# Start instance 2 (same configuration, different port)
docker run -d \
  --name ncps-2 \
  -p 8502:8501 \
  kalbasit/ncps:latest serve \
  --cache-hostname=cache.example.com \
  --cache-database-url=postgresql://user:pass@postgres:5432/ncps \
  --cache-storage-s3-bucket=ncps-cache \
  --cache-storage-s3-endpoint=https://s3.amazonaws.com \
  --cache-storage-s3-region=us-east-1 \
  --cache-redis-addrs=redis:6379

# Set up load balancer (nginx, haproxy, etc.) to distribute traffic

Option 2: Using shared filesystem (NFS)

# Ensure NFS share is mounted on all instances at /mnt/ncps-cache

# Start instance 1
docker run -d \
  --name ncps-1 \
  -p 8501:8501 \
  -v /mnt/ncps-cache:/var/lib/ncps \
  kalbasit/ncps:latest serve \
  --cache-hostname=cache.example.com \
  --cache-database-url=postgresql://user:pass@postgres:5432/ncps \
  --cache-storage-local=/var/lib/ncps \
  --cache-redis-addrs=redis:6379

# Start instance 2 (mounting the same NFS share)
docker run -d \
  --name ncps-2 \
  -p 8502:8501 \
  -v /mnt/ncps-cache:/var/lib/ncps \
  kalbasit/ncps:latest serve \
  --cache-hostname=cache.example.com \
  --cache-database-url=postgresql://user:pass@postgres:5432/ncps \
  --cache-storage-local=/var/lib/ncps \
  --cache-redis-addrs=redis:6379

Note: If --cache-redis-addrs is not provided, ncps runs in single-instance mode using local locks.

ncps uses Redis to coordinate multiple instances:

• Download Deduplication - Only one instance downloads each package, others wait and share the result
• LRU Coordination - Only one instance runs cache cleanup at a time
• Retry with Backoff - Failed lock acquisitions retry with exponential backoff and jitter
• Lock Expiry - Locks automatically expire (via TTL) to prevent deadlocks

This prevents duplicate downloads and ensures cache consistency across all instances.

See the distributed locking documentation for:

• OpenTelemetry metrics for lock operations
• Monitoring dashboards
• Troubleshooting guide
• Performance tuning

First, retrieve the public key from your running ncps instance:

curl http://your-ncps-hostname:8501/pubkey

Add ncps to your configuration.nix:

nix.settings = {
  substituters = [
    "http://your-ncps-hostname:8501"  # Use https:// if behind reverse proxy
    "https://cache.nixos.org"
    # ... other substituters
  ];

  trusted-public-keys = [
    "your-ncps-hostname=<paste-public-key-here>"
    "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
    # ... other keys
  ];
};

Edit your nix.conf file (typically /etc/nix/nix.conf or ~/.config/nix/nix.conf):

substituters = http://your-ncps-hostname:8501 https://cache.nixos.org
trusted-public-keys = your-ncps-hostname=<paste-public-key-here> cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

Contributions are welcome! We appreciate bug reports, feature requests, documentation improvements, and code contributions.

1. Clone the repository:

   git clone https://github.com/kalbasit/ncps.git
   cd ncps

2. Start development server:

   # Using local filesystem storage (default, no dependencies required)
   ./dev-scripts/run.sh
   
   # Using S3/MinIO storage (requires dependencies)
   nix run .#deps  # In a separate terminal
   ./dev-scripts/run.sh s3

3. Make your changes and test:

   # Format code
   nix fmt
   
   # Run linter with auto-fix
   golangci-lint run --fix
   
   # Run tests with race detector
   go test -race ./...

4. Submit a pull request

For detailed contribution guidelines, development workflow, code quality standards, and testing procedures, please see CONTRIBUTING.md.

• 📖 Read the Contributing Guide
• 🐛 Report bugs via issues
• 💬 Ask questions in discussions
• 📧 Contact maintainers for security issues

This project is licensed under the MIT License - see the LICENSE file for details.