AI Agent Framework for Self-Hosted LLMs — deploy on your infrastructure, keep data private.

🎯 Built for companies and developers who need:

• 100% on-premise AI agents (no data leaves your network)
• Any OpenAI-compatible LLM (vLLM, Ollama, llama.cpp, text-generation-webui)
• Production-ready security (battle-tested by 1500+ hackers)
• Simple deployment (docker compose up and you're done)

Unlike cloud-dependent solutions, LocalTopSH runs entirely on your infrastructure:

• ✅ Russia, Belarus, Iran — sanctions don't apply to self-hosted
• ✅ China — no Great Firewall issues
• ✅ Air-gapped networks — zero internet required
• ✅ On-premise data centers — full compliance

# Option A: vLLM (recommended for production)
vllm serve gpt-oss-120b --api-key dummy --port 8000

# Option B: Ollama (easy setup)
ollama serve  # Default port 11434

# Option C: llama.cpp (minimal resources)
llama-server -m your-model.gguf --port 8000

git clone https://github.com/vakovalskii/topsha
cd topsha

# Run setup script (creates directories with correct permissions)
./setup.sh

# Edit secrets with your credentials
nano secrets/telegram_token.txt  # Your bot token from @BotFather
nano secrets/base_url.txt        # http://your-llm-server:8000/v1
nano secrets/api_key.txt          # Your API key (or "dummy" if not needed)
nano secrets/model_name.txt       # Model name (e.g. gpt-oss-120b)
nano secrets/zai_api_key.txt      # Z.AI search key (optional)

Alternative (manual setup):

# Create directories
mkdir -p secrets workspace workspace/_shared
chmod -R 777 workspace  # Important: Docker containers need write access

# Create secrets
echo "your-telegram-token" > secrets/telegram_token.txt
echo "http://your-llm-server:8000/v1" > secrets/base_url.txt
echo "dummy" > secrets/api_key.txt
echo "changeme123" > secrets/admin_password.txt
chmod 600 secrets/*.txt

docker compose up -d

# Check status
docker compose ps

# View logs
docker compose logs -f

• Telegram Bot: Message your bot
• Admin Panel: http://localhost:3000 (login: admin / password from secrets/admin_password.txt)
• API: http://localhost:4000/api

# Change default admin password (REQUIRED for production!)
echo "your-secure-password" > secrets/admin_password.txt

# Optionally change admin username via environment variable
# Edit docker-compose.yml and set ADMIN_USER=your_username

# Rebuild admin container
docker compose up -d --build admin

⚠️ Default credentials: admin / changeme123 — change them before exposing to network!

┌─────────────────────────────────────────────────────────────────────────────────┐
│                           YOUR INFRASTRUCTURE                                    │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  ┌─────────────────┐     ┌─────────────────┐     ┌─────────────────────────────┐│
│  │   Telegram      │     │   LocalTopSH    │     │   Your LLM Backend          ││
│  │   (optional)    │────▶│   Agent Stack   │────▶│   ────────────────────────  ││
│  └─────────────────┘     │                 │     │   vLLM / Ollama / llama.cpp ││
│                          │  ┌───────────┐  │     │   gpt-oss-120b              │���
│  ┌─────────────────┐     │  │   core    │  │     │   Qwen-72B                  ││
│  │   Admin Panel   │────▶│  │  (agent)  │  │     │   Llama-3-70B               ││
│  │   :3000         │     │  └───────────┘  │     │   Mistral-22B               ││
│  └─────────────────┘     │        │        │     │   Your fine-tuned model     ││
│                          │        ▼        │     └─────────────────────────────┘│
│                          │  ┌───────────┐  │                                    │
│                          │  │  sandbox  │  │     No data leaves your network!  │
│                          │  │ (per-user)│  │                                    │
│                          │  └───────────┘  │                                    │
│                          └─────────────────┘                                    │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘

🔥 Stress-tested by 1500+ hackers in @neuraldeepchat

Attack attempts: Token extraction, RAM exhaustion, container escapes

Result: 0 secrets leaked, 0 downtime

# Run security doctor (46 checks)
python scripts/doctor.py

# Run E2E tests (10 checks)
python scripts/e2e_test.py --verbose

echo "http://vllm-server:8000/v1" > secrets/base_url.txt
echo "dummy" > secrets/api_key.txt
echo "gpt-oss-120b" > secrets/model_name.txt

echo "http://ollama:11434/v1" > secrets/base_url.txt
echo "ollama" > secrets/api_key.txt
echo "llama3:70b" > secrets/model_name.txt

echo "http://your-server:8000/v1" > secrets/base_url.txt
echo "your-api-key" > secrets/api_key.txt
echo "your-model-name" > secrets/model_name.txt

Web panel at :3000 for managing the system (protected by Basic Auth):

# Default credentials
Username: admin
Password: (from secrets/admin_password.txt, default: changeme123)

# Change password
echo "your-secure-password" > secrets/admin_password.txt
docker compose up -d --build admin

# Change username (optional)
# In docker-compose.yml, set environment variable:
# ADMIN_USER=your_username

Admin panel is bound to 127.0.0.1:3000 for security. For remote access:

# On your local machine
ssh -L 3000:localhost:3000 user@your-server

# Then open http://localhost:3000 in browser

• Internal AI assistant with full data privacy
• Code review bot that never leaks proprietary code
• Document analysis without sending files to cloud

• Experiment with open models (Llama, Mistral, Qwen)
• Fine-tuned model deployment with agent capabilities
• Reproducible AI workflows in isolated environments

• Russia/Belarus/Iran — no API access restrictions
• China — no Great Firewall issues
• Air-gapped networks — military, government, finance

• High-volume workloads — pay for GPU, not per-token
• Predictable costs — no surprise API bills
• Scale without limits — your hardware, your rules

Symptom: 500 Internal Server Error when changing access mode, or errors like:

Permission denied: '/workspace/_shared/admin_config.json'
Permission denied: '/workspace/123456789/'

Cause: Docker containers (running as root) can't write to bind-mounted workspace/ directory.

Solution:

# Fix permissions on host
chmod -R 777 workspace/

# Or run setup script again
./setup.sh

Prevention: Always use ./setup.sh for initial setup — it creates directories with correct permissions.

Symptom: Bot shows online but doesn't reply to messages.

Diagnosis:

1. Check if core container is running: docker ps | grep core
2. Check logs: docker logs core --tail 50
3. Verify access mode: Admin panel → Config → Access

Common causes:

• Access mode set to admin_only but ADMIN_USER_ID not configured
• Bot token invalid or expired
• LLM API unreachable (check base_url in secrets)

Solution:

# Set admin user ID
echo "YOUR_TELEGRAM_ID" > .env
# Add line: ADMIN_USER_ID=123456789

# Or set access mode to public (for testing)
docker compose restart core

Symptom: docker ps shows containers constantly restarting.

Diagnosis:

docker logs core --tail 100
docker logs bot --tail 100

Common causes:

• Missing or invalid secrets (telegram_token, api_key, base_url)
• LLM server not accessible
• Port conflicts (3000, 4000, 4001 already in use)

Solution:

1. Verify all required secrets exist and are valid
2. Test LLM connection: curl http://your-llm-server:8000/v1/models
3. Check port availability: netstat -tuln | grep -E '3000|4000|4001'

We believe in building real infrastructure, not hacks.

MIT

• Architecture: ARCHITECTURE.md — detailed system design
• Security: SECURITY.md — security model and patterns
• Telegram: @neuraldeepchat