Production Deployment Guide
This guide is intended for production deployments. For testing, please see a Quick Start guide for using Docker Compose. Also, docker compose could serve as a reference implementation for describing how services works together.
Components and services
Jitsu consists of several components and services. The main components are:
- 🔄 Console is responsible for management UI. Console is written in Next.js. Resources:
jitsucom/console(docker image), source code, configuration reference - 🕸️ Ingest provides events ingestion API. Ingest is written in Go. Resources:
jitsucom/ingest(docker image), source code, configuration reference - ⚙️ Rotor is responsible for running functions. Rotor is written in Node.js. Resources:
jitsucom/rotor(docker image), source code, configuration reference - 🚚 Bulker is responsible for batching and sending events to the warehouse destination. Bulker is written in Go. Resources:
jitsucom/bulker(docker image), source code, configuration reference - 🔌 Syncctl is responsible for running Jitsu Connectors on Kubernetes cluster. Resources:
jitsucom/syncctl(docker image), source code, configuration reference.- Syncctl is optional. It is required only if you want to use Jitsu Connectors sync feature. Otherwise, you can skip it's configuration. Please note, that Syncctl requires a Kubernetes cluster to run syncs
All Jitsu services are:
- Available as Docker images
- Stateless and can be scaled horizontally
- Are being configured with environment variables exclusively
In addition to internal services, jitsu has a number of 3rd-party dependencies
- PostgreSQL - for configuration and metadata storage
- Redis is used for multiple purposes:
- Store logs of function executions, and event cache
- Functions state storage
- Kafka is a main transport layer for sending events between components
- MongoDB is an optional. It's required for Persistent Storage and Identity Stitching
Service Diagram
See the diagram below for a better understanding of the architecture:
Enabling Jitsu Connectors
Jitsu Connectors requires Kubernetes cluster.
🔌 Syncctl is the service responsible for running connectors sync jobs. It is not required to run all Jitsu component in kubernetes cluster. It is enough to have syncctl configured with a valid config of kubernetes cluster.
How to enable
To enable connectors sync support, configure syncctl service.
For console service set SYNCS_ENABLED, SYNCCTL_URL and SYNCCTL_AUTH_KEY env variables.
After starting all Jitsu components, you will see Connectors menu item in the top bar of Jitsu Console UI.
To fill connectors catalog, you need to trigger catalog-refresh console API endpoint in browser with authorized Jitsu user:
http://JITSU_PUBLIC_URL/api/admin/catalog-refresh
Running Jitsu on Kubernetes cluster
It is totally possible to run all Jitsu components in Kubernetes cluster. In fact, Jitsu Cloud is running on Google Kubernetes Engine.
There are no official Jitsu documentation or helm chart yet, but there's a community-driven helm chart available at stafftastic/jitsu-chart
You can also use our quick-start docker-compose as a reference.
Configuration Reference
See Configuration Reference for list of all environment variables and their descriptions, use docker-compose as a reference