This playbook will build an HA Kubernetes cluster with k3s, kube-vip, MetalLB, Cert-Manager and Traefik via ansible. This is an example of my user cluster that I deploy workloads to. Internal Traefik for internal workloads, External Traefik for external workloads.
Many thanks to TechnoTim for inspiration and guidance. Additional thanks to ThePCGeek for patiently answering questions and providing insight on Discord.
This is based on the work from this fork which is based on the work from 212850a/k3s-ansible which is based on the work from k3s-io/k3s-ansible. It uses kube-vip to create a load balancer for control plane, and metal-lb for its service LoadBalancer.
If you want more context on how this works, see:
π Documentation (including example commands)
πΊ Video
Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
- Debian
- Ubuntu
- CentOS
on processor architecture:
- x64
- arm64
- armhf
- Deployment environment must have Ansible 2.4.0+. If you need a quick primer on Ansible you can check out TechnoTim's docs on setting up Ansible.
serverandagentnodes should have passwordless SSH access.
First create a new directory based on the sample directory within the inventory directory:
cp -R inventory/sample inventory/my-clusterSecond, edit inventory/my-cluster/hosts.ini to match the system information gathered above
For example:
[master]
192.168.30.38
192.168.30.39
192.168.30.40
[node]
192.168.30.41
192.168.30.42
[k3s_cluster:children]
master
nodeIf multiple hosts are in the master group, the playbook will automatically set up k3s in HA mode with etcd.
This requires at least k3s version 1.19.1 however the version is configurable by using the k3s_version variable.
If needed, you can also edit inventory/my-cluster/group_vars/all.yml to match your environment.
Start provisioning of the cluster using the following command:
ansible-playbook site.yml -i inventory/my-cluster/hosts.iniAfter deployment control plane will be accessible via virtual ip-address which is defined in inventory/group_vars/all.yml as apiserver_endpoint
ansible-playbook reset.yml -i inventory/my-cluster/hosts.iniYou should also reboot these nodes due to the VIP not being destroyed
To copy your kube config locally so that you can access your Kubernetes cluster run:
scp debian@master_ip:~/.kube/config ~/.kube/configSee the commands here.
Be sure to see this post on how to troubleshoot common problems
You may want to kickstart your k3s cluster by using Vagrant to quickly build you all needed VMs with one command.
Head to the vagrant subfolder and type vagrant up to get your environment setup.
After the VMs have got build, deploy k3s using the Ansible playbook site.yml by the
vagrant provision --provision-with ansible command.
This repo is really standing on the shoulders of giants. To all those who have contributed.
Thanks to these repos for code and ideas:
