Cloud Native Distributed Storage in Kubernetes with Longhorn
Storage in Kubernetes is hard, complicated, and messy.Configuring volumes, mounts, and persistent volumes claims and getting it right can be a challenge.It’s also challenging to manage that storage and replicate it across all your Kubernetes clusters.It’s also been very challenging to do this on bare metal, outside of a cloud provider.That’s where Longhorn comes.Longhorn is an open source, a CNCF distributed block storage system for Kubernetes.It comes with a UI, backups, snapshots, cluster disaster recovery, and it does all this with or without Rancher.Rancher is NOT a requirement.
Installation
Additional Dependencies
There are some additional dependencies you might want to install on target nodes prior to configuring
1
2
3
4
sudo apt update
sudo apt install nfs-common open-iscsi
#start the service now and on reboot
sudo systemctl enable open-iscsi --now
Install Methods
Rancher app catalog
See the app catalog within Rancher
Kubectl
1
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
1
2
3
kubectl get pods \
--namespace longhorn-system \
--watch
See more at https://longhorn.io/docs/1.0.0/deploy/install/install-with-kubectl
Helm
helm3
1
2
kubectl create namespace longhorn-system
helm install longhorn ./longhorn/chart/ --namespace longhorn-system
1
kubectl -n longhorn-system get pod
Taints
This is not required, nor do I taint nodes anymore.I allow Longhorn storage to use any available space on any node that is not running
etcd
/ control plane.You can simply skip this step and it will work like this.If you’re still convinced you need dedicated nodes, it’s much easier doing it in the Longhorn UI after a node joins the cluster than with taints.
I ended up tainting my storage nodes using this command
1
2
kubectl taint nodes luna-01 luna-02 luna-03 luna-04 CriticalAddonsOnly=true:NoExecute
kubectl taint nodes luna-01 luna-02 luna-03 luna-04 StorageOnly=true:NoExecute
Then applying that toleration to Longhorn in settings
StorageOnly=true:NoExecute;CriticalAddonsOnly=true:NoExecute
This ensures that the storage nodes won’t take on any general workloads and still allow Lonhorn to use these as storage.
Links
⚙️ See all the hardware I recommend at https://l.technotim.live/gear
🚀 Don’t forget to check out the 🚀Launchpad repo with all of the quick start source files