Why I picked kubernetes over docker-swarm for my home lab?

I have been working lately on building a small lab at my home to run containers. In the beginning I decided to use docker swarm because of its simplicity. Since then there has not been a single day when I haven’t given a second thought to it. After all of it today, I finally gave up and decided to dump swarm for kubernetes.

These are the exact thoughts which were going on in my mind:

  • If all I want is to run containers across multiple nodes then there’s nothing better than swarm — I still agree with this. Although I do not want to just run containers, I want to run a full blown production cluster.
  • docker swarm init and docker swarm join -> Can it get any better than this?
  • Kubernetes is an elephant and also a fad
  • My EKS (Elastic Kubernetes Service) cluster is doing great. I wish my home lab was also like it.
  • …. and many more.

As you might have guessed already, Kubernetes won the battle in my head and here are some reasons why I decided to dump docker swarm :(

Kubernetes is “the thing” right now. There’s too much hype around it but I don’t think that hype is wrong. Docker disrupted the way we packaged and shipped applications. Kubernetes will disrupt the way we run and manage them in production.

A lot of people realise it and and have started understanding the benefits and that has led to

  • Creation of amazing community for k8.
  • Large userbase and fanbase
  • Active maintenance and stability
  • Many open source projects to orchestrate a lot of workflows.

And due to the above reasons there’s a lot of value in sticking with Kubernetes as compared to docker swarm which has comparatively a much smaller user base.

Helm is a package manager for Kubernetes. Because of the popularity of kubernetes — you’ll most likely get a fully functional helm chart for any application. Configure the params as per your need and get the stuff up and running with best practices taken care of.

With docker swarm we do not have a popular package manager like helm

Entire reason to go with docker swarm was the simplicity of having it up and running in a few seconds.

After going through some tooling I found microk8s and k3s — they have considerably lesser memory footprint and provide you the simplicity of running highly available k8 clusters with a few commands. I personally found microk8s to be extremely good for beginners

Kubernetes has moved too fast. It’s no longer true that you can run kubernetes only on devices with large cpu and memory. There are various kubernetes distributions which can be run on systems with 1GB RAM too. I personally explored microk8s and k3s before finally settling for microk8s

You might have guessed already that I am now using microk8s to manage run kubernetes on my home lab. I was amazed by the simplicity of it. Over the simplicity it comes with a lot of things built in — in form of “Add ons”. It took me only a few minutes to have a public ingress and a full blown monitoring stack up. Doing the same with docker swarm with best practices would have taken me 2 days at least.

Conclusion

  1. In the long run — having good technical know how of kubernetes is far more valuable than docker swarm. But always remember — fundamentals are primary and tools are secondary
  2. Kubernetes has a huge network effect which gives you access to a large set of tutorials and projects which makes it very simple to perform complex tasks.
  3. Docker swarm doesn’t have a package manager like helm. Publicly available helm charts can drastically reduce time to deploy complex applications. With docker swarm either you have dig through and find a good compose file or build your own.
  4. Tools like microk8s, k3s, etc have made it too simple to run highly available kubernetes cluster.

I am looking forward to doing some amazing things with my cluster. I will also soon write a blog on how I was able to set up kubernetes cluster in under 30 mins with microk8s.

SRE at Linkedin