Building my home-lab with docker-swarm/kubernetes — Part 1

How it started?

Sometime last year I started collecting old hardware to build a lab setup where I can experiment with multiple tools without paying a premium to the cloud providers. I realised that the processors on our old laptops/PCs are quite capable these days and rather than throwing them out in trash we can use them for many meaningful things. So I set out to collecting hardware which looked like trash to others and started setting them up in my bedroom (literally) to run containers.

(UPDATE: I ended up choosing Kubernetes in the end) Before we go any further I would like to clarify why I have picked up docker-swarm for this? My initial hunch was to go with “Kubernetes”. I have a EKS cluster on AWS which I use for running production workloads and I love it. Still, I believe that Kubernetes is a hammer and if you are too much involved with a hammer everything you come across will look like a nail.

  1. Docker-swarm takes only a few seconds to get up and running as compared to Kubernetes which is a pain to setup.
  2. All I want is to run containers easily across multiple nodes with minimal effort — and that’s what docker swarm gives me.
  3. I want to build things which can be easily replicated by everyone. Kubernetes has a huge entry barrier and many people don’t even need it.

What I am trying to do?

  1. Run websites which might not require 100% uptime
  2. Run github self hosted runners to use with github actions
  3. Ensure that I have a good logging and monitoring setup done
  4. Run asynchronous applications on these cluster to reduce load on my AWS EC2 boxes.

Where I am right now?

As of now I have following hardware with me for my lab

  1. Dell Vostro PC -> i5 4th Gen, 120GB SSD, 8GB RAM
  2. PC 2 -> i5 4th Gen, 240GB SSD, 20GB RAM
  3. Laptop 1 -> i5 , 120GB SSD, 8GB RAM
  4. Laptop 2 -> i3, 120GB SSD,4GB RAM

Out of these 4, I am currently using only two because I’m too lazy to setup the remaining 2 right now — but hopefully they’ll be up and running very soon in future.

Overall Plan

  1. Setup docker on all nodes — obviously
  2. Create an EC2 instance on AWS — mainly to get a public IP address and an always up instance on cloud to bring in public traffic to my lab (I don’t have a public IP from my ISP)
  3. Setup openvpn on EC2 instance and connect all my nodes with it — to ensure all nodes are in a single network.
  4. Init docker swarm cluster
  5. Setup prometheus + grafana for monitoring
  1. Setup a public and private Ingress using Traefik. Private ingress for applications which I want to be reachable only from within my home or VPN. Public ingress for applications in which I want to bring in external traffic from internet.
  2. Automate letsencrypt based cert generation for both internal as well as external applications.
  3. Setup a private docker registry to store docker images
  1. Setup self-hosted github action runners
  2. Setup logging pipeline using EFK (Elasticsearch + Fluentd + Kibana) stack.

By this time I should have a basic foundation ready. I have tons of ideas in my mind about it but nothing concrete that I can write about as of now.

Looking forward to publishing a blog in this series every week!

UPDATE:

  1. Why I picked up Kubernetes over docker-swarm for my home lab?
  2. Connected my home servers to a cloud VM using openVPN
  3. My experience with microk8s while setting up the cluster
  4. Kubernetes cluster setup completed using k3s

SRE at Linkedin

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store