Building my home-lab with docker-swarm/kubernetes — Part 1

How it started?

Sometime last year I started collecting old hardware to build a lab setup where I can experiment with multiple tools without paying a premium to the cloud providers. I realised that the processors on our old laptops/PCs are quite capable these days and rather than throwing them out in trash we can use them for many meaningful things. So I set out to collecting hardware which looked like trash to others and started setting them up in my bedroom (literally) to run containers.

  1. All I want is to run containers easily across multiple nodes with minimal effort — and that’s what docker swarm gives me.
  2. I want to build things which can be easily replicated by everyone. Kubernetes has a huge entry barrier and many people don’t even need it.

What I am trying to do?

  1. Run websites which might not require 100% uptime
  2. Run github self hosted runners to use with github actions
  3. Ensure that I have a good logging and monitoring setup done
  4. Run asynchronous applications on these cluster to reduce load on my AWS EC2 boxes.

Where I am right now?

As of now I have following hardware with me for my lab

  1. PC 2 -> i5 4th Gen, 240GB SSD, 20GB RAM
  2. Laptop 1 -> i5 , 120GB SSD, 8GB RAM
  3. Laptop 2 -> i3, 120GB SSD,4GB RAM

Overall Plan

Phase 1 Goals

  1. Setup docker on all nodes — obviously
  2. Create an EC2 instance on AWS — mainly to get a public IP address and an always up instance on cloud to bring in public traffic to my lab (I don’t have a public IP from my ISP)
  3. Setup openvpn on EC2 instance and connect all my nodes with it — to ensure all nodes are in a single network.
  4. Init docker swarm cluster
  5. Setup prometheus + grafana for monitoring

Phase 2 Goals

  1. Setup a public and private Ingress using Traefik. Private ingress for applications which I want to be reachable only from within my home or VPN. Public ingress for applications in which I want to bring in external traffic from internet.
  2. Automate letsencrypt based cert generation for both internal as well as external applications.
  3. Setup a private docker registry to store docker images

Phase 3 Goals

  1. Setup self-hosted github action runners
  2. Setup logging pipeline using EFK (Elasticsearch + Fluentd + Kibana) stack.

Phase 4 Goals

By this time I should have a basic foundation ready. I have tons of ideas in my mind about it but nothing concrete that I can write about as of now.

  1. Connected my home servers to a cloud VM using openVPN
  2. My experience with microk8s while setting up the cluster
  3. Kubernetes cluster setup completed using k3s

--

--

SRE at Linkedin

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store