Building my home-lab with docker-swarm/kubernetes — Part 1

How it started?

Sometime last year I started collecting old hardware to build a lab setup where I can experiment with multiple tools without paying a premium to the cloud providers. I realised that the processors on our old laptops/PCs are quite capable these days and rather than throwing them out in trash we can use them for many meaningful things. So I set out to collecting hardware which looked like trash to others and started setting them up in my bedroom (literally) to run containers.

(UPDATE: I ended up choosing Kubernetes in the end) Before we go any further I would like to clarify why I have picked up docker-swarm for this? My initial hunch was to go with “Kubernetes”. I have a EKS cluster on AWS which I use for running production workloads and I love it. Still, I believe that Kubernetes is a hammer and if you are too much involved with a hammer everything you come across will look like a nail.

  1. Docker-swarm takes only a few seconds to get up and running as compared to Kubernetes which is a pain to setup.
  2. All I want is to run containers easily across multiple nodes with minimal effort — and that’s what docker swarm gives me.
  3. I want to build things which can be easily replicated by everyone. Kubernetes has a huge entry barrier and many people don’t even need it.

What I am trying to do?

  1. Run websites which might not require 100% uptime
  2. Run github self hosted runners to use with github actions
  3. Ensure that I have a good logging and monitoring setup done
  4. Run asynchronous applications on these cluster to reduce load on my AWS EC2 boxes.

Where I am right now?

As of now I have following hardware with me for my lab

  1. Dell Vostro PC -> i5 4th Gen, 120GB SSD, 8GB RAM
  2. PC 2 -> i5 4th Gen, 240GB SSD, 20GB RAM
  3. Laptop 1 -> i5 , 120GB SSD, 8GB RAM
  4. Laptop 2 -> i3, 120GB SSD,4GB RAM

Out of these 4, I am currently using only two because I’m too lazy to setup the remaining 2 right now — but hopefully they’ll be up and running very soon in future.

Overall Plan

  1. Setup docker on all nodes — obviously
  2. Create an EC2 instance on AWS — mainly to get a public IP address and an always up instance on cloud to bring in public traffic to my lab (I don’t have a public IP from my ISP)
  3. Setup openvpn on EC2 instance and connect all my nodes with it — to ensure all nodes are in a single network.
  4. Init docker swarm cluster
  5. Setup prometheus + grafana for monitoring
  1. Setup a public and private Ingress using Traefik. Private ingress for applications which I want to be reachable only from within my home or VPN. Public ingress for applications in which I want to bring in external traffic from internet.
  2. Automate letsencrypt based cert generation for both internal as well as external applications.
  3. Setup a private docker registry to store docker images
  1. Setup self-hosted github action runners
  2. Setup logging pipeline using EFK (Elasticsearch + Fluentd + Kibana) stack.

By this time I should have a basic foundation ready. I have tons of ideas in my mind about it but nothing concrete that I can write about as of now.

Looking forward to publishing a blog in this series every week!

UPDATE:

  1. Why I picked up Kubernetes over docker-swarm for my home lab?
  2. Connected my home servers to a cloud VM using openVPN
  3. My experience with microk8s while setting up the cluster
  4. Kubernetes cluster setup completed using k3s

--

--

--

SRE at Linkedin

Love podcasts or audiobooks? Learn on the go with our new app.

How to solve the issue of start center cannot be edited for being modified by someone

Java-Arrays

How to Pass the AWS Certified Solutions Architect — Associate Exam

How to Create A POST Request in FastAPI

A workstation header image

Complexity

Performance Testing with Gatling

Multi-threading (iii) dealing with concurrency issue

42:Camera Shake!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vikas Yadav

Vikas Yadav

SRE at Linkedin

More from Medium

Kubernetes & Docker Swarm — A Comparison

Host Docker Engine on Virtual Machine

Kubernetes Installation on Openstack VM’s with Kubeadm

Kubernetes Pods

Kubernetes CRD Versioning for Operators Developers