How to bring internet traffic into Kubernetes cluster running at home— Home-lab Part 5

4 min readJun 21, 2021

It’s been a while since I’ve been running a kubernetes cluster at my home built using scrap hardware. The cluster is doing great and I’m amazed by how we can build such an amazing thing with hardware that I used to consider of no use.

I wanted to host web applications in the cluster and ensure that:

I can access my web applications from anywhere if needed
I can use the lab to host some web applications which do not have strict uptime requirements.

Bringing in traffic is not so trivial but together we can make this work!

Basic Strategy

Get a t3.micro EC2 instance on AWS and host an OpenVPN server on it
Connect all home-lab nodes to the VPN
Do some iptables magic on the ec2 instance to forward all traffic coming on port 80 and 443 to home-lab
Deploy nginx-ingress in home-lab
Configure the ingress so that you can expose applications over the Internet as well as ensure only private network access when needed.

1 and 2 are already taken care of in this blog: How to connect your home systems to a VPN on AWS in under 30 mins?. For the rest follow along.

Existing System

The existing system looks like this. The EC2 instance has a public IP and it would be the entry point of public traffic. I have connected all my nodes to the VPN although connecting just one should also have been fine. More details about my existing setup available here: https://vik-y.medium.com/how-to-connect-your-home-systems-to-a-vpn-on-aws-in-under-30-mins-home-lab-part-2-6ef64802ccba

Step 1: Port Forward using Iptables

You need to forward any traffic coming in on port 80 or 443 on the EC2 instance to my node with VPN IP: 10.8.0.3 . The EC2 instance has a public IP so it can receive traffic from the internet. Any traffic received on port 80 or 443 it will simply forward to port 80/443 on 10.8.0.3 which will have nginx-ingress running. Once your nginx-ingress receives the traffic everything will start workings seamlessly.

How to do the port forwarding? First of all identify the interface of your ec2 instance which is bound to the internet. For most cases, it would be eth0

Now run these commands on the EC2 instance:

# Forward all traffic received on port 80 
sudo iptables -t nat -A  PREROUTING -i eth0 -p tcp -m multiport --dports 80 -m comment --comment "90 Redirect incoming http traffic to home-pc" -j DNAT --to-destination 10.8.0.3:80# Forward all traffic received on port 443
sudo iptables -t nat -A  PREROUTING -i eth0 -p tcp -m multiport --dports 443 -m comment --comment "90 Redirect incoming http traffic to home-pc" -j DNAT --to-destination 10.8.0.3:443

And that’s it you are done. If you have nginx or any other ingress pre-installed, you can check if everything is working fine by making a curl call to your public IP (Make sure your EC2 instance security groups permit port 80 and 443):

> curl http://PUBLIC_IP/
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

CAUTION: This brings all traffic to your ingress from the internet which you might not want. To avoid this problem you might want to use annotation nginx.ingress.kubernetes.io/whitelist-source-range in your ingress.

For anything which needs to be private, I use something like this in my ingress which returns 403 for all traffic coming in via the internet.

annotations:
  kubernetes.io/ingress.class: nginx
  nginx.ingress.kubernetes.io/whitelist-source-range: "192.168.1.1/24"

Step 2: Install nginx-ingress (Optional)

If you don’t have an ingress installed already you do the following to install it:

Create nginx.yaml file.

controller:
  # We are just overriding the externalTrafficPolicy property so that we get the actual host IP which
  # we can then use this for selectively blocking requests
  service:
    externalTrafficPolicy: "Local"

Now install nginx ingress.

helm upgrade \
  -n ingress \
  nginx-controller \
  ingress-nginx/ingress-nginx \
  -f nginx.yaml \
  --install \
  --create-namespace

You might not need the above nginx.yaml config, I used it specifically for externalTrafficPolicy because I want to selectively disable traffic coming in from private network and public network in the future. If you already have an ingress setup. If you want to use the nginx.ingress.kubernetes.io/whitelist-source-range annotation then you must use the above nginx.yaml with helm.

Conclusion

With the above instructions, you should be able to easily bring HTTP and HTTPS traffic easily to your cluster running inside a private network. You might want to prevent public access for some apps and for that make sure you use the annotations correctly as suggested in the article.

In future articles, I’ll blog about deploying cert-manager in your cluster and easily serving HTTPS websites right from your home.

If you like this article you might be interested in following my home-lab series: https://vik-y.medium.com/building-my-home-lab-with-docker-swarm-part-1-13cceb3c4f1f