Kubernetes Load Balancing Demo

Written by Matt Rosenberger. Originally published on October 4th, 2020.

I created this demo (Github Link) to better understand Kubernetes (k8s for now on) and observe its load balancing in action. Using microk8s as the Kubernetes implementation and Artillery for load testing, load balancing worked as expected!

Note: This is not a Kubernetes tutorial. This write-up assumes you have experience with VMs and an understanding of k8s basics from their documentation.

Virtualization Setup

Running this demo was all done on my laptop that runs Windows 10 Home Edition. 3 VirtualBox virtual machines will be used for the k8s cluster.

Note: Every shell interaction in these VMs for this demo will be done as root.

Steps:

Create a new Ubuntu 20.04. During installation:
- Name the machine box1
- Enable SSH
- Select microk8s and docker to be installed
Update all software packages managed by apt and snap.
Run echo "alias kubectl='microk8s kubectl'" > ~/.bash_aliases
Shutdown VM and configure VM settings' network adapter to use Bridged instead of NAT because VMs need to communicate with each other.
Create 2 full clones of the VM.
For each of the cloned VMs:
- Start the VM
- Run hostnamectl set-hostname box2 (or box3)
- Edit /etc/hosts to change box1 to box2 (or box3)
- Shutdown the VM
Start all 3 VMs.

All 3 VMs are running. I use PuTTY to SSH into all 3 machines (copy/pasting is much easier in PuTTY vs. VirtualBox console) and I am able to go into each VM and ping all of the other VMs thanks to bridged networking.

microk8s Cluster

On each VM, run

 microk8s enable ha-cluster

to make sure high availability is enabled. From here, follow microk8s' guide to clustering to create the cluster. In this demo, box1 is the primary node and box2 and box3 are the worker nodes.

Password Generator Application

For this demo, I wrote a Spring Boot application that generates passwords. A REST controller takes password policy as input and serves up passwords generated based on that policy. Tomcat port uses default port 8080.

I included a way to waste CPU if the hogCpuValue is set in the request. Setting this value to 30 will run random number generator 30,000 times for each password character generated. We need to do this so that load balancing occurs when a node is spending too much time on the API request.

server.tomcat.threads.max is set to 1 so that one instance of the running application will only process one API request at a time. I also include Spring Boot Actuator to properly shutdown the running Spring Boot Application and enabled it in the application.properties file.

Docker

With the application built (gradlew build) and working (gradlew bootRun), it's time to create the docker image. I SCP'd the fat jar (found in build/libs) over to the primary node along with the Dockerfile since I can't build docker images on Windows 10 Home Edition.

Note: From this point on, all Docker and k8s commands are run on the primary node (box1) through SSH

On the primary node, run

 docker build . -t localhost:32000/password-gen:v1.0.0

to build the image and

 docker run -p 8080:8080 localhost:32000/password-gen:v1.0.0

to run the image. If you have IntelliJ, you can run http_requests/request.http to test the running Docker container, and run http_requests/shutdown.http to shutdown the Docker container using the Spring Boot Actuator endpoint (after you modify the host to your primary VM address and port to 8080).

The Docker image needs to be uploaded to a Docker registry so that we can deploy it into k8s. Fortunately, microk8s ships with a Docker registry provided:

microk8s registry is enabled by running
```
 microk8s enable registry
```
Docker image's tag name uses the registry's hostname (localhost) and registry's port (32000)

To push the Docker image to the registry hosted by microk8s, run

 docker push localhost:32000/password-gen:v1.0.0

Kubernetes

Run the following to deploy the image to Kubernetes:

 kubectl create deployment password-gen --image=localhost:32000/password-gen:v1.0.0

. This will automatically create a deployment with the full name deployment.apps/password-gen. On deployment creation, only one pod is running in the k8s cluster. Run the following to run 6 pods in the cluster:

 kubectl scale --replicas=6 deployment.apps/password-gen

Port 8080 needs to be exposed from k8s, so run

 kubectl expose deployment.apps/password-gen --port=8080 --type=NodePort

Then, to extract the exposed port that forwards to inside the containers' port 8080, run

 kubectl get services

and find the port that proceeds 8080: on the deployment.apps/password-gen line.

Change the ports in http_requests/request.http and http_requests/shutdown.http and rerun. You should get passwords generated when you run request.http. A pod should go down when running shutdown.http, but it will soon come right back up thanks to k8s.

Updating a deployment's image can be done with one command. Change the version to 1.0.1 in Application.java, build.gradle, and Dockerfile. Rebuild to get the new passwordgen-1.0.1.jar file, copy the updated Dockerfile and newly built jar file to where k8s is hosted. Rerun the docker build command setting version to 1.0.1 and then push the new docker version to the registry. To update the image, run

 kubectl set image deployment.apps/password-gen password-gen=localhost:32000/password-gen:v1.0.1

Artillery

Artillery will now be used to run the load test. Open load_test.yaml and update the port with the exposed port. When the load test runs, one request will be created every second for 20 seconds.

Before invoking the load test, go to each SSH session and run htop to observe activity on each VM node.

Now run

artillery run load_test.yaml

Load Balancing

This screenshot has the 3 PuTTY sessions connected to each k8s clustered node on the left and Artillery execution on the right.

From the screenshot, load balancing is working. Observations running the load test multiple times:

The java -jar /passwordgen-1.0.0.jar commands really are being run from inside docker containers hosted by k8s. None of these VirtualBox VMs have Java installed.
box1, the primary node, never acted as a worker node.
The URL that Artillery hit was always box1's address, and box1 always forwarded the work to box2 and box3.
If I had more pods running or if I was only running 2 nodes, box1 would probably act as a worker node.
k8s would try to put all of the work on box3. When box3 became overworked, k8s would give work to box2. When box2 became overworked and box3 was completing all of its work, k8s would go back to giving box3 more work.
I have 16 GB of RAM on this laptop. I was using close to 100% memory running this load balancing test.

Summary

This demo was a success! microk8s and docker installation couldn't be simpler. Learning a little bit of Artillery to get this load balancing demo setup maybe took an hour.

There are some approaches I would like to tweak with this demo:

Stop doing everything as root! Research correct way to setup users/groups.
I would like to experiment with using distroless images vs. Alpine Linux images to cut down on size and memory usage
Change build process to use Docker layers vs. fat jar (reference)

If you have any questions about this demo, send me a Tweet @mattrr78!