Lightweight K8S Lab - Rancher + K3S Integrated Deployment - NETSEC


Learning, Sharing, Creating

Cybersecurity Memo

Thursday, July 23, 2020

Lightweight K8S Lab - Rancher + K3S Integrated Deployment

Kubernetes as container orchestration platform is very hot topic now. In this post, I am trying to follow Rancher and K3S guide to integrate them. 
  • Rancher is an amazing GUI for managing and installing Kubernetes clusters. It addresses the operational and security challenges of managing multiple Kubernetes clusters across any infrastructure, while providing DevOps teams with integrated tools for running containerized workloads.
  • K3s is designed to be a single binary of less than 40MB that completely implements the Kubernetes API. In order to achieve this, they removed a lot of extra drivers that didn't need to be part of the core and are easily replaced with add-ons.

It is much easier than what I thought before for installing both and integrate them. One command for Rancher and one command for K3S, then modify k3s service file to change container engine from containered to docker. After that, another command to import K3S into Rancher. That's it.

Online Private / Public Kubernetes Cluster Providers:

  • 1. Okteto Cloud: is a development-oriented Kubernetes platform that helps developers boost their productivity while building cloud-native applications.
  • 2. Red Hat OpenShift Online: is Red Hat's public cloud application deployment and hosting platform that is based on a containerized infrastructure. 
  • 3. KubeSail


Install Docker environment

Although K3S integrates Containerd by default, for many reasons, for the convenience of subsequent deployment, we will replace Containerd with Docker here.

curl -fsSL | sh

Install Rancher Server

The name of Rancher Server sounds like it has to install a lot of things, but it is not. Rancher Server is actually just a Docker image, and the entire Rancher program is packaged using Docker. So the configuration is relatively simple, only one command is needed:
docker run -d -v /data/docker/rancher-server/var/lib/rancher/:/var/lib/rancher/ --restart=unless-stopped --name rancher-server -p 80:80 -p 443:443 rancher/rancher:stable
Wait for a few minutes,  then visit your Server IP to enter the first configuration interface of Rancher Server.

Add Cluster

Import an existing cluster

Copy the third script for next step to import cluster into Rancher.

Install K3S cluster

Let's start the deployment of the K3S cluster.
The official website provides a very useful one-command installation script, we only need to use the one-command script to complete the installation of the K3S environment:
curl -sfL | sh -
After the installation is complete, we need to adjust the K3S service configuration file to disable traefik . 
Note: Traefik is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease. It simplifies networking complexity while designing, deploying, and running applications. Traefik is deployed by default when starting the server.
Modify the configuration file of the K3S service:
vim /etc/systemd/system/
The contents of the file are as follows:
Description=Lightweight Kubernetes

ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s server
Here we need to modify the value of ExecStart and modify it to:
/usr/local/bin/k3s server --docker --no-deploy traefik
After saving and exiting, execute the command to reload the new service configuration file:
systemctl daemon-reload
Restart the K3S service after completion:
service k3s restart
Wait for tens of seconds, and then confirm whether the K3S cluster is ready through the command:
k3s kubectl get node
You will get a result similar to the following:

root@K3S-1:~# vim /etc/systemd/system/
root@K3S-1:~# systemctl daemon-reload
root@K3S-1:~# service k3s restart
root@K3S-1:~# k3s kubectl get node
k3s-1   Ready    master   58s   v1.18.6+k3s1

Import the K3S cluster to Rancher

On the current Rancher Server, the cluster status is displayed as Pending , like this:

This is because we have not yet imported the cluster. In this step, we will import the cluster and establish a connection between Rancher Server and the K3S cluster.
On the K3S master node (in general, the first node is the master controller, also called the Server node), execute the command to import the cluster:
curl --insecure -sfL | kubectl apply -f -
Note: The import commands of each cluster are different, please do not copy the import commands in the tutorial directly!
After that, the following information will be returned in the Shell, indicating that the cluster import configuration is successful:
root@K3S-1:~# curl --insecure -sfL | kubectl apply -f -
error: no objects passed to apply
root@K3S-1:~# curl --insecure -sfL

kind: ClusterRole
  name: proxy-clusterrole-kubeapiserver
- apiGroups: [""]
  - nodes/metrics
  - nodes/proxy
  - nodes/stats
  - nodes/log
  - nodes/spec
      - name: k8s-ssl
          path: /etc/kubernetes
          type: DirectoryOrCreate
      - name: var-run
          path: /var/run
          type: DirectoryOrCreate
      - name: run
          path: /run
          type: DirectoryOrCreate
      - name: cattle-credentials
          secretName: cattle-credentials-26fff6d
          defaultMode: 320
      - hostPath:
          path: /etc/docker/certs.d
          type: DirectoryOrCreate
        name: docker-certs
    type: RollingUpdate
      maxUnavailable: 25%
root@K3S-1:~# curl --insecure -sfL | kubectl apply -f - created created
namespace/cattle-system created
serviceaccount/cattle created created
secret/cattle-credentials-26fff6d created created
deployment.apps/cattle-cluster-agent created
daemonset.apps/cattle-node-agent created

Go back to the Rancher interface and wait for tens of seconds, we will find that the Pending state has changed to the Waiting state:

This status shows that Rancher has received the registration request from K3S and is completing the registration of the K3S cluster. Wait a couple of seconds to complete the import of the K3S cluster, and status will become Active:

At this point, we have successfully completed the connection between Rancher 2.x and K3S, and can operate the K3S cluster just like the K8S cluster.

YouTube Video:


No comments:

Post a Comment