Bare Metal/VM Installation - kubeadm¶

This walkthrough will show how to install Kinetica DB. For this example the Kubernetes cluster will be running on 3 VMs with Ubuntu Linux 22.04 (ARM64).

This solution is equivalent to a production bare metal installation and does not use Docker, Podman or QEMU.

The Kubernetes cluster requires 3 VMs consiting of one Master node k8smaster1 and two Worker nodes k8snode1 & k8snode2.

Kubernetes Node Installation¶

Setup the Kubernetes Nodes¶

Edit /etc/hosts¶

SSH into each of the nodes and run the following: -

sudo vi /etc/hosts

x.x.x.x k8smaster1
x.x.x.x k8snode1
x.x.x.x k8snode2

where x.x.x.x is the IP Address of the corresponding nose.

Disable Linux Swap¶

Next we need to disable Swap on Linux: -

sudo swapoff -a

sudo vi /etc/fstab

comment out the swap entry in /etc/fstab on each node.

Linux System Configuration Changes¶

We are using containerd as the container runtime but in order to do so we need to make some system level changes on Linux.

cat << EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay

sudo modprobe br_netfilter

cat << EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sudo sysctl --system

Container Runtime Installation¶

Install containerd¶

sudo apt update

sudo apt install -y containerd

Create a Default containerd Config¶

sudo mkdir -p /etc/containerd

sudo containerd config default | sudo tee /etc/containerd/config.toml

Enable System CGroup¶

Change the SystemdCgroup value to true in the containerd configuration file and restart the service

sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

sudo systemctl restart containerd
sudo systemctl enable containerd

Install Pre-requisite/Utility packages¶

sudo apt update

sudo apt install -y apt-transport-https ca-certificates curl gpg git

Download the Kubernetes public signing key¶

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

Add the Kubernetes Package Repository¶

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

Install the Kubernetes Installation and Management Tools¶

sudo apt update

sudo apt install -y kubeadm=1.29.0-1.1  kubelet=1.29.0-1.1  kubectl=1.29.0-1.1 

sudo apt-mark hold kubeadm kubelet kubectl

Initialize the Kubernetes Cluster¶

Initialize the Kubernetes Cluster by using kubeadm on the k8smaster1 control plane node.

sudo kubeadm init --pod-network-cidr 10.1.1.0/22 --apiserver-advertise-address 192.168.2.180 --kubernetes-version 1.29.2

You should now deploy a pod network to the cluster. Run kubectl apply -f [podnetwork].yaml with one of the options listed at: Cluster Administration Addons

Make a note of the portion of the shell output which gives the join command which we will need to add our worker nodes to the master.

kubeadm join 192.168.2.180:6443 --token wonuiv.v93rkizr6wvxwe6l \
--discovery-token-ca-cert-hash sha256:046ffa6303e6b281285a636e856b8e9e51d8c755248d9d013e15ae5c5f6bb127

Setup kubeconfig¶

Before we add the worker nodes we can setup the kubeconfig so we will be able to use kubectl going forwards.

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Connect & List the Kubernetes Cluster Nodes¶

We can now run kubectl to connect to the Kubernetes API Server to display the nodes in the newly created Kubernetes CLuster.

kubectl get nodes

Join the Worker Nodes to the Cluster¶

Once installed we run the join on the worker nodes. Note that the command which was output from the kubeadm init needs to run with sudo

sudo kubeadm join 192.168.2.180:6443 --token wonuiv.v93rkizr6wvxwe6l \
    --discovery-token-ca-cert-hash sha256:046ffa6303e6b281285a636e856b8e9e51d8c755248d9d013e15ae5c5f6bb127

Now we can again run

kubectl get nodes

Now we can see all the nodes are present in the Kubernetes Cluster.

Install Kubernetes Networking¶

We now need to install a Kubernetes CNI (Container Network Interface) to enable the pod network.

We will use Cilium as the CNI for our cluster.

curl -LO https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-arm64.tar.gz
sudo tar xzvfC cilium-linux-arm64.tar.gz /usr/local/bin
rm cilium-linux-arm64.tar.gz

Install cilium¶

You can now install Cilium with the following command:

cilium install
cilium status 

If cilium status shows errors you may need to wait until the Cilium pods have started.

You can check progress with

kubectl get po -A

Check cilium Status¶

Once Cilium the Cilium pods are running we can check the status of Cilium again by using

cilium status 

We can now recheck the Kubernetes Cluster Nodes

kubectl get nodes

and they should have Status Ready

Kubernetes Node Preparation¶

Label Kubernetes Nodes¶

Now we go ahead and label the nodes. Kinetica uses node labels in production clusters where there are separate 'node groups' configured so that the Kinetica Infrastructure pods are deployed on a smaller VM type and the DB itself is deployed on larger nodes or gpu enabled nodes.

If we were using a Cloud Provider Kubernetes these are synonymous with EKS Node Groups or AKS VMSS which would be created with the same two labels on two node groups.

kubectl label node k8snode1 app.kinetica.com/pool=infra
kubectl label node k8snode2 app.kinetica.com/pool=compute

additionally in our case as we have created a new cluster the 'role' of the worker nodes is not set so we can also set that. In many cases the role is already set to worker but here we have some latitude.

kubectl label node k8snode1 kubernetes.io/role=kinetica-infra
kubectl label node k8snode2 kubernetes.io/role=kinetica-compute

Install Storage Class¶

Install a local path provisioner storage class. In this case we are using the Rancher Local Path provisioner

kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.26/deploy/local-path-storage.yaml

Set Default Storage Class¶

kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Install Kinetica for Kubernetes using Helm¶

Add the Helm Repository¶

helm repo add kinetica-operators https://kineticadb.github.io/charts
helm repo update

Download a Starter Helm values.yaml¶

Now we need to obtain a starter values.yaml file to pass to our Helm install. We can download one from the github.com/kineticadb/charts repo.

    wget https://raw.githubusercontent.com/kineticadb/charts/master/kinetica-operators/values.onPrem.k8s.yaml

Helm Install Kinetica¶

helm -n kinetica-system upgrade -i \
kinetica-operators kinetica-operators/kinetica-operators \
--create-namespace \
--values values.onPrem.k8s.yaml \
--set db.gpudbCluster.license="LICENSE-KEY" \
--set dbAdminUser.password="PASSWORD" \
--set global.defaultStorageClass="local-path"

Monitor Kinetica Startup¶

After a few moments, follow the progression of the main database pod startup with:

kubectl -n gpudb get po gpudb-0 -w