How to Safely Upgrade a k3s Cluster

I’ve been running a small home k3s cluster for a while now — one Raspberry Pi 5 as the control plane and two Intel NUC8i7s as workers. It’s a fun setup, but upgrading Kubernetes always feels like it deserves a bit of care. This post documents how I think about and execute k3s upgrades for this specific setup.

My Cluster Layout

NAME     ROLE            IP             OS              ARCH
athena   control-plane   10.10.10.244   Ubuntu 24.04    arm64 (Pi 5)
nuc242   worker          10.10.10.242   Ubuntu 24.04    amd64
nuc243   worker          10.10.10.243   Ubuntu 24.04    amd64

A few things worth noting about this setup:

  • Cilium is the CNI — flannel is disabled
  • Flux handles GitOps, so the k3s helm controller and Traefik are both disabled
  • No servicelb — I use a dedicated load balancer instead
  • The control plane has a NoSchedule taint so workloads only run on the NUCs

Before You Upgrade: Know Your Config

The first thing I’d recommend is making sure you know how k3s was installed and what flags it’s running with. For modern k3s installs, configuration lives in:

cat /etc/rancher/k3s/config.yaml

Mine looks like this:

# TLS configuration
tls-san:
  - 10.10.10.244
  - k3s.uclab8.net

# Disabled components
disable:
  - traefik
  - servicelb

# Network configuration
flannel-backend: none
disable-network-policy: true

# Node configuration
node-ip: 10.10.10.244
node-external-ip: 10.10.10.244
node-taint:
  - "node-role.kubernetes.io/control-plane=true:NoSchedule"

This file is preserved across upgrades, which means you don’t need to re-pass all your flags every time. The installer picks it up automatically.

For workers, check the equivalent on each node:

ssh [email protected] "cat /etc/rancher/k3s/config.yaml"

Check the Latest Version

curl -s https://update.k3s.io/v1-release/channels/stable

Or just browse the k3s releases on GitHub.

One important rule: don’t skip minor versions. If you’re on 1.31, go to 1.32 first, then 1.33. Skipping versions is unsupported and can cause issues.

The Upgrade Process

Step 0: Save Your Node Token

Before touching anything, save your node token somewhere safe. You’ll need it if you ever have to re-join a worker.

sudo cat /var/lib/rancher/k3s/server/node-token

Step 1: Upgrade the Control Plane

Always upgrade the control plane first. SSH into athena and run:

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.33.4+k3s1" sh -

The installer is idempotent — it swaps the binary, restarts the service, and picks up your existing config.yaml. Once it’s done, verify everything is healthy:

kubectl get nodes
kubectl get pods -A

Wait until the control plane node shows Ready and all system pods are running before moving on.

Step 2: Upgrade Workers (One at a Time)

For each worker, the process is: drain → upgrade → uncordon.

Drain the node (from your local machine):

kubectl drain nuc242 --ignore-daemonsets --delete-emptydir-data

Upgrade k3s on the worker:

ssh [email protected]

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.33.4+k3s1" \
  K3S_URL="https://10.10.10.244:6443" \
  K3S_TOKEN="<your-node-token>" \
  sh -

Uncordon and verify:

kubectl uncordon nuc242
kubectl get pods -A | grep nuc242

Wait for the Cilium pod on that node to be Running before draining the next worker. Then repeat for nuc243.

What About Cilium and Flux?

You don’t need to touch either of them during a k3s upgrade. Cilium runs as a DaemonSet and will reschedule naturally as nodes come back up. Flux continues reconciling from your Git repo throughout the process — it doesn’t care about the underlying k3s version.

The one thing to watch for: after a node restarts, wait for Cilium to report healthy on that node before moving on. You can check with:

cilium status
# or
kubectl get pods -n kube-system -l k8s-app=cilium -o wide

After the Upgrade

A quick sanity check once all nodes are upgraded:

kubectl get nodes -o wide          # all Ready, correct version
kubectl get pods -A                # nothing crashlooping
cilium status                      # CNI healthy
kubectl get helmrelease -A         # Flux reconciliations OK
flux get all                       # everything reconciled

Final Thoughts

k3s upgrades are honestly pretty painless once you know your config is captured properly in config.yaml. The hardest part is usually just remembering to not skip minor versions and to be patient between steps — let each node fully recover before draining the next one.

For a homelab setup like this, I don’t bother with the system-upgrade-controller — the manual process is simple enough and gives me more control. But if you’re running a larger fleet or want full GitOps-driven upgrades, it’s worth looking into.

my DevOps Odyssey

“Σα βγεις στον πηγαιμό για την Ιθάκη, να εύχεσαι να ‘ναι μακρύς ο δρόμος, γεμάτος περιπέτειες, γεμάτος γνώσεις.” - Kavafis’ Ithaka.



A practical guide to upgrading a k3s cluster with a mixed ARM/x86 setup, Cilium as CNI, and Flux for GitOps — without breaking anything

2026-03-01

Series:lab

Categories:Kubernetes

Tags:#k3s, #flux, #cilium, #gitops, #lab


How to Safely Upgrade a k3s Cluster: