> ## Documentation Index
> Fetch the complete documentation index at: https://docs.shiftlabs.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Upgrade

> Upgrade Kubernetes clusters with rolling updates and impact analysis

Kubernetes cluster upgrades update the control plane and worker nodes to new versions. The platform provides pre-upgrade validation, impact analysis, and rolling upgrade capabilities to minimize downtime.

## Key Concepts

<CardGroup cols={2}>
  <Card title="Upgrade Check" icon="magnifying-glass">
    Pre-flight validation that verifies version availability and node readiness.
  </Card>

  <Card title="Impact Analysis" icon="chart-line">
    Assessment of cluster health, affected workloads, and risk level before upgrade.
  </Card>

  <Card title="Rolling Upgrade" icon="rotate">
    Sequential upgrade of nodes with cordon/drain to maintain availability.
  </Card>

  <Card title="Selective Upgrade" icon="list-check">
    Option to upgrade specific nodes instead of the entire cluster.
  </Card>
</CardGroup>

## Required Permissions

| Action              | Permission                                    |
| ------------------- | --------------------------------------------- |
| View upgrade impact | `iam:project:infrastructure:kubernetes:read`  |
| Run upgrade check   | `iam:project:infrastructure:kubernetes:read`  |
| Execute upgrade     | `iam:project:infrastructure:kubernetes:write` |

## Version Upgrade Rules

Kubernetes enforces strict version upgrade paths:

| Rule                       | Description                                                |
| -------------------------- | ---------------------------------------------------------- |
| **No major version jumps** | Cannot upgrade across major versions (e.g., 1.x to 2.x)    |
| **One minor at a time**    | Cannot skip minor versions (e.g., 1.28 → 1.30 not allowed) |
| **No downgrades**          | Cannot downgrade to a lower minor version                  |
| **Patch upgrades**         | Can upgrade to any higher patch version within same minor  |

**Valid upgrade paths:**

* `v1.28.5` → `v1.28.10` (patch upgrade)
* `v1.28.5` → `v1.29.0` (minor upgrade)

**Invalid upgrade paths:**

* `v1.28.5` → `v1.30.0` (skips 1.29)
* `v1.29.0` → `v1.28.5` (downgrade)

## Upgradeable Components

| Component          | Description                              |
| ------------------ | ---------------------------------------- |
| **Kubernetes**     | kubeadm, kubelet, kubectl                |
| **containerd**     | Container runtime                        |
| **CNI Plugin**     | Network plugin (Calico, Flannel, Cilium) |
| **Metrics Server** | Resource metrics provider                |

## How to Set Target Versions

<Steps>
  <Step title="Open Cluster Settings">
    Navigate to the cluster and open the **Settings** or **Upgrade** tab.
  </Step>

  <Step title="Configure Target Versions">
    Set target versions for:

    * Kubernetes version
    * containerd version (optional)
    * CNI plugin version (optional)
    * Metrics server version (optional)
  </Step>

  <Step title="Save Configuration">
    Save the upgrade configuration. This does not start the upgrade.
  </Step>
</Steps>

## How to Run Upgrade Check

Upgrade check validates prerequisites before upgrade.

<Steps>
  <Step title="Open Upgrade Page">
    Navigate to the cluster upgrade page.
  </Step>

  <Step title="Click Upgrade Check">
    Click **Upgrade Check** to run pre-flight validation.
  </Step>

  <Step title="Review Results">
    The check verifies:

    * Target version availability in package repositories
    * SSH connectivity to all nodes
    * Node readiness status
  </Step>
</Steps>

<Info>
  Upgrade check must pass before you can proceed with the upgrade.
</Info>

## How to View Upgrade Impact

Impact analysis assesses risk and identifies potential issues.

<Steps>
  <Step title="Open Upgrade Page">
    Navigate to the cluster upgrade page.
  </Step>

  <Step title="View Impact Analysis">
    The impact analysis shows:

    * Current vs target version comparison
    * Cluster health status
    * Node readiness details
    * Affected workload count
    * Risk level assessment
    * Recommendations
  </Step>

  <Step title="Address Issues">
    Resolve any errors or warnings before proceeding.
  </Step>
</Steps>

## Pre-Upgrade Requirements

The upgrade will be blocked if these conditions are not met:

| Requirement               | Details                                                            |
| ------------------------- | ------------------------------------------------------------------ |
| **Master node count**     | Standalone: exactly 1 master. Multinode: 1 or odd number (3, 5, 7) |
| **First master**          | One master must be marked as "first master"                        |
| **SSH connectivity**      | All nodes must be accessible via SSH                               |
| **Nodes ready**           | All nodes must be in Ready state                                   |
| **etcd healthy**          | etcd pods must be running                                          |
| **Control plane healthy** | API server, controller-manager, scheduler must be running          |

## Risk Levels

| Level        | Criteria                                         | Recommendation                |
| ------------ | ------------------------------------------------ | ----------------------------- |
| **Critical** | Errors found in validation                       | Do not proceed until resolved |
| **High**     | Cluster unhealthy or critical workloads affected | Proceed with caution          |
| **Medium**   | Multiple warnings or >50 affected pods           | Plan maintenance window       |
| **Low**      | No significant issues                            | Safe to proceed               |

## How to Upgrade a Cluster

<Steps>
  <Step title="Run Upgrade Check">
    Ensure upgrade check passes with no errors.
  </Step>

  <Step title="Review Impact Analysis">
    Verify risk level is acceptable and review recommendations.
  </Step>

  <Step title="Create etcd Backup">
    Create an etcd backup before starting (strongly recommended).
  </Step>

  <Step title="Start Upgrade">
    Click **Upgrade** to begin the rolling upgrade.
  </Step>

  <Step title="Monitor Progress">
    Watch the operation progress as nodes are upgraded sequentially.
  </Step>
</Steps>

<Warning>
  Upgrade causes temporary disruption as pods are evicted and rescheduled. Plan upgrades during maintenance windows for production clusters.
</Warning>

## Upgrade Process

The platform follows this sequence:

1. **Master nodes first** - Control plane upgraded before workers
2. **For each node:**
   * Cordon (mark unschedulable)
   * Drain (evict pods)
   * Upgrade kubeadm, kubelet, kubectl
   * Upgrade containerd (if configured)
   * Uncordon (allow scheduling)
   * Wait for node Ready

<Info>
  The first master node runs `kubeadm upgrade apply`. Subsequent nodes run `kubeadm upgrade node`.
</Info>

## How to Upgrade Selective Nodes

Upgrade specific nodes instead of the entire cluster.

<Steps>
  <Step title="Open Upgrade Page">
    Navigate to the cluster upgrade page.
  </Step>

  <Step title="Select Nodes">
    Choose specific nodes to upgrade from the node list.
  </Step>

  <Step title="Configure Options">
    Set upgrade options:

    * **Batch size** - Number of nodes to upgrade in parallel
    * **Node order** - Masters first, workers first, or as listed
  </Step>

  <Step title="Start Selective Upgrade">
    Click **Upgrade Selected** to begin.
  </Step>
</Steps>

<Warning>
  Selective upgrade is for advanced use cases. Ensure version consistency across the cluster after selective upgrades.
</Warning>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Upgrade check fails with version not found">
    * Verify the target version exists in Kubernetes apt repository
    * Check that the version format is correct (e.g., `1.29.0` not `v1.29`)
    * Ensure nodes have internet access to download packages
  </Accordion>

  <Accordion title="Cannot upgrade: invalid version path">
    * You cannot skip minor versions - upgrade one minor version at a time
    * Example: From 1.27.x, upgrade to 1.28.x first, then 1.29.x
  </Accordion>

  <Accordion title="Upgrade blocked due to node not ready">
    * Check kubelet status on the affected node
    * Verify network connectivity to control plane
    * Review node conditions for pressure (memory, disk, PID)
  </Accordion>

  <Accordion title="Upgrade fails during drain">
    * Some pods may have PodDisruptionBudgets preventing eviction
    * Pods with local storage may not drain without force
    * Review drain logs for specific pod failures
  </Accordion>

  <Accordion title="Node not coming back after upgrade">
    * SSH to the node and check kubelet status
    * Verify kubelet and containerd are running
    * Check kubelet logs for certificate or connectivity errors
    * Ensure firewall allows required ports
  </Accordion>

  <Accordion title="Cluster unreachable after control plane upgrade">
    * Wait for API server to restart (may take a few minutes)
    * Check API server pod status in kube-system
    * Verify kubeconfig credentials are still valid
    * Check etcd health
  </Accordion>
</AccordionGroup>

## FAQ

<AccordionGroup>
  <Accordion title="How long does an upgrade take?">
    Depends on cluster size. Each node takes several minutes for drain, upgrade, and restart. A 5-node cluster typically completes in 15-30 minutes.
  </Accordion>

  <Accordion title="Is there downtime during upgrade?">
    Applications with multiple replicas spread across nodes experience minimal disruption. Single-replica workloads will have brief downtime when their node is drained.
  </Accordion>

  <Accordion title="Can I rollback an upgrade?">
    Kubernetes does not support direct rollback. Restore from etcd backup if critical issues occur. Always backup before upgrading.
  </Accordion>

  <Accordion title="Should I upgrade containerd with Kubernetes?">
    Not required for every K8s upgrade. Check containerd compatibility matrix for your target K8s version.
  </Accordion>

  <Accordion title="What about CNI plugin upgrades?">
    CNI plugins are upgraded separately. The platform can upgrade Calico, Flannel, or Cilium alongside Kubernetes.
  </Accordion>

  <Accordion title="How do I handle a failed mid-upgrade?">
    The cluster may be in a mixed-version state. Resume the upgrade for remaining nodes or manually upgrade via SSH.
  </Accordion>

  <Accordion title="Can I upgrade a single-master cluster?">
    Yes, but there will be control plane downtime during the master upgrade. Consider adding masters for high availability before upgrading production clusters.
  </Accordion>
</AccordionGroup>

## Best Practices

* **Test in non-production first** - Upgrade staging clusters before production
* **Create etcd backup** - Always backup before upgrading
* **Check release notes** - Review Kubernetes changelog for breaking changes
* **Plan maintenance window** - Schedule upgrades during low-traffic periods
* **Monitor after upgrade** - Verify application health after completion
* **Keep versions current** - Don't fall too far behind; smaller jumps are safer