Skip to main content

Upgrade the Crusoe CSI driver

Crusoe CSI driver versions before v0.4.12 are deprecated and will become unsupported after November 8, 2026. After this date, clusters on older CSI drivers may begin to experience shared disk mount failures as Crusoe rolls out file storage platform changes, and Crusoe will no longer provide support for mount issues on unsupported driver versions. If you're unsure whether your cluster is affected, start with Check your current driver version. Otherwise, skip to Upgrade the CSI driver to complete the upgrade procedure.

November 8, 2026 deprecation​

warning

Crusoe CSI driver versions earlier than v0.4.12 are deprecated. Crusoe Managed Kubernetes (CMK) clusters that mount shared disks through the Crusoe CSI driver (fs.csi.crusoe.ai) should upgrade to CSI driver v0.4.12 or later (Helm chart 0.10.21+).

The upgrade itself is low-impact. Existing mounts and I/O continue uninterrupted; only disk control-plane operations (mount, unmount, attach, resize) pause for the 30–60 seconds the CSI driver pods take to restart, and Kubernetes retries those automatically, so a pod starting in that window mounts late rather than failing.

Starting with v0.4.12, the CSI driver resolves the NFS storage hostname itself and passes explicit IP addresses to the kernel NFS client, so mounts no longer depend on the node's DNS configuration. If in-driver resolution fails for any reason, the driver falls back to the previous behavior and lets the kernel resolve the hostname, so the upgrade introduces no new failure mode.

If your cluster runs in a private network without outbound internet access, the upgrade is required: v0.4.12 gets the storage IP addresses directly from the disk API and mounts without needing DNS. The older CSI driver versions (below v0.4.6) don't use the disks API for resolution and rely on static entries instead. The CSI upgrade is needed because customers on lower CSI versions may have mount issues such as mount.nfs: Protocol not supported, mount.nfs: Required key not available, and pods stuck in ContainerCreating.

Who is affected​

Your cluster is affected if it meets all of the following:

  • Mounts a shared disk through the CSI driver (fs.csi.crusoe.ai).
  • Runs Crusoe Managed Kubernetes (CMK) with CSI driver v0.4.11 or older (Helm chart 0.10.20 or older).

Your cluster isn't affected if any of the following apply:

  • Already on CSI driver v0.4.12 or newer (Helm chart 0.10.21 or newer).
  • Uses only block or SSD volumes (csi.crusoe.ai).
  • Mounts shared disks manually over NFS from a VM (not through the CSI driver).

Check your current driver version​

Before upgrading, verify the CSI driver version running on your cluster. Check the container image tags of the actual running pods instead of relying on helm list, which reports the chart's default app version and not the actual running image.

kubectl get ds,deploy -n crusoe-system \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.template.spec.containers[*].image}{"\n"}{end}' \
  | grep crusoe-csi-driver:

If any image tag is v0.4.11 or older, follow the steps below to upgrade the CSI driver to v0.4.12 or newer.

Prerequisites​

Before you start the upgrade, confirm you have:

  • Access to the Crusoe CLI or Crusoe Console.
  • Valid Crusoe authentication credentials.
  • An existing CMK cluster with the Crusoe CSI driver installed.
  • Helm.
note

The Crusoe CSI driver is only supported on CMK. Other configurations are supported on a best-effort basis.

Service impact during upgrade​

Review the expected impact before you start the upgrade:

  • Existing mounts aren't affected during the upgrade.
  • I/O doesn't interrupt during the upgrade.
  • Disk control plane operations (mount, unmount, resize) are temporarily disrupted during the upgrade because the CSI driver pods handle these operations. The CSI pods restart as part of the upgrade process.
info

It's your responsibility to periodically update add-ons in your cluster to ensure continued compatibility, performance, and access to the latest features. During the upgrade process, you might see intermittent issues with volume attachment, detachment, mounting, or creation until the new CSI pods are fully up and running.

note

If your cluster is on CSI driver v0.4.10 or older, some nodes may need a one-time manual step after the upgrade. See Topology label collision on older clusters.

Upgrade the CSI driver​

Complete the four steps below to upgrade the CSI driver with Helm.

1. Ensure the CSI driver Helm repo is installed and up-to-date​

  1. Check if the CSI driver chart already exists:

    helm repo list | grep csi-driver

    • If the Helm repo is already present, update it to the latest version:

      helm repo update crusoe-csi-driver

      Output:

      Hang tight while we grab the latest from your chart repositories...
      ...Successfully got an update from the "crusoe-csi-driver" chart repository
      Update Complete. ⎈Happy Helming!⎈

    • If the Helm repo is absent, add it:

      helm repo add crusoe-csi-driver https://crusoecloud.github.io/crusoe-csi-driver-helm-charts/charts

      Output:

      "crusoe-csi-driver" has been added to your repositories

  2. Find the latest available Helm chart version:

    helm search repo crusoe-csi-driver

    Output:

    NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
    crusoe-csi-driver/crusoe-csi-driver     0.10.23         v0.4.13         A Helm chart for the Crusoe Cloud CSI Driver

2. Create override values for authentication​

Create a YAML file named override_values.yaml with the following contents (do not replace any values):

crusoe:
  secrets:
    crusoeApiKeys:
      # The Crusoe CSI Driver requires access to the Crusoe API in order to manage and mount Crusoe Storage Disks.
      # It is expected that the secret will contain two keys at accessKeyPath and secretKeyPath containing
      # the base64 encoded access key and secret key respectively.
      secretName: "crusoe-secrets"
      accessKeyPath: "CRUSOE_ACCESS_KEY"
      secretKeyPath: "CRUSOE_SECRET_KEY"

3. Upgrade the Helm release​

  1. Note the name of the CSI driver release:

    helm list -n crusoe-system

    Output:

    NAME                     NAMESPACE        REVISION        UPDATED                                 STATUS      CHART                      APP VERSION
    crusoe-csi-driver       crusoe-system     2               2026-08-23 22:23:22.458459 -0700 PDT    deployed	  crusoe-csi-driver-0.10.11  v0.4.4

  2. Upgrade the Helm release to the latest chart version:

    helm upgrade crusoe-csi-driver crusoe-csi-driver/crusoe-csi-driver --version 0.10.23 -f override_values.yaml -n crusoe-system

    Output:

    Release "crusoe-csi-driver" has been upgraded. Happy Helming!
    NAME: crusoe-csi-driver
    LAST DEPLOYED: Thu Sep 18 18:35:32 2025
    NAMESPACE: crusoe-system
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None

4. Verify the upgrade​

  1. Verify the Helm release is installed with the latest chart version:

    helm list -n crusoe-system

    Output:

    NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                     APP VERSION
    crusoe-csi-driver       crusoe-system   3               2026-08-24 17:20:10.155221 -0700 PDT    deployed        crusoe-csi-driver-0.10.23 v0.4.13

  2. Verify the CSI driver pods are up and running:

    kubectl get pods -n crusoe-system | grep csi

    Output:

    crusoe-csi-driver-fs-controller-7975568785-kpv6s   5/5     Running   0          2m41s
    crusoe-csi-driver-fs-node-dwjvr                    3/3     Running   0          2m42s
    crusoe-csi-driver-fs-node-fgf7r                    3/3     Running   0          2m34s
    crusoe-csi-driver-fs-node-nxxct                    3/3     Running   0          2m37s
    crusoe-csi-driver-ssd-controller-5f8c6cb8b-zclsk   5/5     Running   0          2m41s
    crusoe-csi-driver-ssd-node-459c6                   3/3     Running   0          2m37s
    crusoe-csi-driver-ssd-node-4dbx2                   3/3     Running   0          2m41s
    crusoe-csi-driver-ssd-node-5tb2s                   3/3     Running   0          2m34s

When all pods are running, your upgrade is complete.

Topology label collision on older clusters​

If your cluster was on CSI driver v0.4.10 or older before the upgrade, some node-driver-registrar containers may enter CrashLoopBackOff after the new pods roll out. The container logs a line like:

detected topology value collision: driver reported
"fs.csi.crusoe.ai/supports-shared-disks":"true" but existing label is
"fs.csi.crusoe.ai/supports-shared-disks":"false"

Starting in v0.4.11, the driver reports shared-disk support on every instance type. Nodes that ran an older driver on an instance type that did not support shared disks at the time already wrote the node label fs.csi.crusoe.ai/supports-shared-disks=false. Kubernetes treats CSI topology labels as immutable per node, so the upgraded driver cannot change that node's label from false to true, and the registrar on that node crashloops. This affects only nodes that existed before the upgrade. New and recreated nodes register the current value when they join, so a cluster already on v0.4.11 or newer is not affected.

Find the affected nodes:

kubectl get nodes -L fs.csi.crusoe.ai/supports-shared-disks

The nodes showing false are the ones to fix. For each one, either recreate the node (or its node pool), or overwrite the label in place:

kubectl label node <node> fs.csi.crusoe.ai/supports-shared-disks=true --overwrite

The node-driver-registrar on that node recovers on its next restart. To trigger an immediate retry, delete the crusoe-csi-driver-fs-node pod on that node.

Release notes and changelog​

The following table summarizes notable driver releases. For the full history, see the crusoe-csi-driver Helm charts repository on GitHub.

VersionDateChange
v0.4.122026-06NFS hostname resolved in userspace instead of kernel DNS upcall.
v0.4.112026-06Shared FS allowed on all instance types and slice counts; removes virtiofs-era per-SKU gates
v0.4.92026-05Shared FS enabled on c2a and s2a CPU instances
v0.4.62026-04NFS mount target resolved dynamically from the disk API (dns_name / vips), removing hard-coded per-datacenter addressing; Kubernetes 1.34 sidecar compatibility
v0.4.52026-03NodeGetVolumeStats implemented; minimum version for kubelet_volume_stats_* metrics and PV capacity alerting
v0.4.32026-02DNS-based NFS mounting for ICAT datacenters (introduced in v0.4.2, activated by a flag-lookup fix in v0.4.3)
v0.4.02025-10NFS mount support for shared FS disks (previously virtiofs only); mode selected server-side per project
v0.3.32025-09Idempotent mounting; prevents duplicate-mount errors on retries
v0.3.22025-06Shared FS support on gb200-186gb.4x instances
v0.3.02025-05Breaking: auth flags renamed; --crusoe-csi-access-key / --crusoe-csi-secret-key to --crusoe-access-key / --crusoe-secret-key
v0.2.12025-05Driver-side error logging for all gRPC failures (debuggable from pod logs)
v0.1.92025-03Fix: volume detach no longer hangs when the node VM was already deleted; volumes were previously stuck and unreusable