Skip to main content

Rotating Registry Tokens in Kubernetes

By default, CCR tokens are static and short-lived. We recommend rotating tokens frequently. The Crusoe Token Rotator Helm chart runs a periodic job in your cluster to refresh the tokens and update your Kubernetes Secret. The source code and chart can be found in the crusoe-registry-token-rotator-helm-charts repository.

Overview​

The Token Rotator runs as a Kubernetes CronJob. On a defined schedule, it:

  1. Uses your CMK cluster's Crusoe API credentials to generate a new short-lived registry token.
  2. Updates the crusoe-image-pull-secrets in your cluster with the new token.
  3. Cleans up any old tokens managed by the rotator.

This ensures that your Kubernetes nodes always have valid credentials to pull images without manual intervention.

Prerequisites​

Before installing the rotator, you will need:

  • An existing CCR Repository.
  • A Kubernetes cluster with helm >= 3.x and kubectl >= 1.21.x installed.

Create Namespace and Secret (Non-CMK Clusters Only)​

If you are not using Crusoe Managed Kubernetes (CMK), you will need Crusoe API keys. You can create them by following the instructions in "Manage your API Keys".

You must then create both the crusoe-system namespace and the crusoe-secrets secret:

kubectl create namespace crusoe-system

kubectl create secret generic crusoe-secrets \
  --from-literal=CRUSOE_ACCESS_KEY=<your-access-key> \
  --from-literal=CRUSOE_SECRET_KEY=<your-secret-key> \
  -n crusoe-system

Installation​

Step 1: Download the Rotator​

First, ensure your kubectl context is set to the correct cluster:

kubectl config current-context

Now clone the repository with the Token Rotator Helm chart:

git clone https://github.com/crusoecloud/crusoe-registry-token-rotator-helm-charts.git
cd crusoe-registry-token-rotator-helm-charts

Step 2: Configure the Rotator​

Update at least the following fields in charts/crusoe-registry-token-rotator/values.yaml:

  • targetSecret.registryUrl (required): Set to your CCR repository URL
  • targetSecret.registryUsername (required): Set to your CCR username, which is the email address you use in Crusoe
  • targetSecret.namespaces (optional): Update if you want the Secret created in namespaces other than default

For example:

image:
  repository: ghcr.io/crusoecloud/crusoe-registry-token-rotator
  tag: "latest"
  pullPolicy: IfNotPresent
targetSecret:
  name: crusoe-image-pull-secrets
  namespaces:
    - default
    # - <namespace1>
    # - <namespace2>
  registryUrl: "<crusoe-registry-url>"
  registryUsername: "<crusoe-registry-username>"
crusoeCredentialsSecretName: crusoe-secrets
schedule: "0 */6 * * *"
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 50m
    memory: 64Mi

Step 3: Install the Token Rotator Helm Chart​

helm install crusoe-registry-token-rotator ./charts/crusoe-registry-token-rotator \
  --namespace crusoe-system

Or to upgrade:

helm upgrade --install crusoe-registry-token-rotator ./charts/crusoe-registry-token-rotator \
  --namespace crusoe-system

Step 4: Verify the installation​

After installing the chart, verify that the release and its resources were created successfully:

helm list --namespace crusoe-system

You should see output similar to the following:

NAME                         NAMESPACE      REVISION UPDATED                              STATUS   CHART                               APP VERSION
crusoe-registry-token-rotator crusoe-system 1        2026-01-06 13:55:57.593474 -0700 PDT  deployed crusoe-registry-token-rotator-1.0.0

To verify the CronJob was created, run:

kubectl get cronjobs -n crusoe-system

You should see output similar to:

NAME                            SCHEDULE      TIMEZONE   SUSPEND   ACTIVE   LAST SCHEDULE   AGE
crusoe-registry-token-rotator   0 */6 * * *   <none>     False     0        5h29m           9d

Step 5: (Optional) Trigger a Test Run of the CronJob​

By default, the CronJob runs on a schedule. To manually trigger a run for testing:

kubectl create job --from=cronjob/crusoe-registry-token-rotator crusoe-registry-token-rotator-manual-test -n crusoe-system

You can monitor the job with:

kubectl get jobs -n crusoe-system
kubectl logs job/crusoe-registry-token-rotator-manual-test -n crusoe-system

Troubleshooting​

  • Crusoe API Permissions: Ensure the API key used in crusoe-secrets is from a user with at least reader permissions in the Crusoe project.
  • Note on Rotator Permissions: To manage Secrets in each of the namespaces you specify, the rotator is granted GET, CREATE, UPDATE, and PATCH on Secrets in Kubernetes RBAC.