Encrypt and decrypt your data with managed keys
Customer-managed encryption keys (CMEK) let you encrypt and decrypt your data with a KMS key you own in AWS or GCP. You register the key with Crusoe, and Crusoe uses it at runtime through role assumptions (AWS) or workload identity federation (GCP) that you control.
Select your key provider below to see the setup steps for that platform.
Customer-managed encryption keys (CMEK) are currently only available for Serverless Fine-Tuning. When you register a CMEK for a project in Crusoe, all future jobs will automatically use the CMEK for encryption and decryption.
- AWS KMS
- GCP Cloud KMS
Set up CMEK with AWS KMS
Setting up CMEK with AWS KMS involves granting Crusoe permission to assume an IAM role in your AWS account, then registering the KMS key with Crusoe. The following sections explain how access is established and walk through configuration.
How access works
To give CMEK access to the KMS key you store in AWS, you need to:
- Create a single AWS Identity and Access Management (IAM) role in your AWS account that:
- Trusts Crusoe's
CrusoeCMEKrole to assume it, with your Crusoe project ID as theExternalId, for tenant isolation. - Has permission to call
kms:Encrypt,kms:Decrypt,kms:GenerateDataKey, andkms:ReEncrypt*on the specific KMS key you want CMEK to use. Currently, CMEK only callsEncryptandDecrypt.
- Trusts Crusoe's
- Add the role's Amazon Resource Name (ARN) to Crusoe for storage and use at runtime.
Trust chain values
Crusoe provides two values that remain stable across the lifetime of your account: the Crusoe CMEK role ARN and an ExternalId.
| Value | Source | Where you use it | Description |
|---|---|---|---|
| Crusoe CMEK role ARN | arn:aws:iam::1805901 | Principal.AWS in your role's trust policy | The ARN of the Crusoe CMEK role that you'll use in your AWS IAM role's trust policy. |
ExternalId | Your Crusoe project ID(s) | sts:ExternalId condition in your trust policy | The ID that registers the key. To share one KMS key across many projects, list every project ID in the sts:ExternalId condition (StringEquals accepts an array) and register the key in each project. |
Configure CMEK
Use the AWS console (UI) or the AWS CLI to create a KMS key and an IAM role that trusts the CrusoeCMEK role, and then register the key with Crusoe.
Prerequisites
- An AWS account with permission to create KMS keys and IAM roles.
- A Crusoe project ID (used as the
ExternalId). To find your project ID in the console, go to projects and click the copy icon next to the project name. - Access to the Crusoe console to register the key.
- AWS Console (UI)
- AWS CLI
1. Create (or pick) the KMS key
From the KMS console (in your chosen region), go to Customer managed keys and select Create key. Then, fill in the following fields:
- For Key type, enter
Symmetric. - For usage, select
Encrypt and decrypt. Click Next. - For Alias, enter
crusoe-CMEK. - Assign yourself as key administrator and user.
- Click Finish.
- Open the key and copy its ARN:
arn:aws:kms:<region>:<account-id>:key/<key-uuid>.
2. Create the IAM role (trust and KMS policy)
-
In the IAM console, go to Roles → Create role → Custom trust policy, then paste the following and replace the
ExternalIdwith your Crusoe project ID:{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Principal": { "AWS": "arn:aws:iam::180590199243:role/CrusoeCMEK" },"Action": "sts:AssumeRole","Condition": {"StringEquals": { "sts:ExternalId": "<your-crusoe-project-id>" }}}]} -
Click Next (skip Attaching managed policies), name the role
CrusoeCMEKKmsAccess, and click Create role. -
Open the role → Add permissions → Create inline policy → JSON, and paste the following, replacing
<your-kms-key-arn>with your key ARN:{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": ["kms:Encrypt","kms:Decrypt","kms:GenerateDataKey","kms:ReEncrypt*"],"Resource": "<your-kms-key-arn>"}]} -
Name it
KmsAccessand click Create policy. Copy the role ARN from the role summary.
3. Register the key with Crusoe
To register your key with Crusoe:
1. Create (or pick) the KMS key
aws kms create-key --description "KEK for Crusoe CMEK"
# Note the resulting key ARN: arn:aws:kms:<region>:<your-account-id>:key/<key-uuid>
2. Create the IAM role with the trust and key policies
# Crusoe CMEK role ARN (constant)
CRUSOE_CMEK_ROLE_ARN="arn:aws:iam::180590199243:role/CrusoeCMEK"
# Your Crusoe project ID (the project that registers the key in step 3).
# To share this key across several Crusoe projects, use a JSON array here:
# "sts:ExternalId": ["<project-id-1>", "<project-id-2>"]
EXTERNAL_ID="<your-crusoe-project-id>"
# Your inputs
KMS_KEY_ARN="arn:aws:kms:<region>:<your-account-id>:key/<key-uuid>"
ROLE_NAME="CrusoeCMEKKmsAccess"
cat > trust.json <<EOF
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": { "AWS": "${CRUSOE_CMEK_ROLE_ARN}" },
"Action": "sts:AssumeRole",
"Condition": { "StringEquals": { "sts:ExternalId": "${EXTERNAL_ID}" } }
}]
}
EOF
aws iam create-role \
--role-name ${ROLE_NAME} \
--assume-role-policy-document file://trust.json
cat > kms.json <<EOF
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["kms:Encrypt", "kms:Decrypt", "kms:GenerateDataKey", "kms:ReEncrypt*"],
"Resource": "${KMS_KEY_ARN}"
}]
}
EOF
aws iam put-role-policy \
--role-name ${ROLE_NAME} \
--policy-name KmsAccess \
--policy-document file://kms.json
# Note the resulting role ARN—you register it with Crusoe in step 3.
aws iam get-role --role-name ${ROLE_NAME} --query Role.Arn --output text
3. Register the key with Crusoe
To register your key with Crusoe:
A project may register at most one active external key. A second registration returns 409 Conflict. Delete the existing key first (a soft-deleted key doesn't block a new one).
Crusoe returns an external_key_id (a Crusoe-side identifier) and validates the setup before activating it: CMEK assumes your role, using your Crusoe project ID as the ExternalId, and performs a no-op Encrypt and Decrypt round trip on the key. If validation fails—for example, if the trust policy or ExternalId doesn't match, or the role can't call kms:Encrypt or kms:Decrypt on the key—the registration is rejected with an error describing what failed, so you can fix the trust or key policy and retry. On success, the registration is active and ready for CMEK-protected workloads.
CMEK currently doesn't support key replacement. Registering a new key doesn't re-encrypt data that was encrypted under a previous key. See Revoking access before deleting a registered key.
To use the same key from another Crusoe project, add that project's ID to the sts:ExternalId list in the role's trust policy, then repeat this step from the other project.
Rotation behavior
Both the underlying KMS key and the CrusoeCMEK role are designed to rotate without breaking your CMEK registration. The following sections describe what to expect for each.
Underlying KMS key
Nothing changes on your side. AWS KMS embeds the key version in the ciphertext, so CMEK transparently uses the new version after rotation. No re-registration is required.
CrusoeCMEK role
The CrusoeCMEK role ARN is intentionally stable and won't change. If a migration is ever required, Crusoe will provide explicit migration instructions with a notice.
Revoking access
Delete the role or detach the KMS policy:
aws iam delete-role-policy --role-name CrusoeCMEKKmsAccess --policy-name KmsAccess
aws iam delete-role --role-name CrusoeCMEKKmsAccess
Alternatively, unregister the key through the Crusoe console. This leaves your AWS resources in place and CMEK stops calling them.
Deleting the KMS key causes permanent data loss. CMEK doesn't currently support key replacement: files uploaded to the project are encrypted under the registered key. If that key is deleted in AWS, those files are permanently unrecoverable. Crusoe can't decrypt them, and registering a new key doesn't re-encrypt or recover existing data.
If the key stops working without being deleted (for example, the key is disabled, the role is deleted, or the trust or key policy is broken), data isn't lost. Decryption fails until you restore access, and existing files decrypt again when access is restored.
Reporting
You can audit key usage from your own account. Every CMEK call assumes your role and reaches your KMS key, so it appears in your AWS CloudTrail. The STS role session name is CMEK-<crusoe-project-id>, which lets you attribute each call to the Crusoe project that made it.
Troubleshooting
If registration or CMEK operations fail, match the symptom to one of the scenarios below.
AccessDenied on AssumeRole when CMEK tries to use the key
- Confirm the
ExternalIdin your trust policy is the Crusoe project ID the key is registered in (every project ID, if shared across projects). - Confirm the
Principalin your trust policy isarn:aws:iam::180590199243:role/CrusoeCMEK(exact, no typos).
Role can be assumed but the KMS call fails
- Check that
kms.jsonwas attached and references the right key ARN. Runaws kms describe-key --key-id <arn>from a session assumed into the role to verify access. - Don't add
Conditionblocks (for example,kms:EncryptionContext:*) to the role's KMS policy or the key policy. Registration validation doesn't send encryption context, so such conditions reject it, and condition denials during later operation surface as opaque failures rather than actionable errors.
Set up CMEK with GCP Cloud KMS
Setting up CMEK with GCP Cloud KMS involves configuring workload identity federation so Crusoe can access a service account in your GCP project, then registering the Cloud KMS key with Crusoe.
How access works
To give CMEK access to the Cloud KMS key you store in GCP, you need three components:
- A workload identity pool and OpenID Connect (OIDC) provider in your GCP project that trusts Crusoe's federation identity with your Crusoe project ID embedded for tenant isolation.
- A service account in your GCP project that the federated identity can impersonate.
- A
roles/cloudkms.cryptoKeyEncrypterDecrypterrole binding on the Cloud KMS key for the service account.
Crusoe exchanges a short-lived Google ID token (bearing the audience crusoe-cmek/<crusoe-project-id>) at the OIDC provider, uses the service account, and calls Encrypt and Decrypt on your key. The three customer-controlled gates are the provider's audience configuration, the service account's impersonation grant, and the KMS Encrypter/Decrypter role.
Trust chain values
Crusoe provides a federation service account unique ID and an audience format. These values remain stable across the lifetime of your account.
| Value | Source (prod) | Where you use it | Description |
|---|---|---|---|
| Crusoe federation SA unique ID | 118355219843430367466 | Attribute condition in the OIDC provider (assertion.sub) | The unique identifier of the Crusoe CMEK federation service account. Used to restrict which Crusoe service account can use the provider. |
| Audience | crusoe-cmek/<crusoe-project-id> | Allowed audience list in the OIDC provider and ID token audience | Format that encodes your Crusoe project ID for tenant isolation. To share a key across Crusoe projects, add each project's audience. |
Configure CMEK
Use the GCP console (UI) or gcloud CLI to create a Cloud KMS key, set up workload identity federation, create a service account, and register the key with Crusoe.
Prerequisites
- A GCP project with permission to create Cloud KMS keys, workload identity pools, and service accounts.
- A Crusoe project ID (used in the audience format). To find your project ID in the console, go to projects and click the copy icon next to the project name.
- Access to the Crusoe console to register the key.
- GCP Console (UI)
- gcloud CLI
1. Create (or pick) a Cloud KMS key
From the Cloud KMS console, go to Keys and select Create Key Ring. Then:
- Name the key ring
crusoe-cmek-ring. - Select a location (region or multi-region).
- Click Create and open the key ring.
- Click Create Key. For Key type, select
Symmetric encrypt/decrypt. - Name it
crusoe-cmek-keyand click Create. - Copy the key's versionless resource name from the key details page:
projects/<project-number>/locations/<location>/keyRings/<ring>/cryptoKeys/<key-name>(without/cryptoKeyVersions/N).
2. Create a workload identity pool and OIDC provider
- From Workload Identity Federation in the IAM console, click Create Pool.
- Name it
crusoe-cmek-pooland add a description. Click Continue. - Click Add Provider. For Provider type, select OpenID Connect (OIDC).
- Name it
crusoe-cmekand set Provider URI tohttps://accounts.google.com. - Set Audience to
crusoe-cmek/<your-crusoe-project-id>(replace<your-crusoe-project-id>with your actual Crusoe project ID). - Expand Attribute mapping and map:
- Google subject attribute to
assertion.sub.
- Google subject attribute to
- Expand Attribute condition and add the condition:
assertion.sub == '118355219843430367466'(the Crusoe federation SA unique ID). - Click Save and copy the provider's resource name:
projects/<project-number>/locations/global/workloadIdentityPools/crusoe-cmek-pool/providers/crusoe-cmek.
3. Create a service account and grants
- From the Service Accounts page in the IAM console, click Create Service Account.
- Name it
crusoe-cmekand click Create and Continue. - Grant the role
roles/iam.workloadIdentityUserto the principal:Replaceprincipal://iam.googleapis.com/projects/<project-number>/locations/global/workloadIdentityPools/crusoe-cmek-pool/subject/118355219843430367466<project-number>with your GCP project number (not project ID). - Click Done. Copy the service account email:
crusoe-cmek@<project-id>.iam.gserviceaccount.com. - Grant
roles/cloudkms.cryptoKeyEncrypterDecrypterto the service account:- From the Cloud KMS key details page, click Grant access.
- Add the service account email and select
roles/cloudkms.cryptoKeyEncrypterDecrypter. - Click Save.
4. Register the key with Crusoe
To register your key with Crusoe:
- From the Crusoe console, click Encryption Keys. If you have multiple projects, select a project in the top-left corner first, then click Encryption Keys.
- Click Register Key.
- For Provider, select GCP Cloud KMS.
- Fill in the Key Resource Name (versionless, from step 1), Provider Resource Name (from step 2), and Service Account Email (from step 3).
1. Create (or pick) a Cloud KMS key
# Set variables
PROJECT_ID="<your-gcp-project-id>"
PROJECT_NUMBER=$(gcloud projects describe ${PROJECT_ID} --format='value(projectNumber)')
REGION="us-central1" # Choose your preferred region
KEY_RING_NAME="crusoe-cmek-ring"
KEY_NAME="crusoe-cmek-key"
# Create the key ring
gcloud kms keyrings create ${KEY_RING_NAME} --location=${REGION} --project=${PROJECT_ID}
# Create the symmetric encryption key
gcloud kms keys create ${KEY_NAME} \
--location=${REGION} \
--keyring=${KEY_RING_NAME} \
--purpose=encryption \
--project=${PROJECT_ID}
# Get the versionless resource name
gcloud kms keys describe ${KEY_NAME} \
--location=${REGION} \
--keyring=${KEY_RING_NAME} \
--format='value(name)' \
--project=${PROJECT_ID}
# Note this output: projects/<project-number>/locations/<location>/keyRings/<ring>/cryptoKeys/<key>
2. Create a workload identity pool and OIDC provider
# Create the workload identity pool
gcloud iam workload-identity-pools create crusoe-cmek-pool \
--location=global \
--display-name="Crusoe CMEK" \
--project=${PROJECT_ID}
# Create the OIDC provider
gcloud iam workload-identity-pools providers create-oidc crusoe-cmek \
--location=global \
--workload-identity-pool=crusoe-cmek-pool \
--display-name="Crusoe CMEK OIDC Provider" \
--issuer-uri="https://accounts.google.com" \
--allowed-audiences="crusoe-cmek/<your-crusoe-project-id>" \
--attribute-mapping="google.subject=assertion.sub" \
--attribute-condition="assertion.sub == '118355219843430367466'" \
--project=${PROJECT_ID}
# Get the provider resource name
gcloud iam workload-identity-pools providers describe crusoe-cmek \
--location=global \
--workload-identity-pool=crusoe-cmek-pool \
--format='value(name)' \
--project=${PROJECT_ID}
# Note this output: projects/<project-number>/locations/global/workloadIdentityPools/crusoe-cmek-pool/providers/crusoe-cmek
3. Create a service account and grants
# Create the service account
gcloud iam service-accounts create crusoe-cmek \
--display-name="Crusoe CMEK Service Account" \
--project=${PROJECT_ID}
# Get the service account email
SA_EMAIL="crusoe-cmek@${PROJECT_ID}.iam.gserviceaccount.com"
# Grant the workload identity user role to the federated principal
gcloud iam service-accounts add-iam-policy-binding ${SA_EMAIL} \
--role=roles/iam.workloadIdentityUser \
--member="principal://iam.googleapis.com/projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/crusoe-cmek-pool/subject/118355219843430367466" \
--project=${PROJECT_ID}
# Get the Cloud KMS key resource name from step 1 and save it
KMS_KEY_NAME="<versionless-key-name-from-step-1>"
# Grant Cloud KMS Encrypter/Decrypter role to the service account on the specific key
gcloud kms keys add-iam-policy-binding ${KEY_NAME} \
--location=${REGION} \
--keyring=${KEY_RING_NAME} \
--member=serviceAccount:${SA_EMAIL} \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter \
--project=${PROJECT_ID}
# Display the values to register with Crusoe
echo "Register these values with Crusoe:"
echo "Key Resource Name: ${KMS_KEY_NAME}"
echo "Provider Resource Name: projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/crusoe-cmek-pool/providers/crusoe-cmek"
echo "Service Account Email: ${SA_EMAIL}"
4. Register the key with Crusoe
To register your key with Crusoe:
- From the Crusoe console, click Encryption Keys. If you have multiple projects, select a project in the top-left corner first, then click Encryption Keys.
- Click Register Key.
- For Provider, select GCP Cloud KMS.
- Fill in the Key Resource Name (versionless, from step 1), Provider Resource Name (from step 2), and Service Account Email (from step 3).
IAM bindings can take a minute or two to propagate. If registration fails immediately with an access error, wait and retry.
A project may register at most one active external key. A second registration returns 409 Conflict. Delete the existing key first (a soft-deleted key doesn't block a new one).
Crusoe returns an external_key_id (a Crusoe-side identifier) and validates the setup before activating it: CMEK exchanges a short-lived ID token at the OIDC provider, impersonates the service account, and performs a no-op Encrypt and Decrypt round trip on the key. If validation fails—for example, if the audience doesn't match, the attribute condition rejects the token, the service account can't be impersonated, or the service account lacks KMS permissions—the registration is rejected with an error describing what failed, so you can fix the provider, IAM binding, or key policy and retry. On success, the registration is active and ready for CMEK-protected workloads.
CMEK currently doesn't support key replacement. Registering a new key doesn't re-encrypt data that was encrypted under a previous key. See Revoking access before deleting a registered key.
To use the same key from another Crusoe project, add that project's audience (crusoe-cmek/<other-crusoe-project-id>) to the provider's allowed audience list, then repeat this step from the other project.
Rotation behavior
Both the underlying Cloud KMS key and the Crusoe federation service account are designed to rotate without breaking your CMEK registration. The following sections describe what to expect for each.
Underlying Cloud KMS key
Use the versionless key name when you register (and when Crusoe uses it). New encryptions automatically use the current primary key version after you rotate. No re-registration is required. The versionless name ensures that version changes don't break the registration.
Do not destroy old key versions. Encrypted data keys were wrapped by a specific key version and can only be unwrapped by the same version. Deleting a version permanently breaks decryption of data wrapped by it.
Instead of destroying versions, disable them:
gcloud kms keys versions disable <version-id> \
--key=${KEY_NAME} \
--location=${REGION} \
--keyring=${KEY_RING_NAME} \
--project=${PROJECT_ID}
Disabling a key version cuts off future encryptions with that version and can be reversed by enabling it again.
Federation service account
The Crusoe federation service account unique ID (118355219843430367466) is stable across the account lifetime. Google never reuses unique IDs. If a service account rotation is ever required, Crusoe will provide explicit migration instructions with at least 90 days notice.
Revoking access
The fastest way to revoke CMEK access is to disable all Cloud KMS key versions:
# List key versions
gcloud kms keys versions list --key=${KEY_NAME} \
--location=${REGION} \
--keyring=${KEY_RING_NAME} \
--project=${PROJECT_ID}
# Disable each version
gcloud kms keys versions disable <version-id> \
--key=${KEY_NAME} \
--location=${REGION} \
--keyring=${KEY_RING_NAME} \
--project=${PROJECT_ID}
CMEK operations fail within seconds.
Alternatively, revoke access via IAM. CMEK caches the federated service account's OAuth 2.0 access token for up to 15 minutes. That token carries the service account's identity but not its authorization, so the 15-minute window applies differently to each revocation:
- Remove the service account's
roles/cloudkms.cryptoKeyEncrypterDecryptergrant on the key. Cloud KMS checks authorization on every request, so this takes effect on the very next CMEK request. Note that IAM changes can still take a few minutes to propagate within GCP. - Remove the service account's
roles/iam.workloadIdentityUsergrant, delete the OIDC provider, or disable the workload identity pool. These prevent Crusoe from obtaining a new credential, but an already-issued credential can remain valid for up to 15 minutes, so access isn't revoked immediately.
For the fastest revocation, disable the Cloud KMS key versions in GCP as described above. Unregistering the key in the Crusoe console leaves the GCP resources in place—CMEK stops calling them—so it isn't a substitute for disabling the key in GCP.
Deleting the Cloud KMS key causes permanent data loss. CMEK doesn't currently support key replacement: files uploaded to the project are encrypted under the registered key. If that key is deleted in GCP, those files are permanently unrecoverable. Crusoe can't decrypt them, and registering a new key doesn't re-encrypt or recover existing data. Disable key versions instead; disabling is reversible.
If the key stops working without being deleted (for example, the key is disabled, the provider is deleted, the service account is deleted, or the IAM bindings are broken), data isn't lost. Decryption fails until you restore access, and existing files decrypt again when access is restored.
Reporting
Enable Data Access audit logs for the Cloud KMS API in your GCP project to record all Encrypt and Decrypt calls. To do this:
- From the IAM & Admin console, go to Audit Logs.
- Search for or select Cloud KMS API.
- Enable Data Access logging (Admin Read, Data Read, and Data Write).
- Click Save.
Audit log entries record which service account made the call and include delegation info showing the federated Crusoe identity. Log entries are named after the service account, which lets you attribute calls to the specific Crusoe project.
To distinguish multiple Crusoe projects that share one Cloud KMS key, register each Crusoe project with a separate service account (for example, crusoe-cmek-project-a and crusoe-cmek-project-b). Each service account gets the same Cloud KMS key permissions, but audit logs will show which service account was used, attributing the call to the specific Crusoe project.
Troubleshooting
If registration or CMEK operations fail, match the symptom to one of the scenarios below.
Unauthorized (access to customer Cloud KMS key denied)
This error can occur at different stages of the federation chain. Diagnose by checking each gate:
Audience rejected: The OIDC provider doesn't recognize the audience in the ID token.
- Confirm the Audience in the OIDC provider matches
crusoe-cmek/<your-crusoe-project-id>. - If sharing the key across Crusoe projects, confirm every project's audience is in the provider's Allowed audiences list.
Attribute condition rejected: The ID token doesn't pass the condition.
- Confirm the Attribute condition is exactly
assertion.sub == '118355219843430367466'. - This condition restricts the provider to the Crusoe federation service account.
Impersonation denied: The federated principal can't impersonate the service account.
- Confirm the service account has
roles/iam.workloadIdentityUsergranted to the principalprincipal://iam.googleapis.com/projects/<project-number>/locations/global/workloadIdentityPools/crusoe-cmek-pool/subject/118355219843430367466. - Ensure you used the project number (not project ID) in the principal string.
- Run
gcloud iam service-accounts get-iam-policy <service-account-email>to verify the binding.
Cloud KMS access denied: The service account can't call Encrypt or Decrypt on the key.
- Confirm the service account has
roles/cloudkms.cryptoKeyEncrypterDecrypteron the specific Cloud KMS key. - Run
gcloud kms keys get-iam-policy <key-name> --location=<region> --keyring=<keyring>to verify the binding.
400 Bad Request (resource_name must not pin a key version)
You registered a version-pinned key name (for example, projects/.../cryptoKeys/my-key/cryptoKeyVersions/1).
- Register the versionless key name instead:
projects/.../cryptoKeys/my-key(no/cryptoKeyVersions/Nsuffix).
409 Conflict (project already has a key)
The project already has an active registered key.
- Soft-delete the existing key first (through the Crusoe console or API), then register the new key.
409 Conflict (Cloud KMS key disabled or in invalid state)
The Cloud KMS key or all of its versions are disabled.
- Enable at least one key version:
gcloud kms keys versions enable <version-id> --key=<key-name> --location=<region> --keyring=<keyring>.
General diagnosis: IAM propagation and project number
- IAM bindings can take a minute or two to propagate. If you just created a role binding and registration fails immediately, wait and retry.
- Use the project number, not project ID, in the principal string for the impersonation grant and in the provider resource name. The project ID and project number are different.
Next steps
Once your key is registered and validated, CMEK is active for the project. All future fine-tuning jobs in that project automatically use the CMEK for encryption and decryption —no per-job configuration is needed. To create a fine-tuning job that uses your registered key, see Fine-tune a model.