Managing Persistent Disks

info

Crusoe Cloud is currently in private beta. If you do not currently have access, please request access to continue.

Creating Persistent Disks

CLI
UI
Terraform

Use the storage disks create command to create a disk with your size. In the example below we will create 100 GiB disk called "data-1."

crusoe storage disks create \
  --name data-1 \
  --size 100GiB \
  --location us-northcentral1

name, size, and location are required arguments. The location can be either "us-northcentral1" or "us-east1". When attaching a disk to a VM, the disk must be in the same location as the VM.

In order to create a Disk via the Crusoe Cloud console:

Visit the Crusoe Cloud console
Click the "Storage" tab in the left nav
Click the "Create Disk" button
Input a name for the disk, using only letters, numbers, - and _
Set the desired size of the disk from 1GiB to 10TiB
Click the "Create" button

Although Crusoe VMs come enabled with a 128 GB OS disk that is persistent, this disk is often not large enough for most AI or ML applications. For this reason, creating and attaching a persistent Disk is fundamental to Crusoe Cloud. The following is intended to help get you started in using Terraform to provision a persistent disk and attach the disk to a VM in Crusoe Cloud.

Copy and paste the code below in a text-editor of your choice and name the file main.tf. The example below creates a Disk:

// Crusoe Provider
terraform {
  required_providers {
    crusoe = {
      source = "registry.terraform.io/crusoecloud/crusoe"
    }
  }
}

resource "crusoe_storage_disk" "new_data_disk" {
  name = "new-data-disk"
  size = "200GiB" // "1GiB" to "10TiB"
  location = "us-northcentral1-a" // or "us-east1a"
}

name, size, and location are required arguments.

name can only include lowercase ascii characters, numbers and -.

size must be in format [Number][unit] where valid units are GiB (gibibyte) and TiB (tebibyte). Acceptable sizes are from 1GiB to 10TiB (required).

location is either in us-northcentral1-a or us-east1-a. When attaching a disk to a VM, the disk must be in the same location as the VM.

Viewing all disks

CLI
UI
Terraform

Use the storage disks list command to list existing disks.

crusoe storage disks list

To list existing disks using Terraform, the following code snippet can be used to populate a Terraform data source using the Crusoe Terraform provider.

# list disks
data "crusoe_storage_disks" "disks" {}

output "crusoe_disks" {
  value = data.crusoe_storage_disks.disks
}

Update an existing disk

CLI
UI
Terraform

Use the storage disks resize <name> command to resize existing disks using the --size flag. Here's an example:

crusoe storage disks resize <name> --size <size>

To update an existing disk using the Crusoe Terraform provider, you can change the fields of an existing disk resource and run terraform apply. The Crusoe Terraform provider will apply the changes to the disk.

terraform {
  required_providers {
    crusoe = {
      source = "registry.terraform.io/crusoecloud/crusoe"
    }
  }
}

resource "crusoe_storage_disk" "new_data_disk" {
  name = "new-data-disk"
  size = "200GiB" -> "400GiB"
  location = "us-northcentral1-a"
}

Currently, only the "size" of the disk can be changed. Changes to the "name" or "location" of the disk will force a re-creation of the disk (deletion and then creation of a new disk).

Deleting a disk

info

Warning: deleting a disk is a permanant action.

CLI
UI
Terraform

Use the storage disks delete <name> command to delete a disk of your choice. As an example, you can delete a disk by replacing DISK_NAME with the name of the disk you wish to delete:

crusoe storage disks delete DISK_NAME

A disk can be deleted by using the terraform destroy command provided by the Terraform CLI tool.

Attaching Persistent Disks

Once the disk is created above we can attach and the disk to an instance. In both cases the instance must be in the stopped state.

CLI
UI
Terraform

Use the compute vms attach-disks command to attach a disk to an instance. You can attach multiple disks to an instance with this command as well using a comma separated list of disk names.

crusoe compute vms attach-disks my-vm --disks name=data-1,mode=read-write

Now if you want to create a VM with a disk attached, you can copy and paste the code below in a text-editor of your choice and name the file main.tf. The example below creates a VM that uses a single Nvidia A40 GPU called my-new-vm with the disk new-data-disk created and attached in the us-northcentral1-a location:

terraform {
  required_providers {
    crusoe = {
      source = "registry.terraform.io/crusoecloud/crusoe"
    }
  }
}

locals {
  my_ssh_key = file("~/.ssh/id_ed25519.pub")
}

resource "crusoe_storage_disk" "new_data_disk" {
  name = "new-data-disk"
  size = "200GiB"
  location = "us-northcentral1-a"
}

// new VM
resource "crusoe_compute_instance" "my_vm" {
  name = "my-new-vm"
  type = "a40.1x"
  location = "us-northcentral1-a"

  # specify the base image
  image = "ubuntu22.04:latest"

  disks = [
      // disk attached at startup
      {
        id = crusoe_storage_disk.new_data_disk.id
        mode = "read-write" // other option: "read-only"
        attachment_type = "data"
      }
    ]

  ssh_key = local.my_ssh_key
}

In the disks section of the VM resource, id, mode and attachment_type are required.

id is the id of the disk, which you can append as crusoe_storage_disk.new_data_disk.id.

mode is either read-only or read-write.

attachment_type is only set to data currently.

After saving the code to a main.tf file, the following commands serve as the process to create a resource in Crusoe Cloud using Terraform:

terraform init - Initializes a working directory containing Terraform configuration files.

terraform plan - the output of this command will show the resources Terraform plans on creating.

terraform apply - this command will create the resources.

You can confirm that terraform successfully created the resources through the console, but if you prefer CLI, you can also run: crusoe storage disks list

Which will show you the Disks you have created in your account.

Formatting and Mounting Persistent Disks

Once the instance is started, login and use the command lsblk to inspect the available disks attached to the instance. These persistent ssd disks will have the vd[b-z] name. In situations where you have multiple disks with the same size, to associate which disk name corresponds to which disk in the consle you can navigate to /dev/disk/by-id you will see the id after virtio-... matches the serial number for that disk in the console. For example:

lrwxrwxrwx 1 root root  9 Jun 14 17:52 virtio-39734E8567D2ECA55C1 -> ../../vdb

39734E8567D2ECA55C1 matches the serial number in the Crusoe Cloud Instance details console.

Create the filesystem on the block device by running:

mkfs.ext4 /dev/vdb

Mount the volume by creating a /scratch directory and mouting /dev/vdb to /scratch

mount -t ext4 /dev/vdb /scratch

Some additional mount options exist within the ext4 filesystem by passing -O ... for example -O noatime,nodiratime,data=writeback will avoid writing access times, as well disabling journaling if your workloads can benefit from these optimizations.

Managing Persistent Disks

Creating Persistent Disks​

Viewing all disks​

Update an existing disk​

Deleting a disk​

Attaching Persistent Disks​

Formatting and Mounting Persistent Disks​