Deploy a self-serve deployment

Self-serve deployments give you reserved inference capacity on Crusoe's optimized inference engine and managed infrastructure. You choose a base model (or a fine-tuned adapter) and a deployment configuration, and Crusoe handles engine selection, tuning, autoscaling, and rate limiting. You get predictable performance, dedicated throughput, no shared rate limits, and per-GPU-hour billing you control.

Prerequisites

• Install the OpenAI Python client (pip install openai httpx), if you use the Python examples.
• For Low-Rank Adaptation (LoRA) adapters, bring a checkpoint from a successful training job completed with Serverless Fine-Tuning.

1. Log in or create an account

Log in to the Crusoe Cloud Console or Create an account.

After you log in, switch to the Intelligence Foundry using the product selector in the top-right of the console.

2. Generate an API key and authenticate

To create an API key through the console:

1. From the console, select the organization name in the top-left corner of the console and click Manage Organization.
2. Select Security > Inference API keys from the left navigation.
3. Click Create.
4. (Optional) Enter an alias for your key.
5. (Optional) Enter an expiration date for your key.
6. Copy the API key. Make sure that you save the key in a secure location before leaving the page.

Authenticate against the API

All API calls require an Intelligence API key and the API URL.

1. Export the token and base URL in your shell:

   export API_TOKEN='<your token>'
   export URL='https://api.inference.crusoecloud.com/v1/chat/completions'

2. Construct an OpenAI client pointed at the Crusoe gateway. Every Python example on this page assumes you have this openai_client in scope:

   from openai import OpenAI
   import httpx, os
   
   openai_client = OpenAI(
       api_key=os.environ["API_TOKEN"],
       url=f"os.environ['URL']",
       http_client=httpx.Client(proxy=None, trust_env=False),
   )

The full OpenAPI specification is published at api.intelligence.crusoecloud.com/docs.

3. Choose a base model and deployment configuration

Define your deployment by selecting the base model you want to serve and the deployment configuration you want to optimize for.

Deployment configurations

Each deployment offers one or more optimization profiles. Pick the configuration that matches your workload requirements and Crusoe will apply the corresponding engine, hardware, and optimizations for you—no hand-tuning required.

Configuration	Optimization	Best for
Responsiveness	Low latency, optimized for time-to-first-token	Interactive applications, real-time inference, latency-sensitive workloads
Throughput	Cost efficiency at scale, optimized for token volume	Batch processing, high-volume workflows, cost-per-token minimization
Balanced	Hybrid blend of throughput and responsiveness, optimized to support moderate token volume and latency	General purpose production traffic

Supported models

Self-serve deployments support the following base models for both configurations. You can also deploy LoRA adapters trained on these base models through Serverless Fine-Tuning.

Lab	Model	Input modalities	Output modalities
Moonshot AI	Kimi K2.6	Text, Image	Text
DeepSeek	DeepSeek V4 Flash	Text	Text
Z AI	GLM 5.1	Text	Text
OpenAI	GPT-OSS 120B	Text	Text
OpenAI	GPT-OSS 20B	Text	Text
Qwen	Qwen3.6 35B	Image, Text	Text
Qwen	Qwen3.6 27B	Image, Text	Text
Google	Gemma 4 31B IT	Image, Text	Text
Qwen	Qwen3.5 9B	Image, Text	Text
Meta	Llama 3.3 70B Instruct	Text	Text
Meta	Llama 3.1 8B Instruct	Text	Text
Qwen	Qwen3 8B	Text	Text
Qwen	Qwen3.5 2B	Image, Text	Text
Qwen	Qwen3 235B A22B Instruct	Text	Text

4. Create a deployment

Create a deployment from the console to get started.

1. Sign in to the console and switch to Intelligence Foundry using the workspace menu in the top-right corner.

2. Select Self-Serve Deployments from the Inference section of the left navigation. The page lists every deployment in your project with its status, model, hardware, replicas, and metadata.

3. Click Create deployment and complete the form:

   • Select a base model or fine-tuned adapter
   • Select a deployment configuration (Responsiveness, Throughput, or Balanced)
   • Select the number of replicas you want your deployment to support

   The hourly cost associated with your chosen deployment configuration is displayed before you confirm.

4. Click any row to view activity logs, endpoint metadata, and its current replica count.

5. Check deployment status

A new deployment provisions reserved capacity, which can take up to 40 minutes to complete. Check the status of your deployment from the deployment table on the Self-Serve Deployments page or from the deployment details page for your chosen deployment. The status updates to Ready when the deployment is available to serve traffic.

A deployment moves through the following states during its lifecycle:

State	Description
Creating	Capacity is being provisioned and the engine is starting up.
Ready	The deployment is ready to serve traffic.
Scaling up	The deployment is scaling up its active replicas.
Scaling down	The deployment is scaling down its active replicas.
Syncing	The deployment alias is being updated.
Failed	The deployment couldn't be created or updated.
Deleting	The deployment is being torn down.
Deleted	The deployment has been removed.

6. Run inference

When the deployment is Ready, send requests to it through the OpenAI-compatible Chat Completions API using your API key. Pass the deployment alias as the model.

Python

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ['URL'],
    api_key=os.environ['API_TOKEN'],
)

response = client.chat.completions.create(
    model='<deployment alias>',
    messages=[
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Summarize the theory of relativity in one sentence."}
    ],
)

print(response.to_json())

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.URL,
  apiKey: process.env.API_TOKEN,
});

client.chat.completions
  .create({
    model: "<deployment alias>",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      {
        role: "user",
        content: "Summarize the theory of relativity in one sentence.",
      },
    ],
  })
  .then((response) => console.log(response));

curl

curl $URL \
  --request 'POST' \
  --header 'Content-Type: application/json' \
  --header 'Accept: text/event-stream' \
  --header "Authorization: Bearer $API_TOKEN" \
  --data '{
    "model": "<deployment alias>",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Summarize the theory of relativity in one sentence."}
    ]
  }'

Because your deployment runs on reserved capacity, its throughput is bounded by replica count rather than a shared rate limit. To add headroom for traffic spikes, increase the replica count on the deployment in the next step.

7. Manage deployments

You can update a running deployment (for example, to change replica count or the deployment alias) or delete one you no longer need. Billing stops when the deployment is deleted.

To view all deployment management options, select the three-dot icon on any deployment row on the Self-Serve Deployments page. The menu exposes options to edit the deployment alias, update the replica count, or delete the deployment.

Edit a deployment alias

To update the alias for a deployment:

1. Define a unique name for your deployment.
2. Confirm your new deployment name.

The deployment status updates to Syncing while the alias is updated, and returns to Ready when the update is complete.

Update replica counts

To adjust the number of replicas for a deployment:

1. Select a value that reflects your expected traffic load and fits within your allotted quota.
2. Confirm your new replica count.

The deployment status updates to Scaling up or Scaling down based on the direction of the change, and returns to Ready when the update is complete. The deployment can still serve traffic while the replica count is being adjusted.

Delete a deployment

After you confirm deletion, the deployment status updates to Deleting, and the deployment disappears from the list once the deletion is complete. Billing stops when the deployment is deleted.

View deployment details

To view the deployment overview, activity log, metadata, and sample code for inferencing, select the deployment alias from the Self-Serve Deployments page.

8. (Optional) Deploy a fine-tuned model

Self-serve works with Crusoe serverless fine-tuning to help you get your tuned models to production with one click. A LoRA adapter you train there is registered in the same model registry as the base models, so it appears in the models list as soon as training completes.

You have two ways to deploy a fine-tuned checkpoint:

• From the Self-Serve Deployments page: Follow the Create a deployment steps, then select your fine-tuned checkpoint from the list after you select the corresponding base model architecture.
• From a Fine-tuned model's Jobs page: Click the three-dot menu next to the checkpoint you want to deploy and select Deploy.

Because fine-tuning and deployment share the same registry and API conventions, you can iterate quickly: train a new adapter, point a new deployment at it, and shift traffic—without rebuilding your serving stack.

Next steps

• Need optimization beyond the standard configurations? Contact us about Tailored Deployments
• For an overview of Crusoe's Managed AI options, see Managed AI