Skip to main content

Serverless Inference

Send requests to Crusoe-hosted models over an OpenAI-compatible API or interact with them directly using the Intelligence Foundry's chat interface. Crusoe handles serving, scaling, and optimization—so you don't have to manage any GPU infrastructure.

1. Log in or create an account​

Log in to the Crusoe Cloud Console or Create an account.

After you log in, switch to the Intelligence Foundry app in the bottom-left of the console.

2. Generate an API key​

To create an API key via the console:

  1. From the console, click Admin in the bottom-left corner.
  2. Select Security > Intelligence API keys from the left navigation.
  3. Click Create.
  4. (Optional) Enter an alias for your key.
  5. (Optional) Enter an expiration date for your key.
  6. Copy the API key. Make sure that you save the key in a secure location before leaving the page.

For more information, see Manage API keys.

3. (Optional) Browse available models​

You can use the OpenAI-API compatible endpoint at api.inference.crusoecloud.com to access the models below for Serverless Inference. You can also interact with all of the models using the Intelligence Foundry's chat interface. All Meta models provided by Crusoe are "Built with Llama".

For each model's pricing information, see pricing.

MODELPROVIDERTYPECONTEXT LENGTHLICENSEACCEPTABLE USE POLICY
deepseek-ai/DeepSeek-V3-0324DeepSeekinstruct160kMIT License
deepseek-ai/DeepSeek-V4-FlashDeepSeekinstruct1MMIT License
deepseek-ai/DeepSeek-V4-ProDeepSeekinstruct1MMIT License
google/gemma-4-31b-itGoogleinstruct262kApache License 2.0
meta-llama/Llama-3.3-70B-InstructMetainstruct128kLlama 3.3 Community License AgreementLlama 3.3 Acceptable Use Policy
moonshotai/Kimi-K2.6Moonshotinstruct256KModified MIT License
nvidia/Nemotron-3-Nano-30B-A3BNVIDIAinstruct262kNVIDIA Nemotron Open Model LicenseNVIDIA Acceptable Use Terms
nvidia/Nemotron-3-Nano-Omni-Reasoning-30B-A3BNVIDIAinstruct262kNVIDIA Open Model Agreement
nvidia/Nemotron-3-Super-120B-A12BNVIDIAinstruct262kNVIDIA Nemotron Open Model LicenseNVIDIA Acceptable Use Terms
nvidia/Nemotron-3-Ultra-550BNVIDIAinstruct262kNVIDIA Nemotron Open Model LicenseNVIDIA Acceptable Use Terms
nvidia/Nemotron-3-VoiceChatNVIDIAspeech-to-speech131kNVIDIA Software and Model Evaluation LicenseNVIDIA Acceptable Use Terms
nvidia/nemotron-3.5-lightning-30b-a3bNVIDIAinstruct1MNVIDIA Nemotron Open Model LicenseNVIDIA Acceptable Use Terms
openai/gpt-oss-120bOpenAIinstruct128kApache License 2.0Acceptable Use Policy
qwen/Qwen3-235B-A22B-Instruct-2507Qweninstruct131kApache License 2.0Apache License 2.0
zai/GLM-5.1Z.aiinstruct202kMIT License
zai/GLM-5.2Z.aiinstruct256kMIT License

4. Send a request​

Use the OpenAI-compatible endpoint at api.inference.crusoecloud.com. The example below queries meta-llama/Llama-3.3-70B-Instruct:

Need higher rate limits or reserved capacity?

When interacting with a Serverless endpoint, you might receive a 429 Too Many Requests response due to rate limits. If you need to exceed the default rate limits, you can:

  • Contact us for a rate limit increase. This is recommended if you expect your initial launch traffic to exceed the default limits.
  • If you need reserved inference capacity, use Self-Serve Deployments.
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("CRUSOE_API_KEY"),
    base_url="https://api.inference.crusoecloud.com/v1",
)

completion = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful, concise assistant."},
        {"role": "user", "content": "Who is Robinson Crusoe?"},
    ],
)

print(completion.choices[0].message.content)

Additional resources: