Skip to main content

Serverless Inference

Use serverless inference to interact with supported models through Crusoe's Intelligence Foundry APIs. Models are served on Crusoe's proprietary inference engine with MemoryAlloy, a cluster-wide memory fabric with cache-aware routing that maximizes cache hits, improving TTFT and throughput.

Use Serverless Inference when:

  • Traffic is variable or unpredictable
  • You're prototyping or in early development
  • Per-token pricing is preferable to reserved capacity
Getting startedRetrieve an API key and run your first inference request against a hosted model using the OpenAI SDK.Available modelsBrowse the base models supported by Serverless Inference on the Intelligence Foundry.Rate limitsUnderstand tokens-per-minute and requests-per-minute limits, 429 and 503 responses, and rate limit headers for Serverless Inference.Inference metricsMonitor Serverless Inference performance with built-in metrics and a Prometheus-compatible query API.