# Serverless rate limits

Serverless Inference runs models on shared, multi-tenant deployments. To keep capacity fair and latency consistent, Crusoe enforces rate limits that cap how many tokens and requests a project can send to a Serverless Inference endpoint each minute. Limits apply per project and per model, and only to traffic handled by Serverless Inference endpoints.

## How rate limits work

Two limits are tracked each minute:

- **Tokens Per Minute (TPM)**: The total input and output tokens your project can process for a given model in a one-minute window.
- **Requests Per Minute (RPM)**: The total requests your project can send to a given model in a one-minute window.

Each limit is enforced independently. When your project exceeds either one, the endpoint returns a `429 Too Many Requests` response.

## Default limits

Your default limits depend on whether your organization has a payment method on file:

| Account status       | TPM per model | RPM per model |
| -------------------- | ------------- | ------------- |
| No payment method    | 500,000       | 30            |
| Payment method added | 2,000,000     | 600           |

To automatically raise both limits to 2,000,000 TPM and 600 RPM, [add a payment method](https://console.crusoecloud.com/billing/payments). If your product needs higher limits than that before launching, [contact us](https://www.crusoe.ai/contact-sales) to discuss your expected volume.

:::info 503 Service Unavailable
A 2,000,000 TPM limit doesn't guarantee a successful response. Serverless Inference deployments are shared, so you can still receive a `503 Service Unavailable` while a model scales to meet aggregate traffic, even when you're within your limits.
:::

## Responses when a limit is reached

Serverless Inference returns different status codes depending on whether your project exceeded its limit or the shared endpoint is experiencing unusually high load:

| HTTP status               | Description                                                                                                                    |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `429 Too Many Requests`   | Your project exceeded its TPM or RPM limit for the model.                                                                      |
| `503 Service Unavailable` | Your request is within your limits, but the shared endpoint is experiencing unusually high load and can't accept it right now. |

Both `429` and `503` responses are transient, so you can [retry the request with exponential backoff](#retry-with-exponential-backoff) rather than failing immediately. If you're consistently hitting your rate limit or seeing repeated `503`s, [contact us](https://www.crusoe.ai/contact-sales) so we can help raise your limits or advise on capacity planning.

<details>
<summary>Need higher rate limits or reserved capacity?</summary>

- [Contact us](https://www.crusoe.ai/contact-sales) for a rate limit increase. This is recommended if you expect your initial launch traffic to exceed the default limits.
- If you need predictable response rates or reserved inference capacity, use [Self-Serve Deployments](/self-serve-deployments/overview). They run on dedicated hardware with reserved capacity, which gives you predictable performance and often better economics at scale.
</details>

## Retry with exponential backoff

Your rate limit resets every minute, and shared-endpoint load fluctuates continuously, so retrying immediately only adds to the contention. Back off exponentially between attempts instead:

1. Wait a short initial delay (for example, 1 second) before your first retry.
2. Double the delay after each subsequent failed attempt (1s, 2s, 4s, 8s, and so on).
3. Add a small amount of random jitter to each delay so that multiple clients don't retry simultaneously.
4. Cap the delay (for example, at 30–60 seconds) and the number of retries, then surface the error if the request still hasn't succeeded.

Because TPM and RPM limits reset every minute, a request that fails with `429` will typically succeed within the next window. `503` responses usually clear even sooner, as soon as the shared endpoint scales to meet demand.

## Rate limit headers

Every Serverless Inference response includes headers that report your current limit and remaining allowance, so you can throttle client-side without waiting for a `429`:

| Header                           | Example   | Description                                                     |
| -------------------------------- | --------- | --------------------------------------------------------------- |
| `x-ratelimit-limit-tokens`       | `2000000` | Maximum tokens permitted before the TPM limit is exhausted.     |
| `x-ratelimit-remaining-tokens`   | `1700000` | Remaining tokens permitted before the TPM limit is exhausted.   |
| `x-ratelimit-limit-requests`     | `600`     | Maximum requests permitted before the RPM limit is exhausted.   |
| `x-ratelimit-remaining-requests` | `59`      | Remaining requests permitted before the RPM limit is exhausted. |

## Considerations

Rate limits are designed to protect the shared endpoint from overload, not to limit consumption. Because of this, there might be instances where actual throughput exceeds your configured TPM limits.

## Next steps

- [Get started with Serverless Inference](/quickstart/getting-started-with-serverless-inference)
- [Browse available models](/serverless-inference/available-models)
- [Monitor inference metrics](/serverless-inference/inference-metrics)
- [Compare with Self-Serve Deployments](/self-serve-deployments/overview)