Rate limits

Rate limits protect service availability and latency. Requests that exceed a limit return 429 Too Many Requests with a Retry-After header.

What's limited

Requests can be limited by workspace rate limits or service availability:

Condition	Response
Workspace rate limit for the requested model exceeded	429 rate_limit_exceeded
Service temporarily at capacity	429 service_busy

Your workspace has a separate sustained request-rate limit for each model. Traffic to one model does not consume the request-rate allowance for another. If you need a committed rate or a higher limit for a model, contact us.

All inference usage draws from prepaid workspace credit.

Other constraints to design for:

• Rate limits are shared by workspace and model. All API keys in a workspace share the limit for the model they call. The workspace credit balance remains shared across all models.
• Short bursts may still be limited. Design clients to handle 429 responses even when average traffic is below the stated sustained rate.
• Token limits are model-specific. Requests over the context window return 400 Bad Request.

When you hit a limit

Over-limit requests fail fast with 429, code rate_limit_exceeded, and a Retry-After header (in seconds):

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 1

{"error": {"message": "You have exceeded your workspace's request rate. Slow down and retry, or contact support for a higher sustained limit.", "type": "rate_limit_exceeded", "code": "rate_limit_exceeded"}}

A 429 service_busy response means the service is temporarily at capacity. Handle it in the same way: honor Retry-After and retry with backoff. See Errors for details.

An out-of-credit workspace returns 402 insufficient_quota. Add credit before retrying. See Errors.

Backing off well

Wait at least as long as Retry-After, use exponential backoff with jitter for subsequent attempts, and cap the number of retries:

import random
import time

from openai import APIStatusError


def with_backoff(call, max_attempts=5):
    for attempt in range(max_attempts):
        try:
            return call()
        except APIStatusError as err:
            if err.status_code not in (429, 502, 503) or attempt == max_attempts - 1:
                raise
            retry_after = float(err.response.headers.get("retry-after", 1))
            # Full jitter: sleep somewhere in [retry_after, retry_after + 2^attempt)
            time.sleep(retry_after + random.uniform(0, 2 ** attempt))

For applications with multiple workers, enforce a shared client-side concurrency limit for each model. This is more predictable than independent retry loops in each worker.

Watching your usage

The Console's usage page shows completed requests and token volumes. It does not include throttled requests, so monitor 429 responses in your application logs and metrics.