Skip to main content

Going to production

Use this checklist before sending production traffic. Each section links to detailed guidance elsewhere in the documentation.

Keys and secrets​

  • One key per app and environment (prod-backend, staging, cli-tools), and private keys over workspace-shared ones when usage should remain attributable to a person. There are no per-key rate limits.
  • Load keys from a secret manager or environment variable. Never store them in source control. Add the dg_ prefix to secret-scanning rules.
  • Document a rotation procedure. Key changes can take up to one minute to take effect. Rotate shared credentials when access changes and immediately after suspected exposure.
  • Do not include a key in public client-side code. Browser calls are supported, but public JavaScript cannot keep a credential secret. Proxy requests through your backend.

Retries and timeouts​

  • Retry 429, 502, and 503 with exponential backoff and jitter, honoring Retry-After as the floor — code in Rate limits.
  • Never retry 400, 401, 404, or 413 unchanged — they fail identically every time. See Errors.
  • Treat 402 as terminal and alert on it. An exhausted balance stops every key in the workspace; a retry loop cannot fix it.
  • Set client timeouts based on your workload and latency objective. Treat a timeout as transient and retry with backoff.
  • Retry any 5xx by status, not by body. Some 5xx responses do not include a JSON body, so do not require error.code before retrying.

Concurrency and throughput​

  • Use a shared client-side concurrency limit across workers. See Rate limits.
  • Remember limits are workspace-scoped. All keys share one rate ceiling and one credit balance. Set separate concurrency budgets for batch and user-facing traffic.
  • Reuse connections. Use connection pooling and keep-alive. OpenAI SDK clients enable these by default. See Measuring latency.

Observability​

  • Send X-Client-Trace-Id on every request. Celeris echoes it in API responses so you can correlate failures with application logs and support cases. See the API reference.
  • Log or sample the Server-Timing header. Compare server duration with wall-clock latency to separate server-side changes from client and network overhead. See Measuring latency.
  • Track your client-side 429 rate. Throttled requests do not appear on the usage dashboard.

Billing readiness​

  • Monitor spend. The usage page shows completed request counts and token volumes over 24 hours, 7 days, or 30 days. Project spend from those volumes with the pricing rates.
  • Maintain sufficient credit. When the balance is exhausted, all keys in the workspace return 402. Alert on this response.
  • Set max_tokens on every request. This bounds completion latency and cost. See Prompt engineering.

Deployment geometry​

  • Deploy callers close to the serving region. Celeris is available in us-east-1. Typical short requests complete in roughly 50–150 ms within the region; network distance adds to end-to-end latency.
  • Measure with Server-Timing. Compare server duration with wall-clock latency to estimate client and network overhead. See Measuring latency.

The checklist​

  • Separate API keys per app and environment, loaded from a secret store
  • Rotation procedure written down; keys rotate on departures/exposure
  • No keys in public client-side code
  • Backoff with jitter on 429/502/503, honoring Retry-After
  • 400/401/404/413 fail fast — no blind retries
  • 402 alerts a human; nothing retry-loops on it
  • Client timeouts set and timeout retries bounded
  • Shared concurrency cap sized under your workspace's ceiling
  • X-Client-Trace-Id sent on every request and logged
  • Client-side 429 rate tracked
  • max_tokens set (low) on every call; temperature: 0 on structured calls
  • Callers deployed near the serving region; keep-alive connections
  • Credit balance monitored; top-up owner named