Measuring latency

Every successful API response includes a standard Server-Timing header with server-side latency measurements.

Server-Timing: processing;dur=31.669

Metric	What it measures
processing	Server-side time from receiving the request until response headers are available, including request handling and model startup.

Subtract processing from the time to response headers to estimate client and network overhead. Treat the result as an approximation because the measurements are taken at different points in the request lifecycle.

Reading it

cURL

curl -sS -D - -o /dev/null \
  https://inference.cloud.celeris.ai/celeris-1/v1/chat/completions \
  -H "Authorization: Bearer $CELERIS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"celeris-1","messages":[{"role":"user","content":"ok"}],"max_tokens":4}' \
  | grep -i '^server-timing'

Python

import os

import httpx

with httpx.Client() as http:
    r = http.post(
        "https://inference.cloud.celeris.ai/celeris-1/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['CELERIS_API_KEY']}"},
        json={
            "model": "celeris-1",
            "messages": [{"role": "user", "content": "ok"}],
            "max_tokens": 4,
        },
    )
    r.raise_for_status()
    print(r.elapsed.total_seconds() * 1000, "ms wall clock")
    print("Server-Timing:", r.headers["server-timing"])

TypeScript

// Server-Timing is CORS-exposed (Access-Control-Expose-Headers), so browser
// code can read it straight off the fetch Response:
const url = 'https://inference.cloud.celeris.ai/celeris-1/v1/chat/completions';
const init = {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.CELERIS_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'celeris-1',
    messages: [{role: 'user', content: 'ok'}],
    max_tokens: 4,
  }),
};
const started = performance.now();
const response = await fetch(url, init);
if (!response.ok) throw new Error(`Celeris request failed: ${response.status}`);
const wallClockMs = performance.now() - started;

const serverTiming = response.headers.get('server-timing') ?? '';
const match = serverTiming.match(/processing;dur=([\d.]+)/);
const processingMs = match ? Number(match[1]) : undefined;
const networkMs = processingMs == null ? undefined : Math.max(0, wallClockMs - processingMs);
console.log({serverTiming, wallClockMs, processingMs, networkMs});

Getting the numbers down

• Deploy close to the region. Typical short requests complete in roughly 50–150 ms within the region. Network distance adds to end-to-end latency. See Models § Regions.
• Reuse connections. Use an HTTP client with connection pooling and keep-alive enabled. OpenAI SDK clients do this by default.
• Cap max_tokens. Completion length is the main thing you control that moves inference time.
• Monitor 429 responses in your application. The usage page does not include throttled requests. See Rate limits.