Getting started with the Celeris API

Celeris serves Celeris-1, a low-latency diffusion LLM, through an OpenAI-compatible API, so the official OpenAI SDK is the Celeris client. This notebook makes your first chat completion and covers the three rules every Celeris client must get right:

1. The total window. Input and requested output together can use at most 8192 tokens.
2. Output alignment. Explicit max_tokens values must be positive multiples of 256. When omitted, the service uses 2048.
3. Input budgeting. Prompt tokens, chat-template overhead, and the requested output must fit inside the total window.

Setup

pip install openai
export CELERIS_BASE_URL="https://inference.celeris.ai/celeris-1/v1"
export CELERIS_API_KEY="<your-api-key>"

import os

from openai import OpenAI

client = OpenAI(
    base_url=os.environ.get("CELERIS_BASE_URL") or os.environ.get("OPENAI_BASE_URL"),
    api_key=os.environ.get("CELERIS_API_KEY") or os.environ.get("OPENAI_API_KEY"),
)
MODEL = os.environ.get("CELERIS_MODEL", "celeris-1")

The token window

Celeris requests live in an 8192-token total window. max_tokens reserves the output canvas and must be a positive multiple of 256. If you omit it, the service uses 2048. These helpers validate the reservation and calculate the prompt budget left inside the total window.

TOTAL_TOKEN_LIMIT = 8192
DEFAULT_MAX_OUTPUT_TOKENS = 2048
OUTPUT_TOKEN_BLOCK = 256


def output_token_budget(requested: int | None = None) -> int:
    """Apply the service default and validate an explicit reservation."""
    value = DEFAULT_MAX_OUTPUT_TOKENS if requested is None else requested
    if value <= 0 or value % OUTPUT_TOKEN_BLOCK:
        raise ValueError("max_tokens must be a positive multiple of 256")
    if value >= TOTAL_TOKEN_LIMIT:
        raise ValueError("max_tokens must leave room for the prompt")
    return value


def input_token_budget(max_tokens: int = DEFAULT_MAX_OUTPUT_TOKENS) -> int:
    """Maximum prompt tokens left after the output reservation."""
    return TOTAL_TOKEN_LIMIT - output_token_budget(max_tokens)


for max_tokens in (DEFAULT_MAX_OUTPUT_TOKENS, 256, 6912):
    print(f"max_tokens {max_tokens:>4} -> "
          f"input budget {input_token_budget(max_tokens)} tokens")

max_tokens 2048 -> input budget 6144 tokens
max_tokens  256 -> input budget 7936 tokens
max_tokens 6912 -> input budget 1280 tokens

Fitting the prompt

Celeris does not expose its tokenizer, so estimate ~4 bytes per token plus a small allowance for the chat template, and clip long input before sending, the API will not do it for you.

ESTIMATED_BYTES_PER_TOKEN = 4
CHAT_BASE_TOKENS = 8
CHAT_MESSAGE_TOKENS = 4


def fit_input(system: str, user: str, max_tokens: int) -> str:
    """Clip the user message to the total-token window."""
    overhead = CHAT_BASE_TOKENS + 2 * CHAT_MESSAGE_TOKENS
    system_tokens = -(-len(system.encode("utf-8")) // ESTIMATED_BYTES_PER_TOKEN)
    user_budget_tokens = input_token_budget(max_tokens) - overhead - system_tokens
    max_user_bytes = max(user_budget_tokens, 0) * ESTIMATED_BYTES_PER_TOKEN
    encoded = user.encode("utf-8")
    if len(encoded) <= max_user_bytes:
        return user
    return encoded[:max_user_bytes].decode("utf-8", errors="ignore")

Your first completion

Everything else is standard OpenAI SDK usage.

system = "You are a concise assistant. Answer directly."
user = "Explain what a context window is, in two sentences."

max_tokens = 512  # two-sentence answer; general default is 2048
user = fit_input(system, user, max_tokens)

response = client.chat.completions.create(
    model=MODEL,
    messages=[
        {"role": "system", "content": system},
        {"role": "user", "content": user},
    ],
    max_tokens=max_tokens,
    temperature=0.2,
)

print(response.choices[0].message.content.strip())
if response.usage:
    print(f"\n[usage: {response.usage.prompt_tokens} prompt + "
          f"{response.usage.completion_tokens} completion tokens]")

A context window is the maximum amount of text an AI model can process and remember at one single time during a conversation. If the input exceeds this limit, the model begins to "forget" the earliest parts of the exchange to make room for new information.

[usage: 39 prompt + 52 completion tokens]

Streaming

Celeris's headline feature is speed. Stream to see it: time-to-first-token is the number to watch.

import time

started = time.monotonic()
first_token_at = None

stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "List three uses for a fast, small LLM. One short line each."}],
    max_tokens=256,  # three short lines; general default is 2048
    temperature=0.2,
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content if chunk.choices else None
    if delta:
        if first_token_at is None:
            first_token_at = time.monotonic()
        print(delta, end="", flush=True)

print()
if first_token_at is not None:
    print(f"\n[first token in {(first_token_at - started) * 1000:.0f} ms, "
          f"total {(time.monotonic() - started) * 1000:.0f} ms]")

1. Real-time text classification and autocomplete.
2. Local-device processing for privacy-sensitive data.
3. High-volume summarization with low latency.

[first token in 319 ms, total 321 ms]

Next steps

• Omit max_tokens to use the 2048-token service default. Set a smaller, block-aligned budget for labels, compact JSON, or other short outputs.
• See the chatbot example for a streaming web app that showcases latency, and the alert triage agent for a multi-step tool-use loop under the same token budget.

---

Runnable source for this example: examples/getting-started in the celeris-cookbook repository.