Skip to main content

Image input

celeris-1 understands images as well as text. Send one or more images in the same request as your prompt and the model reads them together, so you can ask questions about a screenshot, extract fields from a scanned document, or compare two photos with the same low-latency call you already use for text.

Images work on both chat completions and the Responses API. The two spell their content parts differently. Everything else here applies to both: how many you can send, encoding, sizing, and token cost.

Images are input only. celeris-1 reads images and replies with text; it does not generate images.

Sending images​

Instead of passing content as a string, pass it as an array of content parts, one part per piece of the message. A text part carries your prompt and an image_url part carries the image, encoded as a base64 data URL:

# `base64 -w0` is GNU-only; this form works on both Linux and macOS.
IMAGE_B64=$(base64 < receipt.png | tr -d '\n')

curl https://inference.celeris.ai/celeris-1/v1/chat/completions \
  -H "Authorization: Bearer $CELERIS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "celeris-1",
    "messages": [
      {"role": "system", "content": "Extract the requested fields. Reply with JSON only."},
      {"role": "user", "content": [
        {"type": "text", "text": "Fields: merchant, total, purchase_date"},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,'"$IMAGE_B64"'"}}
      ]}
    ],
    "max_tokens": 256,
    "temperature": 0
  }'

The response shape is unchanged: text in choices[0].message.content, token counts in usage. Every parameter that works for a text request (max_tokens, temperature, seed, stop, stream, tools) works the same way here.

Content parts are also valid for messages that carry no image at all: a text part on its own is equivalent to passing content as a string, which is useful when you build messages programmatically and only sometimes attach an image.

Several images in one request​

Attach as many image_url parts as you need. The model reads them in the order you list them, so you can refer to them positionally in your prompt, as in "compare the first image to the second", and ask for per-image output:

content = [{"type": "text", "text": "Name the color of each image, in order."}]
for path in ("first.png", "second.png", "third.png"):
    with open(path, "rb") as handle:
        b64 = base64.b64encode(handle.read()).decode()
    content.append(
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}
    )

response = client.chat.completions.create(
    model="celeris-1",
    messages=[{"role": "user", "content": content}],
    max_tokens=256,
)
print(response.choices[0].message.content)

We impose no count limit. What binds in practice is the shared size budget and the context window.

Sending images on the Responses API​

The Responses API takes images too, and takes several of them the same way, but it uses its own spelling for content parts: input_text and input_image rather than text and image_url. The image is the same inline data: URL, passed as a plain string rather than an object.

import base64
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.celeris.ai/celeris-1/v1",
    api_key=os.environ["CELERIS_API_KEY"],
)

with open("receipt.png", "rb") as handle:
    image_b64 = base64.b64encode(handle.read()).decode()

response = client.responses.create(
    model="celeris-1",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Fields: merchant, total, purchase_date"},
                {
                    "type": "input_image",
                    "image_url": f"data:image/png;base64,{image_b64}",
                    "detail": "auto",
                },
            ],
        }
    ],
    max_output_tokens=256,
)
print(response.output_text)

Add an input_image part per image to send more than one; they are read in the order you list them, as on chat completions.

detail is required

Every input_image part must carry detail: one of auto, low, or high. Leave it out, or use the chat spellings here, and the request matches no accepted input shape: the 400 carries a schema error for every shape it was checked against and never names the missing field. Check that every part carries detail, and that the part names are right, before reading further into the body.

Encoding the image​

Pass the image inline as a data: URL:

data:<media-type>;base64,<base64-encoded-bytes>

Declare the media type that matches the bytes you send. Use image/png for a PNG and image/jpeg for a JPEG:

FormatMedia type
PNGimage/png
JPEGimage/jpeg

Other raster formats may be accepted, but they are not part of the documented contract. Convert to PNG or JPEG before sending rather than relying on them.

Send the bytes, not a link

Only inline data: URLs are supported. Pointing image_url.url at an https:// address you host is not supported: fetching it would put a third-party network round trip on the inference path, and the request may fail or add unbounded latency. Download the image in your own application and send the bytes.

Sizing images​

Two limits apply, and image requests usually meet the byte limit first.

The whole JSON body must fit the request size limit. Base64 encoding inflates each image by about 33% on the wire, so it is the encoded images plus your prompt that have to fit, and the limit applies to the request as a whole rather than to each image. The complete body may be up to 64 MiB (67,108,864 bytes). Send images no larger than the task needs, and downscale before you send rather than after a rejection.

An oversized body is rejected with a 413 payload_too_large; a request that streams its body past the limit may instead have its connection torn down. Treat either as a signal to downscale rather than to retry unchanged.

Images also consume prompt tokens from the context window, alongside your text and max_tokens (max_output_tokens on the Responses API).

Resizing and recompressing before you send is almost always worth it. The model rescales the image to its own working resolution, so past a certain size extra pixels buy little accuracy while still costing bytes, latency, and a 413 risk:

from io import BytesIO

from PIL import Image

REQUEST_BUDGET_BYTES = 100_000         # whatever you choose to spend per request...
IMAGE_COUNT = 3                        # ...split across the images it carries
BUDGET_BYTES = REQUEST_BUDGET_BYTES // IMAGE_COUNT

image = Image.open("photo.jpg")
image.thumbnail((896, 896))            # cap the long edge
image = image.convert("RGB")

for quality in (75, 60, 45):
    buffer = BytesIO()
    image.save(buffer, format="JPEG", quality=quality, optimize=True)
    image_bytes = buffer.getvalue()
    if len(image_bytes) <= BUDGET_BYTES:
        break
else:
    raise ValueError(f"still {len(image_bytes)} bytes — downscale further")

Check the encoded size rather than assuming it: a single fixed quality setting can land anywhere from a few kilobytes to several hundred depending on the image, which is why the loop above steps down until it fits.

A smaller upload is also a faster request. For text-heavy images such as scans and screenshots, keep enough resolution that the smallest text you care about stays legible to your own eye. If you cannot read it, neither can the model. Prefer PNG for screenshots and JPEG for photos.

Token usage and cost​

Images are billed as prompt tokens, at the standard prompt rate on the pricing page. There is no separate per-image charge, and every image in the request is charged, not only the first.

An image's token cost comes from the resolution the model processes it at, not from its file size, so you cannot infer it from the byte count of the upload. Recompressing a JPEG to half the bytes does not halve its token cost. Read the real number from the usage block of a completed response, which has this shape (the values below illustrate the field layout; they are not a published per-image cost, so read your own):

"usage": {"prompt_tokens": 284, "completion_tokens": 19, "total_tokens": 303}

The Responses API reports the same thing under different names, input_tokens and output_tokens, so read those instead when you call /v1/responses.

To budget a workload, measure it once against representative inputs: send a request with the images and one with them removed, and the difference in prompt tokens is what they cost you. Images of a similar shape and size cost the same, so that one measurement generalizes across a workload that sends comparable images.

For a whole extraction request costed end to end, covering instruction, image, and reply, see an image extraction, worked through.

Limitations​

  • Images are input only. celeris-1 replies with text and cannot generate, edit, or return images.
  • Inline data: URLs only. Remote https:// image URLs are not supported.

Both the model and the request path treat images as ordinary prompt content, so rate limits, errors, and streaming behave as they do for text. As with any request, a client disconnect does not guarantee cancellation, and work already underway may still be charged, so let requests you care about finish rather than aborting them. See Pricing.