Send images with vision

Text and image input are the two modalities Salyro supports. A vision request is an ordinary chat completion with one difference: the user message carries a list of content parts rather than a string, and one of those parts is an image.

Everything else — the endpoint, the authentication, the response shape, streaming, the error handling — is unchanged.

A message with an image in it

import { readFile } from 'node:fs/promises';

const image = await readFile('./label.jpg');
const dataUri = `data:image/jpeg;base64,${image.toString('base64')}`;

const completion = await client.chat.completions.create({
  model: 'openai/gpt-4o-mini',
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is the batch number on this label?' },
        { type: 'image_url', image_url: { url: dataUri } },
      ],
    },
  ],
});

import base64

with open("label.jpg", "rb") as handle:
    data_uri = "data:image/jpeg;base64," + base64.b64encode(handle.read()).decode()

completion = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is the batch number on this label?"},
                {"type": "image_url", "image_url": {"url": data_uri}},
            ],
        }
    ],
)

curl https://api.salyro.com/v1/chat/completions \
  -H "Authorization: Bearer $SALYRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "What is the batch number on this label?" },
          { "type": "image_url", "image_url": { "url": "https://example.com/label.jpg" } }
        ]
      }
    ]
  }'

The text part is not optional in practice. An image with no question attached gets you a description of the image, which is rarely what you wanted. Say what you are looking for.

A data URI or a URL

Both forms go in the same url field:

• A data URI — data:image/jpeg;base64,... — sends the bytes inline. This is the right default for anything private, anything generated on the fly, and anything that does not already have a stable address.
• A public URL — the provider fetches it. Use this when the image is already hosted somewhere the provider can reach, and remember that "the provider can reach it" means genuinely public: a URL behind your authentication is a URL the provider gets a login page from.

Several images in one message

Add more parts. Order is preserved, so you can refer to them positionally in the text:

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Do these two labels come from the same batch?" },
    { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,..." } },
    { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,..." } }
  ]
}

Each image costs tokens, so this is also the fastest way to make a request expensive. Two images at full resolution is a materially larger request than one.

Which models read images

Vision is a per-model capability, not a per-provider one: a provider you have connected will have models that read images and models that do not.

Send an image to a model that cannot read one and the request fails with an explicit error rather than the image being quietly stripped and the question answered from the text alone. That is the same rule that applies everywhere in the API, and it is the reason you find out at the call rather than in the answer.

GET /v1/models lists what your gateway can name. Establish which of those models you intend to use for vision once, and read it from configuration — see Switch providers without changing your code.

What an image costs

An image is not free and it is not cheap. Providers convert an image into tokens based on its dimensions, so the cost of a vision request is driven by resolution more than by anything else in it.

Two practical consequences:

• Downscale before sending. A photograph straight from a phone camera is far larger than any model needs to read a label. Resizing to the smallest size at which the detail is still legible is the single largest saving available.
• Watch the input side of your usage. Vision traffic shows up as input tokens, and a feature that sends images will have an input-to-output ratio unlike anything text-only. See Usage & costs.

If image traffic is a distinct feature, giving it its own gateway is what makes that cost separable from the rest — Attribute cost per feature or customer covers choosing that boundary.

What is not supported

Two more limits worth knowing before you design around them:

• There is no Files API. You cannot upload an image once and reference it by id across requests. Every request carries its own image, as a data URI or a URL.
• Nothing is remembered between requests. An image sent in one request is not available to the next one unless you send it again, on either endpoint.