Get structured JSON output

When a model is a step inside a pipeline rather than a chat partner, its output has to be parsed. The tempting approach is to ask nicely in the prompt and then write a regular expression around the answer, which works until the day the model opens with "Sure! Here's the JSON you asked for:".

response_format is the alternative: the output is constrained to JSON, and optionally to a schema you supply, at generation time.

Ask for JSON that matches a schema

const completion = await client.chat.completions.create({
  model: 'openai/gpt-4o-mini',
  messages: [
    { role: 'system', content: 'Extract the fields from the invoice text.' },
    { role: 'user', content: invoiceText },
  ],
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'invoice',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          total: { type: 'number' },
          currency: { type: 'string' },
          issued_on: { type: 'string' },
        },
        required: ['total', 'currency', 'issued_on'],
        additionalProperties: false,
      },
    },
  },
});

const invoice = JSON.parse(completion.choices[0].message.content ?? '');

completion = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Extract the fields from the invoice text."},
        {"role": "user", "content": invoice_text},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "total": {"type": "number"},
                    "currency": {"type": "string"},
                    "issued_on": {"type": "string"},
                },
                "required": ["total", "currency", "issued_on"],
                "additionalProperties": False,
            },
        },
    },
)

invoice = json.loads(completion.choices[0].message.content)

curl https://api.salyro.com/v1/chat/completions \
  -H "Authorization: Bearer $SALYRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{ "role": "user", "content": "Total is 240 EUR, issued 2026-08-01." }],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "invoice",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "total": { "type": "number" },
            "currency": { "type": "string" },
            "issued_on": { "type": "string" }
          },
          "required": ["total", "currency", "issued_on"],
          "additionalProperties": false
        }
      }
    }
  }'

The content of the assistant message is a JSON document rather than prose:

{ "total": 240, "currency": "EUR", "issued_on": "2026-08-01" }

It still arrives as a string in message.content — the response envelope does not change shape. You parse it; you just no longer have to find it first.

Writing a schema the model can satisfy

strict: true is the setting that makes the difference between a suggestion and a constraint, and it comes with rules worth knowing before you debug them:

• Every property must be listed in required. There are no optional fields. To express "may be absent", give the field a type that includes null and require it anyway.
• additionalProperties: false is required on every object, including nested ones.
• Keep the schema flat and the names descriptive. issued_on extracts more reliably than d2, because the field name is part of what the model reads.

Salyro passes the schema through unchanged. If the provider rejects it as malformed, that rejection is what you receive — it is not rewritten into something that would have been accepted.

When a model does not support it

This is where the "no silent dropping" rule stops being a principle and becomes useful.

Not every model supports structured output. Sending response_format to one that does not gets you an explicit error:

{
  "error": {
    "message": "The selected model does not support response_format.",
    "type": "invalid_request_error",
    "param": "response_format"
  }
}

param is the field that makes this actionable: it names the parameter that was refused, so the handler does not have to read the message. The body also carries a code, the stable identifier for the specific condition, which is what to branch on when you need to tell two 400s apart.

Handle it as a capability check rather than as an outage:

try {
  return await extractStructured(model, text);
} catch (error) {
  if (error instanceof OpenAI.APIError && error.param === 'response_format') {
    // This model cannot be constrained. Use one that can, rather than
    // falling back to parsing prose.
    return await extractStructured(FALLBACK_MODEL, text);
  }
  throw error;
}

Which models can be constrained is a property of the model, so it is worth establishing once for the models you use rather than discovering it per request. Switch providers without changing your code covers moving a request to a different model when it needs a capability the current one lacks.

Structured output and streaming

They combine, with one caveat: partial JSON is not JSON. A stream delivers the document a fragment at a time, and every fragment before the last one is syntactically incomplete.

Buffer the whole thing, then parse it once. If you want to show progress while a structured extraction runs, show progress — not the JSON. Stream responses covers consuming the stream.

What this does not guarantee