Structured Output & JSON Mode
"Just ask it to output JSON" fails more often than you'd expect. Here's what actually makes output reliable.
Asking a model to "respond in JSON" in plain English usually works, but not reliably enough for production: it can wrap the JSON in markdown fences, add a sentence of preamble before it, produce a field with the wrong type, or occasionally emit invalid JSON outright. For anything downstream that parses the response programmatically, that failure rate compounds.
Most current APIs support passing an explicit JSON schema (or, for some providers, a Pydantic/dataclass-derived schema) that constrains generation at the token level — the model is only allowed to sample tokens that keep the output on a valid path through the schema. This is a much stronger guarantee than prompting for the format: it's enforced by the decoding process, not just requested.
from pydantic import BaseModel
class Extraction(BaseModel):
name: str
age: int
tags: list[str]
# passed as response_format / a tool schema, depending on the provider's API --
# the model's output is now guaranteed to satisfy this shape, not just asked to
import json
from pydantic import ValidationError
raw = response.output_text
try:
data = Extraction.model_validate(json.loads(raw))
except (json.JSONDecodeError, ValidationError) as e:
# schema-constrained decoding makes this rare, not impossible --
# still handle it, especially across provider/model changes
...