why plain prompting isn't enough

Asking a model to "respond in JSON" in plain English usually works, but not reliably enough for production: it can wrap the JSON in markdown fences, add a sentence of preamble before it, produce a field with the wrong type, or occasionally emit invalid JSON outright. For anything downstream that parses the response programmatically, that failure rate compounds.

schema-constrained decoding

Most current APIs support passing an explicit JSON schema (or, for some providers, a Pydantic/dataclass-derived schema) that constrains generation at the token level — the model is only allowed to sample tokens that keep the output on a valid path through the schema. This is a much stronger guarantee than prompting for the format: it's enforced by the decoding process, not just requested.

from pydantic import BaseModel

class Extraction(BaseModel):
    name: str
    age: int
    tags: list[str]

# passed as response_format / a tool schema, depending on the provider's API --
# the model's output is now guaranteed to satisfy this shape, not just asked to

validate anyway

import json
from pydantic import ValidationError

raw = response.output_text
try:
    data = Extraction.model_validate(json.loads(raw))
except (json.JSONDecodeError, ValidationError) as e:
    # schema-constrained decoding makes this rare, not impossible --
    # still handle it, especially across provider/model changes
    ...
Schema constraints guarantee syntactic validity and type correctness; they don't guarantee the values are correct — a schema-valid JSON object can still contain a hallucinated field. Validation and correctness are separate concerns.

where to go from here

Function/Tool Calling — the same schema-constraint mechanism, used to trigger actions instead of just shaping text.
Prompt Engineering Fundamentals — the fallback when a provider doesn't support schema constraints.
Evaluating LLM Applications — measuring extraction accuracy, not just validity.