Schemas and Structured Output

Getting structured output from a model — a typed struct, not a blob of text — is the second half of the generative story. Sema’s answer removes the usual two-declarations problem entirely: there is no schema keyword, because the struct is the schema. This page covers the wire mapping from a struct to model-facing formats, typed decode, deterministic serialization, and the decode-and-repair ladder that turns malformed or contract-violating model output back into conditioning — so you never write the parse → catch → re-prompt → merge round-trip by hand.

The struct is the schema

A Sema struct already carries everything a wire schema needs: field names and types, sem descriptors, where refinements, coerce by normalizers, invariants, and struct-level check semantics(...) (see /neurosymbolic/semantic-values/). Declaring the same shape twice — one type for the program, one schema for the model — is a two-sources-of-truth defect, and it drifts. Sema refuses it: your struct is the schema the model must fill.

struct ParsedMemo:
    sem "Deterministic memo parse used before semantic reconciliation"
    kind: EntryKind sem "Best deterministic entry-kind signal"
    counterparty_hint: str sem "Counterparty text captured from the memo"
    reference: str sem "Bank or processor reference captured from the memo"
    amount: Option[Money] sem "Amount mentioned in the memo when present"

Wire mapping

The compiler derives, per decode-target type, a wire schema artifact — JSON Schema plus a constrained-decoding grammar — the same way it derives the meaning IR for simulate. Field names, types, refinements, and sem descriptors (as field guidance) are all part of it, and it is a public, cached, diffable build product.

• Canonical wire format is JSON. format=yaml / format=toml are accepted at explicit parse sites for config-shaped boundaries.
• A field is required unless its type is Option[T] (absent ⇒ None) or it declares a default.
• Unknown fields are a shape defect by default; extra=ignore opts out per site.
• Enums decode by variant name; payload variants as tagged objects.
• JsonValue remains the escape hatch for genuinely dynamic data, but it never bypasses this section — leaving JsonValue for a typed value goes through parse[T].

Typed decode

Three surfaces, in increasing model involvement:

# deterministic boundary parse — no model, no repair
match parse[Invoice](raw):                    # Result[Invoice, DecodeError]
    case Ok(inv): post(inv)
    case Err(e):  log.warn(e.report())        # staged defect list, field paths, blame

# model-mediated decode with self-repair
patient = decode[Patient](note, by=extractor, retries=3)?

# inside simulate def the protocol is implicit — the return type is the schema
simulate def extract(note: str) -> Patient by extractor:
    sem "Extract structured patient data from the clinical note"
    repair retries=3, patch=fields            # defaults shown; clause optional
    ensure semantics("name is written in Japanese script", result.name, alpha=0.02)

• parse[T](text, format=..., extra=...) runs schema-aligned parsing plus the full contract ladder and never invokes the generator. It returns Result[T, DecodeError].
• decode[T](text, by=model, ...) is parse[T] plus the repair loop.
• A simulate def with a structured return type has decode built in — it is the enforcement layer of the simulate construct. The repair clause (legal only in simulate def bodies) tunes it.

Effect rows follow from the target type’s contracts: a schema with only deterministic contracts gives parse[T] the row !{}; semantics(...) clauses add their judge’s model.invoke; decode[T] and repair rounds add the generator’s model.invoke.

Serialization

The wire mapping is bidirectional. serialize(v, format=json) -> str (prelude, pure !{}) is the deterministic inverse of parse[T]:

• byte-stable across runs and builds (rendering rules recorded in the ABI),
• fields in declaration order, absent Options omitted, enums in tagged form.

Round-tripping is a law, not a hope — every decode-target type carries law roundtrip: parse[T](serialize(v)) == Ok(v), discharged by the property engine (see /neurosymbolic/verification/). One mapping serves every consumer: model-facing decode, checkpoint records, journal payloads, event payloads, and bridge lowering all use this rendering — there is no second, ad-hoc serializer to drift. Serialization endorses nothing: the output string carries the value’s trust label.

serialize is not flatten

serialize(v) is the wire rendering (declaration order, descriptor-free, feeds parsers). flatten(v) is the embedding rendering (sorted keys, descriptor-inclusive, feeds ~=). Both are deterministic ABI artifacts, and they are kept separate on purpose so wire-format evolution never disturbs embedding stability. See /neurosymbolic/semantic-values/.

The decode-and-repair ladder

Validation is staged. Each stage yields a typed defect list, and repair feeds only the defects back to the model. This is the runtime-owned closed loop that earlier libraries left as a user-space try/catch/re-prompt round-robin.

• R0 — syntax. Malformed wire text. When Sema’s own engine serves the call this stage is impossible by construction (grammar-constrained decoding); for unowned models, schema-aligned parsing repairs most local damage, and the residue becomes a parser diagnostic (position, expected tokens) in the repair context.
• R1 — shape. Missing required fields, unknown fields, wrong collection arity. The repair context is a field-path diff; the model is asked to produce only what is missing.
• R2 — types and refinements. Per field: coerce by normalizer, then checked construction (a string where an i32 belongs, a null for a required int), then the where refinement. Each failure carries a ContractViolation payload: field path, descriptor, raw value, normalized value, blame.
• R3 — semantics. Deterministic invariants, then calibrated ensure semantics(...) clauses. Only ensure gates the loop; check clauses stay non-blocking graded metadata, though their Sim evidence rides along in the repair context of a round that is already happening.

Patch semantics

Under patch=fields (the default) a repair round re-prompts with the defect list, the failing fields’ sem descriptors, and a digest of the already-accepted fields; the model returns a patch object containing only the failing field paths, which the runtime merges and re-validates through the full ladder (invariants re-check on every mutation). Two consecutive patch failures on the same field escalate that round to patch=full re-emission. No round widens authority: repair executes under the same policy envelope, budget, and by model as the original call — a repair loop is more attempts, never more capability.

Typing — what a decoded value is worth

Output that passes R0–R2 and deterministic invariants has passed a sound verifier: the value endorses untrusted → validated, and those properties are checked. Calibrated R3 clauses type statistical(α) with union-bound composition and can never endorse above validated. Repair rounds are cost, not semantics — the value that exits carries identical obligations whether it took zero rounds or five.

Termination and loop-breaking

The loop is bounded by retries (default 3) and the enclosing budget (tokens/time/model_calls), whichever binds first. A candidate value already seen this loop (by content hash) ends it immediately as oscillation. Exhaustion yields a typed DecodeError (from parse/decode) or SimulationFailed (from a simulate def) whose payload is the full repair transcript — every round’s defects, patches, and judge evidence — and emits the prelude event RepairExhausted, so escalation (“route to a human queue”, “fall back to the large model”) is an ordinary subscriber. Every round is journaled; replay is exact.

Self-repair needs external grounding to help

The ladder feeds back grounded defects — parser positions, field diffs, contract violations, calibrated verdicts — not “try again.” Intrinsic self-repair without external grounded feedback degrades output; the design deliberately grounds every round in a real defect and bounds it with a real budget.

Failure modes

• Weak schema (everything Option, no refinements) → nothing for the ladder to hold; sema doctor flags all-optional decode targets.
• Repair conditioning on a drifting judge → covered by the site’s monitor (monitor-or-decay); see /governance/monitor/.
• A model that satisfies the letter of where but misses the intent → that is what R3 ensure semantics(...) plus mutation-adequacy-gated contracts exist to catch. See /neurosymbolic/verification/.

How it is checked

• sema check derives and validates the wire schema artifact, flags all-optional decode targets, and verifies the repair clause is only used inside simulate def bodies.
• sema assure discharges the roundtrip law by property fuzzing and runs the deterministic stages of the ladder against generated inputs; model-backed decode replays from the content-addressed cache.
• SEMA_STRICT=1 sema run surfaces DecodeError/SimulationFailed as hard errors, with the full repair transcript in the payload.

Where to go next

• The construct that decodes as its enforcement layer: /neurosymbolic/simulate/.
• The field descriptors and boundary contracts the ladder uses: /neurosymbolic/semantic-values/.
• The ensure/check split that R3 gates on: /neurosymbolic/contracts/.