<!-- Sema documentation — Schemas and Structured Output
     Sema is a new AI-native, neurosymbolic programming language most models
     have not been trained on. Full corpus: https://sema.49.12.246.95.sslip.io/llms-full.txt
     Install language support: https://sema.49.12.246.95.sslip.io/install-skill/ -->

# Schemas and Structured Output

> There is no schema keyword — the struct is the schema. Typed decode from models, deterministic serialization, and the decode-and-repair self-repair ladder.

Getting *structured* output from a model — a typed `struct`, not a blob of text — is
the second half of the generative story. Sema's answer removes the usual
two-declarations problem entirely: **there is no `schema` keyword, because the
`struct` is the schema.** This page covers the wire mapping from a struct to
model-facing formats, typed decode, deterministic serialization, and the
decode-and-repair ladder that turns malformed or contract-violating model output
back into conditioning — so you never write the parse → catch → re-prompt → merge
round-trip by hand.

## The struct is the schema

A Sema [`struct`](/language/types/) already carries everything a wire schema needs:
field names and types, `sem` descriptors, `where` refinements, `coerce by`
normalizers, `invariant`s, and struct-level `check semantics(...)` (see
[/neurosymbolic/semantic-values/](/neurosymbolic/semantic-values/)). Declaring the
same shape twice — one type for the program, one schema for the model — is a
two-sources-of-truth defect, and it drifts. Sema refuses it: your struct *is* the
schema the model must fill.

```sema
struct ParsedMemo:
    sem "Deterministic memo parse used before semantic reconciliation"
    kind: EntryKind sem "Best deterministic entry-kind signal"
    counterparty_hint: str sem "Counterparty text captured from the memo"
    reference: str sem "Bank or processor reference captured from the memo"
    amount: Option[Money] sem "Amount mentioned in the memo when present"
```

## Wire mapping

The compiler derives, per decode-target type, a **wire schema artifact** — JSON
Schema plus a constrained-decoding grammar — the same way it derives the meaning IR
for `simulate`. Field names, types, refinements, and `sem` descriptors (as field
guidance) are all part of it, and it is a public, cached, diffable build product.

- **Canonical wire format is JSON.** `format=yaml` / `format=toml` are accepted at
  explicit parse sites for config-shaped boundaries.
- **A field is required unless** its type is `Option[T]` (absent ⇒ `None`) or it
  declares a default.
- **Unknown fields are a shape defect** by default; `extra=ignore` opts out per site.
- **Enums** decode by variant name; payload variants as tagged objects.
- `JsonValue` remains the escape hatch for genuinely dynamic data, but it never
  bypasses this section — leaving `JsonValue` for a typed value goes through
  `parse[T]`.

## Typed decode

Three surfaces, in increasing model involvement:

```sema
# deterministic boundary parse — no model, no repair
match parse[Invoice](raw):                    # Result[Invoice, DecodeError]
    case Ok(inv): post(inv)
    case Err(e):  log.warn(e.report())        # staged defect list, field paths, blame

# model-mediated decode with self-repair
patient = decode[Patient](note, by=extractor, retries=3)?

# inside simulate def the protocol is implicit — the return type is the schema
simulate def extract(note: str) -> Patient by extractor:
    sem "Extract structured patient data from the clinical note"
    repair retries=3, patch=fields            # defaults shown; clause optional
    ensure semantics("name is written in Japanese script", result.name, alpha=0.02)
```

- **`parse[T](text, format=..., extra=...)`** runs schema-aligned parsing plus the
  full contract ladder and **never invokes the generator**. It returns
  `Result[T, DecodeError]`.
- **`decode[T](text, by=model, ...)`** is `parse[T]` *plus* the repair loop.
- **A `simulate def` with a structured return type** has decode built in — it is the
  enforcement layer of the [`simulate`](/neurosymbolic/simulate/) construct. The
  `repair` clause (legal only in `simulate def` bodies) tunes it.

Effect rows follow from the target type's contracts: a schema with only
deterministic contracts gives `parse[T]` the row `!{}`; `semantics(...)` clauses add
their judge's `model.invoke`; `decode[T]` and repair rounds add the generator's
`model.invoke`.

## Serialization

The wire mapping is bidirectional. `serialize(v, format=json) -> str` (prelude, pure
`!{}`) is the deterministic inverse of `parse[T]`:

- **byte-stable** across runs and builds (rendering rules recorded in the ABI),
- **fields in declaration order**, absent `Option`s omitted, enums in tagged form.

Round-tripping is a **law**, not a hope — every decode-target type carries
`law roundtrip: parse[T](serialize(v)) == Ok(v)`, discharged by the property engine
(see [/neurosymbolic/verification/](/neurosymbolic/verification/)). One mapping
serves every consumer: model-facing decode, checkpoint records, journal payloads,
event payloads, and bridge lowering all use this rendering — there is no second,
ad-hoc serializer to drift. Serialization endorses nothing: the output string
carries the value's trust label.

:::caution[serialize is not flatten]
`serialize(v)` is the **wire** rendering (declaration order, descriptor-free, feeds
parsers). `flatten(v)` is the **embedding** rendering (sorted keys,
descriptor-inclusive, feeds `~=`). Both are deterministic ABI artifacts, and they
are kept separate on purpose so wire-format evolution never disturbs embedding
stability. See [/neurosymbolic/semantic-values/](/neurosymbolic/semantic-values/).
:::

## The decode-and-repair ladder

Validation is **staged**. Each stage yields a typed defect list, and repair feeds
*only the defects* back to the model. This is the runtime-owned closed loop that
earlier libraries left as a user-space try/catch/re-prompt round-robin.

- **R0 — syntax.** Malformed wire text. When Sema's own engine serves the call this
  stage is impossible by construction (grammar-constrained decoding); for unowned
  models, schema-aligned parsing repairs most local damage, and the residue becomes
  a parser diagnostic (position, expected tokens) in the repair context.
- **R1 — shape.** Missing required fields, unknown fields, wrong collection arity.
  The repair context is a field-path diff; the model is asked to produce only what is
  missing.
- **R2 — types and refinements.** Per field: `coerce by` normalizer, then checked
  construction (a string where an `i32` belongs, a `null` for a required `int`), then
  the `where` refinement. Each failure carries a `ContractViolation` payload: field
  path, descriptor, raw value, normalized value, blame.
- **R3 — semantics.** Deterministic `invariant`s, then calibrated `ensure
  semantics(...)` clauses. **Only `ensure` gates the loop**; `check` clauses stay
  non-blocking graded metadata, though their `Sim` evidence rides along in the
  repair context of a round that is already happening.

### Patch semantics

Under `patch=fields` (the default) a repair round re-prompts with the defect list,
the failing fields' `sem` descriptors, and a digest of the already-accepted fields;
the model returns a **patch object containing only the failing field paths**, which
the runtime merges and re-validates through the *full* ladder (invariants re-check on
every mutation). Two consecutive patch failures on the same field escalate that
round to `patch=full` re-emission. No round widens authority: repair executes under
the **same policy envelope, `budget`, and `by` model** as the original call — a
repair loop is more attempts, never more capability.

### Typing — what a decoded value is worth

Output that passes R0–R2 and deterministic invariants has passed a sound verifier:
the value endorses `untrusted → validated`, and those properties are `checked`.
Calibrated R3 clauses type `statistical(α)` with union-bound composition and can
never endorse above `validated`. **Repair rounds are cost, not semantics** — the
value that exits carries identical obligations whether it took zero rounds or five.

### Termination and loop-breaking

The loop is bounded by `retries` (default 3) **and** the enclosing `budget`
(`tokens`/`time`/`model_calls`), whichever binds first. A candidate value already
seen this loop (by content hash) ends it immediately as oscillation. Exhaustion
yields a typed `DecodeError` (from `parse`/`decode`) or `SimulationFailed` (from a
`simulate def`) whose payload is the **full repair transcript** — every round's
defects, patches, and judge evidence — and emits the prelude event `RepairExhausted`,
so escalation ("route to a human queue", "fall back to the large model") is an
ordinary subscriber. Every round is journaled; replay is exact.

:::note[Self-repair needs external grounding to help]
The ladder feeds back *grounded* defects — parser positions, field diffs, contract
violations, calibrated verdicts — not "try again." Intrinsic self-repair without
external grounded feedback degrades output; the design deliberately grounds every
round in a real defect and bounds it with a real budget.
:::

## Failure modes

- **Weak schema** (everything `Option`, no refinements) → nothing for the ladder to
  hold; `sema doctor` flags all-optional decode targets.
- **Repair conditioning on a drifting judge** → covered by the site's monitor
  (monitor-or-decay); see [/governance/monitor/](/governance/monitor/).
- **A model that satisfies the letter of `where` but misses the intent** → that is
  what R3 `ensure semantics(...)` plus mutation-adequacy-gated contracts exist to
  catch. See [/neurosymbolic/verification/](/neurosymbolic/verification/).

## How it is checked

- `sema check` derives and validates the wire schema artifact, flags all-optional
  decode targets, and verifies the `repair` clause is only used inside `simulate def`
  bodies.
- `sema assure` discharges the `roundtrip` law by property fuzzing and runs the
  deterministic stages of the ladder against generated inputs; model-backed decode
  replays from the content-addressed cache.
- `SEMA_STRICT=1 sema run` surfaces `DecodeError`/`SimulationFailed` as hard errors,
  with the full repair transcript in the payload.

## Where to go next

- **The construct that decodes as its enforcement layer:** [/neurosymbolic/simulate/](/neurosymbolic/simulate/).
- **The field descriptors and boundary contracts the ladder uses:** [/neurosymbolic/semantic-values/](/neurosymbolic/semantic-values/).
- **The `ensure`/`check` split that R3 gates on:** [/neurosymbolic/contracts/](/neurosymbolic/contracts/).
