Semantic Operations

Where simulate def lets a model implement a whole function, semantic operations are the ready-made verbs for the common model-mediated transforms — filter a list by meaning, rank by fit, classify a ticket, cluster near-duplicates. They are the neurosymbolic answer to names.filter("that sound Chinese"), made first-party: a semantic namespace of primitive verbs, a ~-marked operator family, and a scoped pipeline that closes the validation loop that earlier libraries left open.

The ~ semantic sigil

~ is Sema’s universal “semantic version” marker, already established by ~=. It extends to a systematic family: the strict operator on the left, its ~-prefixed semantic twin on the right.

Semantic op	Meaning	Strict counterpart
xs ~[query]	select/lookup by meaning (getitem)	xs[i] index
a ~= b	semantic equality → Sim (embedding cosine)	a == b
a ~!= b	semantic inequality	a != b
a ~< b a ~> b a ~<= b a ~>= b	semantic ordering	< > <= >=
a ~in b	semantic membership	a in b
a ~+ b	semantic combine/merge	a + b
a ~- b	semantic remove/difference	a - b
a ~and b a ~or b a ~xor b	semantic (model-judged) logic	and or xor
~not a	semantic negation	not a

Every ~ operation carries model.invoke in its effect row — remoteness to a model is visible to policy, budgets, and monitors, never hidden. A ~ operator never silently replaces its strict counterpart: xs[i] stays exact integer indexing; xs ~[q] is the semantic one. The strict view is the default and the semantic view is explicitly marked, so a program’s model calls are legible on sight.

Sigil hygiene: bitwise NOT is now bitnot

Because ~ is the semantic sigil, the bitwise-NOT that C/Python spell ~ is respelled bitnot x. Bitwise &, |, ^, <<, >> are unchanged. The logic gates stay complete at three tiers: strict and/or/not/xor; bitwise `& | ^ <<

+bitnot; and semantic ~and/~or/~xor/~not`.

The coercion protocol

A semantic operator needs a representation of its operands, and the type decides which. A struct opts in by implementing either method:

struct Image:
    caption: str
    pixels: Tensor[u8]
    def embed(self) -> Embedding !{model.embed}:      # vector representation
        return vision_model.embed(self.pixels)

struct Doc:
    title: str
    body: str
    def sem_text(self) -> str !{}:                    # textual representation
        return f"{self.title}: {self.body}"

similar = img_a ~= img_b        # embeds each Image, cosine-compares the vectors
merged  = doc_a ~+ doc_b        # stringifies each Doc via sem_text, then combines

• embed(self) -> Embedding governs similarity/ordering: ~= and vector ordering embed both operands and cosine-compare. image_a ~= image_b is a genuine vector comparison, with the embedding produced by whatever model the type names — a vision tower for images, a text embedder for prose.
• sem_text(self) -> str governs text-shaped ops (~+, ~-, filter, map, …): the value is rendered through it before inference. Absent both, the runtime falls back to canonical flattening for text and the default embedder for vectors — numbers and plain collections pass through unchanged, so 3 ~< 5 stays numeric and only opted-in types are coerced.

Because coercion can itself invoke a model, a single ~= may chain models — image → vector → compare — entirely under the operator, every step journaled and effect-typed.

The semantic namespace — the verbs

The semantic namespace holds the primitive verbs. Each takes a subject plus a natural-language instruction:

kept   = semantic.filter(names, "names that sound Chinese")
ranked = semantic.rank(candidates, by="fit for the on-call rotation")
mapped = semantic.map(rows, "one-sentence risk note")
gist   = semantic.summarize(report)
label  = semantic.classify(ticket, options=["bug", "feature", "question"])
de     = semantic.translate(text, to="German")
ans    = semantic.query(doc, "what is the counterparty?")
groups = semantic.cluster(facts, threshold=0.9)     # group near-duplicates
merged = semantic.dedup(facts, threshold=0.9)        # keep one per group

The full verb set:

Verb	Does
filter	keep items matching a natural-language predicate
rank	order items by a natural-language criterion (by=)
map	transform each item by an instruction
extract	pull structured fields out of freeform input
summarize	condense a subject to its gist
translate	render text into another language (to=)
classify / choose	assign one of a fixed option set (options=)
query	answer a natural-language question about a subject
combine	merge subjects into one
correct	fix/normalize a subject
unique	exact-match de-duplication
similar	find items close to a subject
cluster	group near-duplicates (threshold=)
dedup	keep one representative per near-duplicate group (threshold=)
select	pick items by meaning

Each verb is shorthand for the same pipeline that select/~ use — the per-verb prompt-shaping is folded into the primitive, not left to the caller. Two of them replace a lot of hand-rolled code:

• cluster / dedup group by ~= similarity (single-linkage over the calibrated cosine, first-seen order preserved). unique is exact-match; dedup is near-match. They collapse the common embed → cluster → merge pipeline (a ~120-line _purify_facts in one ported codebase) to a single verb; the clustering backend is pluggable behind the same call.

From the verified corpus, this is dedup/cluster behaving deterministically under the opt-in deterministic engine (a real embedder plugs into the same ~= path):

items = ["cat", "cat", "dog", "cat", "dog"]
deduped = semantic.dedup(items, 0.99)          # -> ["cat", "dog"]
groups = semantic.cluster(items, 0.99)         # -> [[cat,cat,cat],[dog,dog]]

And in an end-to-end pipeline, semantic.dedup collapses candidate facts before they are written into a report:

# Candidate facts, de-duplicated semantically.
unique_facts = semantic.dedup(["costs fell", "costs fell", "capacity grew"], 0.99)

Thresholds are the calibrated cosine

The threshold argument to cluster/dedup/similar is a cutoff on the same calibrated ~= score described in /neurosymbolic/similarity/. A higher threshold groups only very-close items; 0.99 means “essentially the same.” Because grouping is single-linkage over ~=, the guarantee follows the judge’s calibration.

The processing pipeline

Every semantic operation runs through the same staged pipeline — the same runtime engine that powers simulate and decode-with-repair:

query → [pre-processors] → inference → [post-processors] → [validate + self-repair] → result

Pipelines attach with an ordinary scoped with:

with pipeline(pre=[transcribe_audio, redact_pii], post=[strip, as_json(Invoice)]):
    inv = semantic.extract(recording, "the invoice fields")
    # `recording` is transcribed and redacted before inference; the output is
    # stripped and parsed/validated as an Invoice — and if it fails the Invoice
    # contract, the rejection is fed back and re-inferred (bounded, journaled)
    # until it validates or RepairExhausted is raised.

• Pre-processors are functions (query) -> query' that transform the input before inference. A pre-processor may itself be a simulate def calling another model (audio → text, image → caption) — this is how Sema bridges modalities: the underlying model of a semantic op need not be a language model, and a pre-processor can change which modality reaches it.
• Post-processors are functions (output) -> output' that transform or validate. A post-processor that returns a value transforms; one that returns Err(reason) (or a failing contract / grammar mismatch) rejects, feeding reason into a bounded repair loop — closing the loop over the model exactly as structured decode does. Grammar-constrained validation is just a post-processor: as_json(T) runs the schema ladder, so what returns to the caller is guaranteed to parse and satisfy its contract, or the operation fails honestly.

Pipelines are lexically scoped and compose: an inner with pipeline layers onto the outer stack. With no active pipeline, a semantic op is raw inference — no hooks, no repair.

Static and dynamic semantics

• Static. ~[...], ~<, ~>, and all semantic.* calls derive model.invoke. Results are untrusted until a validating post-processor (a contract / as_json[T]) endorses them — the same trust lattice as every other model output. See /governance/effects/.
• Dynamic. With [engine] deterministic = true (or SEMA_DETERMINISTIC=1) the runtime dispatches semantic inference through the hermetic deterministic engine, so the mechanics — operator dispatch, pre/post hooks, validation and self-repair — are exact and replayable; a real model engine swaps in behind the same interface, and without a backend or that opt-in, semantic ops fail with a typed error. Every semantic op journals a semantic.op record (verb, query digest, repair round, status), so the debugger shows exactly what was asked, how it was pre/post-processed, and how many repair rounds it took.

Failure modes

• A semantic op with no active pipeline and a strict downstream sink — the result is untrusted; the sink rejects it. Add a validating post-processor (as_json[T], a contract) to endorse it.
• A post-processor that keeps rejecting — the repair loop is bounded (MAX_REPAIR rounds); exhaustion raises RepairExhausted, not a fake result.
• Overloading strict operators to become semantic — rejected by design: a ~ op never silently replaces its strict twin, so model calls are never hidden from policy or review.

How it is checked

• sema check verifies verb arity and options, pipeline scoping, and that a ~ op’s effect row admits model.invoke; it flags a misplaced pipeline clause as an unrecognized directive.
• sema run under [engine] deterministic = true executes the deterministic engine so pipeline behavior — hooks, validation, repair rounds — is exactly reproducible.
• sema assure verifies the deterministic post-processor contracts and any as_json[T] schema ladder attached to a pipeline.

Where to go next

• The calibrated similarity cluster/dedup build on: /neurosymbolic/similarity/.
• Applying these verbs to real documents end-to-end: /guides/documents/.
• The self-repair ladder shared with structured decode: /neurosymbolic/schemas/.