# Semantic Operations > The semantic verb namespace (filter, rank, map, classify, summarize, cluster, dedup, and more), the ~ sigil operator family, and processing pipelines. Where [`simulate def`](/neurosymbolic/simulate/) lets a model implement a *whole function*, semantic operations are the ready-made **verbs** for the common model-mediated transforms — filter a list by meaning, rank by fit, classify a ticket, cluster near-duplicates. They are the neurosymbolic answer to `names.filter("that sound Chinese")`, made first-party: a `semantic` namespace of primitive verbs, a `~`-marked operator family, and a scoped pipeline that closes the validation loop that earlier libraries left open. ## The `~` semantic sigil `~` is Sema's universal "semantic version" marker, already established by `~=`. It extends to a systematic family: the strict operator on the left, its `~`-prefixed semantic twin on the right. | Semantic op | Meaning | Strict counterpart | |---|---|---| | `xs ~[query]` | select/lookup by meaning (getitem) | `xs[i]` index | | `a ~= b` | semantic equality → `Sim` (embedding cosine) | `a == b` | | `a ~!= b` | semantic inequality | `a != b` | | `a ~< b` `a ~> b` `a ~<= b` `a ~>= b` | semantic ordering | `<` `>` `<=` `>=` | | `a ~in b` | semantic membership | `a in b` | | `a ~+ b` | semantic combine/merge | `a + b` | | `a ~- b` | semantic remove/difference | `a - b` | | `a ~and b` `a ~or b` `a ~xor b` | semantic (model-judged) logic | `and` `or` `xor` | | `~not a` | semantic negation | `not a` | Every `~` operation carries `model.invoke` in its [effect row](/language/functions-and-effects/) — remoteness to a model is visible to policy, budgets, and monitors, never hidden. A `~` operator **never silently replaces** its strict counterpart: `xs[i]` stays exact integer indexing; `xs ~[q]` is the semantic one. The strict view is the default and the semantic view is explicitly marked, so a program's model calls are legible on sight. :::note[Sigil hygiene: bitwise NOT is now `bitnot`] Because `~` is the semantic sigil, the bitwise-NOT that C/Python spell `~` is respelled `bitnot x`. Bitwise `&`, `|`, `^`, `<<`, `>>` are unchanged. The logic gates stay complete at three tiers: strict `and`/`or`/`not`/`xor`; bitwise `& | ^ << >>` + `bitnot`; and semantic `~and`/`~or`/`~xor`/`~not`. ::: ### The coercion protocol A semantic operator needs a *representation* of its operands, and the type decides which. A struct opts in by implementing either method: ```sema struct Image: caption: str pixels: Tensor[u8] def embed(self) -> Embedding !{model.embed}: # vector representation return vision_model.embed(self.pixels) struct Doc: title: str body: str def sem_text(self) -> str !{}: # textual representation return f"{self.title}: {self.body}" similar = img_a ~= img_b # embeds each Image, cosine-compares the vectors merged = doc_a ~+ doc_b # stringifies each Doc via sem_text, then combines ``` - **`embed(self) -> Embedding`** governs similarity/ordering: `~=` and vector ordering embed both operands and cosine-compare. `image_a ~= image_b` is a genuine vector comparison, with the embedding produced by whatever model the type names — a vision tower for images, a text embedder for prose. - **`sem_text(self) -> str`** governs text-shaped ops (`~+`, `~-`, filter, map, …): the value is rendered through it before inference. Absent both, the runtime falls back to [canonical flattening](/neurosymbolic/semantic-values/) for text and the default embedder for vectors — numbers and plain collections pass through unchanged, so `3 ~< 5` stays numeric and only opted-in types are coerced. Because coercion can itself invoke a model, a single `~=` may chain models — image → vector → compare — entirely under the operator, every step journaled and effect-typed. ## The `semantic` namespace — the verbs The `semantic` namespace holds the primitive verbs. Each takes a subject plus a natural-language instruction: ```sema kept = semantic.filter(names, "names that sound Chinese") ranked = semantic.rank(candidates, by="fit for the on-call rotation") mapped = semantic.map(rows, "one-sentence risk note") gist = semantic.summarize(report) label = semantic.classify(ticket, options=["bug", "feature", "question"]) de = semantic.translate(text, to="German") ans = semantic.query(doc, "what is the counterparty?") groups = semantic.cluster(facts, threshold=0.9) # group near-duplicates merged = semantic.dedup(facts, threshold=0.9) # keep one per group ``` The **full verb set**: | Verb | Does | |---|---| | `filter` | keep items matching a natural-language predicate | | `rank` | order items by a natural-language criterion (`by=`) | | `map` | transform each item by an instruction | | `extract` | pull structured fields out of freeform input | | `summarize` | condense a subject to its gist | | `translate` | render text into another language (`to=`) | | `classify` / `choose` | assign one of a fixed option set (`options=`) | | `query` | answer a natural-language question about a subject | | `combine` | merge subjects into one | | `correct` | fix/normalize a subject | | `unique` | exact-match de-duplication | | `similar` | find items close to a subject | | `cluster` | group near-duplicates (`threshold=`) | | `dedup` | keep one representative per near-duplicate group (`threshold=`) | | `select` | pick items by meaning | Each verb is shorthand for the same pipeline that `select`/`~` use — the per-verb prompt-shaping is folded into the primitive, not left to the caller. Two of them replace a lot of hand-rolled code: - **`cluster` / `dedup`** group by `~=` similarity (single-linkage over the calibrated cosine, first-seen order preserved). `unique` is exact-match; `dedup` is near-match. They collapse the common embed → cluster → merge pipeline (a ~120-line `_purify_facts` in one ported codebase) to a single verb; the clustering backend is pluggable behind the same call. From the verified corpus, this is `dedup`/`cluster` behaving deterministically under the opt-in deterministic engine (a real embedder plugs into the same `~=` path): ```sema items = ["cat", "cat", "dog", "cat", "dog"] deduped = semantic.dedup(items, 0.99) # -> ["cat", "dog"] groups = semantic.cluster(items, 0.99) # -> [[cat,cat,cat],[dog,dog]] ``` And in an end-to-end pipeline, `semantic.dedup` collapses candidate facts before they are written into a report: ```sema # Candidate facts, de-duplicated semantically. unique_facts = semantic.dedup(["costs fell", "costs fell", "capacity grew"], 0.99) ``` :::tip[Thresholds are the calibrated cosine] The `threshold` argument to `cluster`/`dedup`/`similar` is a cutoff on the same calibrated `~=` score described in [/neurosymbolic/similarity/](/neurosymbolic/similarity/). A higher threshold groups only very-close items; `0.99` means "essentially the same." Because grouping is single-linkage over `~=`, the guarantee follows the judge's calibration. ::: ## The processing pipeline Every semantic operation runs through the same staged pipeline — the same runtime engine that powers [`simulate`](/neurosymbolic/simulate/) and [decode-with-repair](/neurosymbolic/schemas/): ``` query → [pre-processors] → inference → [post-processors] → [validate + self-repair] → result ``` Pipelines attach with an ordinary scoped `with`: ```sema with pipeline(pre=[transcribe_audio, redact_pii], post=[strip, as_json(Invoice)]): inv = semantic.extract(recording, "the invoice fields") # `recording` is transcribed and redacted before inference; the output is # stripped and parsed/validated as an Invoice — and if it fails the Invoice # contract, the rejection is fed back and re-inferred (bounded, journaled) # until it validates or RepairExhausted is raised. ``` - **Pre-processors** are functions `(query) -> query'` that transform the input before inference. A pre-processor may itself be a `simulate def` calling another model (audio → text, image → caption) — this is how Sema bridges modalities: the underlying model of a semantic op need not be a language model, and a pre-processor can change *which* modality reaches it. - **Post-processors** are functions `(output) -> output'` that transform *or validate*. A post-processor that returns a value transforms; one that returns `Err(reason)` (or a failing contract / grammar mismatch) **rejects**, feeding `reason` into a bounded repair loop — closing the loop over the model exactly as structured decode does. Grammar-constrained validation is just a post-processor: `as_json(T)` runs the schema ladder, so what returns to the caller is *guaranteed* to parse and satisfy its contract, or the operation fails honestly. Pipelines are lexically scoped and **compose**: an inner `with pipeline` layers onto the outer stack. With no active pipeline, a semantic op is raw inference — no hooks, no repair. ## Static and dynamic semantics - **Static.** `~[...]`, `~<`, `~>`, and all `semantic.*` calls derive `model.invoke`. Results are `untrusted` until a validating post-processor (a contract / `as_json[T]`) endorses them — the same trust lattice as every other model output. See [/governance/effects/](/governance/effects/). - **Dynamic.** With `[engine] deterministic = true` (or `SEMA_DETERMINISTIC=1`) the runtime dispatches semantic inference through the hermetic deterministic engine, so the *mechanics* — operator dispatch, pre/post hooks, validation and self-repair — are exact and replayable; a real model engine swaps in behind the same interface, and without a backend or that opt-in, semantic ops fail with a typed error. Every semantic op journals a `semantic.op` record (verb, query digest, repair round, status), so the debugger shows exactly what was asked, how it was pre/post-processed, and how many repair rounds it took. ## Failure modes - **A semantic op with no active pipeline and a strict downstream sink** — the result is `untrusted`; the sink rejects it. Add a validating post-processor (`as_json[T]`, a contract) to endorse it. - **A post-processor that keeps rejecting** — the repair loop is bounded (`MAX_REPAIR` rounds); exhaustion raises `RepairExhausted`, not a fake result. - **Overloading strict operators to become semantic** — rejected by design: a `~` op never silently replaces its strict twin, so model calls are never hidden from policy or review. ## How it is checked - `sema check` verifies verb arity and options, `pipeline` scoping, and that a `~` op's effect row admits `model.invoke`; it flags a misplaced pipeline clause as an unrecognized directive. - `sema run` under `[engine] deterministic = true` executes the deterministic engine so pipeline behavior — hooks, validation, repair rounds — is exactly reproducible. - `sema assure` verifies the deterministic post-processor contracts and any `as_json[T]` schema ladder attached to a pipeline. ## Where to go next - **The calibrated similarity `cluster`/`dedup` build on:** [/neurosymbolic/similarity/](/neurosymbolic/similarity/). - **Applying these verbs to real documents end-to-end:** [/guides/documents/](/guides/documents/). - **The self-repair ladder shared with structured decode:** [/neurosymbolic/schemas/](/neurosymbolic/schemas/).