# Budgets & Metering > Ambient usage metering and hard spend caps in Sema — with meter as u, with budget(tokens=, calls=), and the scheduler that batches and distributes model calls. Model spend in Sema is **ambient, not threaded.** You do not return a `(result, usage)` tuple from every function and thread it up the call stack; you wrap a scope in `with meter as u:` and every model call inside it accumulates into `u`. When you need a hard limit, `with budget(...)` is the same thing with a cap that *raises* rather than overspends. And when you issue many calls at once, the scheduler batches and distributes them without you wiring threads or queues. ## Why usage is ambient Conventional LLM code threads usage everywhere: each call returns its token counts, each wrapper adds them up, and the accounting logic tangles with the business logic — SymbolicAI's tracker was ~470 lines for exactly this. Sema removes the plumbing. A model call returns *the value only*; the surrounding meter observes usage as a side effect of the effect-handler stack ([effects](/governance/effects/) are handlers, and metering is one of them). Meters nest, and each call attributes to *all* enclosing frames. ## Metering: `with meter as u:` `with meter as u:` accumulates every model call's usage within the block into `u`: ```sema with meter as u: answer = write_report(facts) # returns the value only log.info("run", tokens=u.total_tokens, cost=u.cost, calls=u.total_calls) ``` The meter exposes: - `u.total_calls` — number of model calls - `u.prompt_tokens`, `u.completion_tokens`, `u.total_tokens` - `u.cost` — priced from `[pricing] per_token` in `sema.toml` The verified `research-agent` example reads exactly these fields at the end of a synthesis step, with no tuple threading anywhere in the call chain: ```sema mut calls = 0 mut cost = 0.0 with meter as u: _synthesis = generate("Summarize renewable energy findings", 64) calls = u.total_calls cost = u.cost ``` :::note[Meter vs the `std.usage` struct] The ambient `with meter as u:` scope is the *automatic* accounting path and is what you should reach for. The [`std.usage`](/stdlib/usage/) module exposes a `Usage` struct and an `estimate_cost(usage, pricing)` helper for the cases where you want to carry, merge (`Usage.add`), or price usage *explicitly* — for example aggregating across runs. They share the same accounting model; the meter is the ergonomic default. ::: ## Budgets: `with budget(...)` — a hard cap A budget is a meter with a ceiling. A model call that would push spend *past* the cap raises `BudgetExceeded` rather than silently overspending: ```sema with budget(calls=200, tokens=1_000_000) as b: research = deep_search(query) # BudgetExceeded if it overspends ``` Because `BudgetExceeded` is an ordinary [typed failure](/language/error-handling/), you catch and degrade rather than crash: ```sema expect summary = deep_search(query): return summary except BudgetExceeded as e: return partial_summary_so_far() ``` `b` exposes the same fields as a meter (`b.total_tokens`, `b.cost`, `b.total_calls`), so you can report *and* cap in one scope. Budgets and meters nest freely; an inner budget bounds a sub-computation while an outer meter still sees the whole thing. :::tip A `budget` scope caps *ambient runtime spend*. A [policy](/governance/policy/) can also carry a `budget <= ` rule that bounds a resource dimension per policy scope as a *governance* obligation — exceeding it is a typed denial. Use the `with budget(...)` scope for "stop this run before it costs too much"; use the policy budget rule for "this capability is bounded no matter who calls it." ::: ## Automatic scheduling, batching, and distribution The other half of budget-consciousness is *efficiency*: making model calls fast without the programmer wiring threads, queues, or load balancers. By default a model has **one warm instance** and requests run on it. When a program issues *many* requests at once — `generate_batch(prompts)`, or the SDK `chat_batch` — the scheduler **distributes** them across the configured resources: the local instance plus any remote API endpoints. Configuration is optional and lives in `sema.toml`; the default is a single local instance: ```toml # sema.toml — all optional [scheduler] endpoints = "https://api.example/v1/chat/completions,https://b/v1/..." max_batch = 16 # requests coalesced per flush (the batching-window knob) ``` - **Round-robin partition** across resources is deterministic and order-recoverable — results merge back in submission order. - **`max_batch`** caps how many requests coalesce into one flush. - **Remote resources are I/O-bound**, so their shares are dispatched concurrently across OS threads (payloads are plain strings — safe to send); the local share runs on the main thread. - A **failed or unreachable endpoint degrades to a labelled result** rather than crashing the batch. :::caution[Honest scope of parallelism] Because the tree-walking interpreter is single-threaded and a local candle model is not shareable across threads, *local* requests in a batch are processed sequentially on the one warm instance — the win there is no reload plus one code path. The genuine parallelism is across **remote** resources, and that is where distribution scales. A future concurrent runtime can widen local parallelism behind the same `generate_batch` API without any program change. ::: The scheduler is *substrate*: it is deliberately not exposed as threads, queues, or futures. An ordinary program calls `generate_batch` and gets automatic distribution; experts tune `endpoints` / `max_batch` or supply a custom resource. This pairs naturally with [`simulate`](/neurosymbolic/simulate/) and its `budget tokens=…, time=…` descriptor clause, which bounds a single generative site, whereas `with budget(...)` bounds a whole scope. ## Failure modes - **A budget scope overspends** → `BudgetExceeded` at the call that crosses the cap; no silent overrun. - **A remote endpoint is unreachable** → a labelled degraded result in the batch, not a crashed batch. - **Pricing not configured** → cost fields report from the available data; configure `[pricing] per_token` in `sema.toml` for accurate `u.cost`. ## How it's checked - Model calls run through the effect-handler stack, so metering and budgeting are handlers — the same mechanism that powers record/replay and mocking. - Every model call is journaled with its usage, so `sema run` (and replay) reproduce spend exactly. ## See also - [std.usage](/stdlib/usage/) — the explicit `Usage` struct and `estimate_cost`. - [Simulate](/neurosymbolic/simulate/) — per-site `budget tokens=…, time=…` on a generative function. - [Policies](/governance/policy/) — `budget <= ` as a governance obligation. - [Effects & Capabilities](/governance/effects/) — `model.invoke` / `model.embed` and the handler stack.