#198: Grade budget — scale wall-clock with work + opt-in cost ceilings

.claude/rules/grade-layer.md

@@ -12,12 +12,21 @@ The grade layer sits between the prune engine (#6) and the diff renderer (#8). I
 - **Degraded:** `score: None, passed: False, evidence: "", reasoning: "<failure reason>"`. Three causes route here:
   1. `LLMError` retries exhausted → `reasoning="call failed: GradeLLMError"`. **Also covers a provider-specific safety-filter / no-content response** (Gemini's `finish_reason ∈ {SAFETY, RECITATION, OTHER, ...}` with empty parts is the v0.3 example — `GeminiProvider.extract_text_blocks` raises a typed `LLMResponseFormatError` per DEC-005 of #137, which propagates as an `LLMError` and lands here). The contract is provider-neutral: a future vendor with a content-filter surface MUST route through `LLMResponseFormatError` so the conservative degrade fires uniformly — `grade-artifacts` does NOT switch on provider name.
   2. `GradeOutputError` (parser failure / anchor-contract failure) → `reasoning="call failed: GradeOutputError"`.
-  3. `total_budget_seconds` exceeded → `reasoning="grade budget exceeded ..."`.
+  3. The **effective wall-clock budget** exceeded → `reasoning="grade budget exceeded ({effective_budget}s) before evaluation"` (the value is the *computed effective* budget, not the raw config field — see § "Scaled wall-clock budget + opt-in cost ceilings (#198)").
+  4. An **opt-in cost/calls/tokens ceiling** tripped (#198) → `reasoning` ∈ {`"grade call ceiling exceeded ({N} calls)"`, `"grade cost ceiling exceeded (${X})"`, `"grade token ceiling exceeded ({N} tokens)"`}. Ceilings **degrade, never raise** — no new typed error, no exit-code/AST-scan churn; `GradeBudgetExceededError` stays reserved.
 
 Aggregate `pass_rate` and `mean_score` are computed over the **scored** subset only. `aggregate_complete: bool` is `True` iff every result was scored. **Load-bearing invariant: graceful degrade, never silent drop.** Operators check `aggregate_complete` to know if the report is partial.
 
 The whole run only aborts when the **audit itself** fails (`GradeAuditWriteError` / `GradeAuditRecordTooLargeError`). A partial audit is worse than no audit.
 
+## Scaled wall-clock budget + opt-in cost ceilings (#198)
+
+The grade wall-clock budget **scales with the work**, not a flat constant. `_compute_effective_budget(*, budget_base_seconds, budget_per_pair_seconds, total_budget_seconds, num_pairs, max_concurrent_calls) -> int` (a pure module-level helper in `signalforge.grade.engine`, unit-tested without asyncio) computes `effective = budget_base_seconds + budget_per_pair_seconds × ceil(num_pairs / max_concurrent_calls)`; when `total_budget_seconds` (now `int | None`, default `None`) is set, `effective = min(scaled, total_budget_seconds)`. `effective` is always a finite positive int (`base ≥ 1`), so `asyncio.timeout(effective)` never receives `None`/0. Defaults `budget_base_seconds=60`, `budget_per_pair_seconds=20.0` are grounded in the #179 benchmark (~10s/Sonnet-judge-call; 220 pairs @ concurrency 10 → ~223s observed → ~500s budget at ~2.25× headroom — a runaway backstop, **not** a completion target). Setting `total_budget_seconds: <int>` preserves exact v0.1 absolute-cap semantics for pinned configs.
+
+**Three opt-in ceilings, all `None`=off (DEC-002/003 of #198):** `max_grade_calls` / `max_grade_cost_usd` / `max_grade_tokens`. Soft/best-effort: all pairs dispatch into the one `TaskGroup`, so a ceiling stops scheduling **new** pairs and lets in-flight calls finish (overshoot ≤ `max_concurrent_calls − 1`). `max_grade_calls` is near-hard — the slot is **reserved by incrementing a shared counter inside `_one`, immediately after `async with semaphore:` and BEFORE the first `await`** (the single event loop makes check-then-increment race-free only with no intervening await). `cost_usd`/`tokens` accumulate **after** each call returns; cache-hit pairs (#189) bypass `_one` entirely so they never count. A tripped pair degrades via `_build_degraded`, sets its slot + `completed` counter FIRST (so the synthesis pass skips it — no double-audit), then writes the GradeEvent through the **same `run_in_executor` + `asyncio.shield` path as the happy-path per-pair write** (DEC-017) — the ceiling degrade runs *inside* the concurrent `TaskGroup` region, so a synchronous fsync would block the loop and stall sibling in-flight coroutines (unlike the sequential synthesis pass, which writes sync). Ceiling-degrades count as `completed` (preserving `completed + degraded == total_pairs`). One end-of-run WARNING fires per tripped run: `"grade ceiling exceeded: {run_id, model_unique_id, ceiling, limit, completed_count, degraded_count}"` (`ceiling ∈ {calls, cost_usd, tokens}`), recording the **first** ceiling tripped. The wall-clock budget WARNING's field was renamed `total_budget_seconds` → **`effective_budget_seconds`** (carries the computed value). No `audit_schema_version` bump (GradeEvent shape unchanged; degrade reasons ride the existing `reasoning` field; no full-config-hash on GradeEvent).
+
+**Cost-ceiling pricing resolves ONCE up front (#198 QG).** `_validate_model_provider_compat` checks only the SKU *prefix* (`claude-`/`gpt-`/`gemini-`), NOT membership in `pricing.PRICES` — so a prefix-valid-but-unpriced SKU (a newer Opus, a typo) is accepted at config-load. The engine resolves `lookup(resolved_config.model)` **once before the `TaskGroup`** (when `max_grade_cost_usd` is set) and reuses it per pair; an unpriced SKU therefore raises `EstimateUnknownModelError` (CLI tier 2) **fast at orchestrator entry — before any billable call** — rather than aborting mid-run from inside a coroutine (uncaught by the per-pair `except` → `BaseExceptionGroup` → abort after paid calls). **Lesson: when a per-pair loop needs a lookup that can fail on operator-supplied config, hoist it to fail fast before the paid fan-out; "validator passed" ≠ "every downstream table has the key" (mirrors #187's registered-but-absent-key lesson).**
+
 ## Fail-closed JSONL + sidecar JSON, both end-of-write durable (DEC-006, DEC-012)
 
 Two writers in `signalforge.grade.audit`, both following the project's fail-closed pattern (fourth shipped instance — safety / draft / prune / grade):
@@ -130,7 +139,7 @@ Every `extra="ignore"` production model — `GradingResult`, `GradingReport`, `G
 
 ## `signalforge.yml` top-level namespace: `grade:` (DEC-029)
 
-The grade-stage block is `{ grade: { model, cache_ttl, max_output_tokens, max_retries_*, total_budget_seconds, min_pass_rate, min_mean_score, fail_on_below_threshold, rubric? } }`. Sibling top-level keys are reserved and silently ignored by the grade loader. `GradeConfig` uses `extra="forbid"`; `_GradeConfigFile` uses `extra="ignore"` at the top level. Mirrors the other layers' top-level-namespace pattern verbatim.
+The grade-stage block is `{ grade: { model, provider, cache_ttl, max_output_tokens, max_retries_*, max_concurrent_calls, budget_base_seconds, budget_per_pair_seconds, total_budget_seconds, max_grade_calls, max_grade_cost_usd, max_grade_tokens, min_pass_rate, min_mean_score, fail_on_below_threshold, cache_enabled, rubric? } }` (`budget_base_seconds` / `budget_per_pair_seconds` / `max_grade_calls` / `max_grade_cost_usd` / `max_grade_tokens` added by #198; `total_budget_seconds` reinterpreted as an optional absolute cap). Sibling top-level keys are reserved and silently ignored by the grade loader. `GradeConfig` uses `extra="forbid"`; `_GradeConfigFile` uses `extra="ignore"` at the top level. Mirrors the other layers' top-level-namespace pattern verbatim.
 
 ## Locked defaults: per-provider fast model + 1024 output cap (DEC-026, #187)