Repository navigation
feat(merge-sweep): add failure history tracking and Track A-resolve [OMN-7573] - #1
Conversation
…OMN-7573] Add FailureCategory enum, FailureHistoryEntry/Summary dataclasses, and cross-run failure escalation logic to NodeMergeSweep. CHRONIC (>=5 consecutive failures) and RECIDIVIST (>=3 polishes still failing) PRs skip polish automatically. Add Track A-resolve for MERGEABLE+BLOCKED+GREEN PRs (thread resolution). Fix is_merge_ready to exclude BLOCKED PRs so they fall through to A-resolve. Bump contract version to 1.1.0. Add 6 new tests covering failure history escalation.
📝 WalkthroughWalkthroughThe changes introduce cross-run failure tracking and a new PR classification track (A-resolve) for thread resolution. The contract version incremented to 1.1.0, failure history models were added, and MergeSweepRequest/Result extended to support failure categorization and escalation logic based on historical failure patterns. Changes
Sequence DiagramsequenceDiagram
participant Req as MergeSweepRequest
participant Handler as Handler
participant Classify as Classifier
participant History as FailureHistory
participant Result as MergeSweepResult
Req->>Handler: failure_history, run_id, PRs
Handler->>Classify: _classify_pr() with history context
Classify->>History: check failure_categories
History-->>Classify: STUCK, CHRONIC, RECIDIVIST status
Classify-->>Handler: (track, reason, failure_categories)
alt Track A-resolve
Handler->>Handler: _needs_thread_resolution()
Handler->>Handler: mark A_RESOLVE track
else Track B_POLISH
Handler->>Handler: _should_skip_polish() via history
Handler->>Handler: escalate to SKIP if threshold met
else Track A
Handler->>Handler: mergeable, proceed
end
Handler->>Handler: _compute_failure_summary()
Handler->>Result: failure_history_summary
Result-->>Req: MergeSweepResult
Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py (1)
84-92:cleaned_mergedis never populated.
FailureHistorySummaryexposes this field, but_compute_failure_summary()has no current-run input and always returns the default0. Either derive it fromclassified+history, or leave it out of the 1.1.0 response until it is actually wired up.Also applies to: 307-329
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py` around lines 84 - 92, FailureHistorySummary exposes cleaned_merged but it is never populated; update _compute_failure_summary to compute cleaned_merged (or remove it) — locate the FailureHistorySummary dataclass and the _compute_failure_summary function and either: 1) derive cleaned_merged from the current-run classified results combined with history (e.g., count items in classified where label/state indicates a cleaned+merged outcome and/or where history shows previously cleaned then merged), or 2) remove cleaned_merged from FailureHistorySummary and any response construction until you wire it up; ensure updates reference the FailureHistorySummary class and the _compute_failure_summary method and adjust any callers that expect cleaned_merged accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py`:
- Around line 219-240: The current ordering calls self._needs_thread_resolution
before self._needs_polish so PRs with review_decision == "CHANGES_REQUESTED" are
routed to PRTrack.A_RESOLVE instead of PRTrack.B_POLISH; fix by either moving
the polish block (the if self._needs_polish(pr, require_approval): ... that
returns PRTrack.B_POLISH) above the thread-resolution block (the if
self._needs_thread_resolution(pr): ... that returns PRTrack.A_RESOLVE) or adjust
self._needs_thread_resolution to explicitly ignore PRs where pr.review_decision
== "CHANGES_REQUESTED" so that CHANGES_REQUESTED PRs are evaluated by
self._needs_polish and routed to PRTrack.B_POLISH.
---
Nitpick comments:
In `@src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py`:
- Around line 84-92: FailureHistorySummary exposes cleaned_merged but it is
never populated; update _compute_failure_summary to compute cleaned_merged (or
remove it) — locate the FailureHistorySummary dataclass and the
_compute_failure_summary function and either: 1) derive cleaned_merged from the
current-run classified results combined with history (e.g., count items in
classified where label/state indicates a cleaned+merged outcome and/or where
history shows previously cleaned then merged), or 2) remove cleaned_merged from
FailureHistorySummary and any response construction until you wire it up; ensure
updates reference the FailureHistorySummary class and the
_compute_failure_summary method and adjust any callers that expect
cleaned_merged accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Lite
Run ID: 3d0d63a3-9ce5-48a9-87d4-04528eea5851
📒 Files selected for processing (3)
src/omnimarket/nodes/node_merge_sweep/contract.yamlsrc/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.pytests/test_golden_chain_merge_sweep.py
…orts Adds a new `pr_checks_live` check dimension to HandlerOverseerVerifier that shells out to `gh pr checks <num> --repo <owner>/<name> --json bucket,state, conclusion,name,completedAt` for each PR an agent self-reports as green. Previously the verifier ran five pure-Python checks and rubber-stamped PR completion claims without ever contacting GitHub — the #1 leverage failure mode in overnight dispatch ("agents lie"). Changes: - ModelClaimedPr(pr_number, repo) model for (PR, repo) claim tuples - ModelVerifierRequest gains optional `claimed_prs: list[ModelClaimedPr]` - _check_pr_checks_live: per-PR subprocess shell-out with 30s timeout, per-PR error isolation, `bucket=="pass"` + conclusion whitelist gating; any red check surfaces PR + failing check name in the failure reason - Follows canonical shell-out pattern from node_pr_snapshot_effect (subprocess .run, capture_output=True, text=True, isolated try/except for timeout / OSError / JSONDecodeError) - `pr_checks_live` inserted at priority slot 4 (above allowed_action_scope): a broken PR is stronger evidence of agent lying than a schema quibble - Skipped with a note when `claimed_prs` is empty, preserving zero-I/O semantics for non-PR verifications Tests (6 new + 19 existing overseer tests pass): - pr_checks_live skipped when no PRs claimed (subprocess never invoked) - two green PRs → verdict PASS, pr_checks_live passed - one green + one red → verdict FAIL, failing PR + check name in message - subprocess timeout → verdict FAIL, descriptive timeout message - gh non-zero exit → verdict FAIL with exit code + stderr - malformed JSON → verdict FAIL with parse-error message - existing consumer golden-chain test updated for 6-dim expectation `uv run ruff format/check` clean, `uv run mypy src/ --strict` clean (878 files).
…orts (#354) * OMN-9273: wire overseer-verify to gh pr checks against agent self-reports Adds a new `pr_checks_live` check dimension to HandlerOverseerVerifier that shells out to `gh pr checks <num> --repo <owner>/<name> --json bucket,state, conclusion,name,completedAt` for each PR an agent self-reports as green. Previously the verifier ran five pure-Python checks and rubber-stamped PR completion claims without ever contacting GitHub — the #1 leverage failure mode in overnight dispatch ("agents lie"). Changes: - ModelClaimedPr(pr_number, repo) model for (PR, repo) claim tuples - ModelVerifierRequest gains optional `claimed_prs: list[ModelClaimedPr]` - _check_pr_checks_live: per-PR subprocess shell-out with 30s timeout, per-PR error isolation, `bucket=="pass"` + conclusion whitelist gating; any red check surfaces PR + failing check name in the failure reason - Follows canonical shell-out pattern from node_pr_snapshot_effect (subprocess .run, capture_output=True, text=True, isolated try/except for timeout / OSError / JSONDecodeError) - `pr_checks_live` inserted at priority slot 4 (above allowed_action_scope): a broken PR is stronger evidence of agent lying than a schema quibble - Skipped with a note when `claimed_prs` is empty, preserving zero-I/O semantics for non-PR verifications Tests (6 new + 19 existing overseer tests pass): - pr_checks_live skipped when no PRs claimed (subprocess never invoked) - two green PRs → verdict PASS, pr_checks_live passed - one green + one red → verdict FAIL, failing PR + check name in message - subprocess timeout → verdict FAIL, descriptive timeout message - gh non-zero exit → verdict FAIL with exit code + stderr - malformed JSON → verdict FAIL with parse-error message - existing consumer golden-chain test updated for 6-dim expectation `uv run ruff format/check` clean, `uv run mypy src/ --strict` clean (878 files). * OMN-9273: address CodeRabbit review — cap fan-out, guard JSON shape Two mechanical hardening fixes raised by CodeRabbit: 1. **Fan-out cap** — Add `_MAX_CLAIMED_PRS = 20` constant. `_check_pr_checks_live` fails fast with a descriptive message when `claimed_prs` exceeds the cap, preventing pathological N-PR invocations from stalling verification up to N×30s. 20 keeps the worst-case at ~10min, well above any realistic agent self-report size. 2. **JSON shape guard** — Previously `raw: list[dict[str, Any]] = json.loads(...)` trusted gh's output shape. A valid non-list JSON (e.g. top-level object) would iterate over dict keys (strings) and `AttributeError` on `.get()`, breaking per-PR error isolation and crashing the verifier. Now explicitly assert top-level list + per-item dict, returning a `JSON shape error` message matching the existing parse-error envelope shape. Tests added (3): - too many claimed PRs → fail fast, subprocess.run never invoked - non-list JSON (top-level object) → JSON shape error, no crash - list with non-dict items → JSON shape error, no crash Full suite: 2228 passed (3 new, 2225 prior). ruff + mypy --strict clean. * OMN-9273: coerce check name to str to prevent TypeError in sort/join CodeRabbit flagged that red_checks sourced from untyped JSON could contain None or int values for `name`, causing TypeError when sort/join runs. Coerce via `str(item.get('name') or '<unnamed>')`. New test covers None + int + str mixed rows, asserts sorted/joined output contains each coerced value.
CR Major #1 (durable_evidence_gate.py:~141-159): extract_cited_merge_commits accepted any string sha and let ModelCitedMergeCommit (min_length=7) raise ValidationError on malformed short SHAs — contract violation vs the docstring's "malformed citations are skipped" promise. Added _SHA_RE guard ([0-9a-fA-F]{7,40}) before parse_pr_url; malformed SHAs are now silently skipped. Regression test: test_extract_skips_malformed_short_sha. CR Major #2 (same file, dedupe + cross-set comparison): Citation dedupe and CONTRACT_ON_OCC_MAIN comparison were keyed on raw pr_url, so /pull/123 and /pull/123/files for the same PR were treated as distinct citations — caused double gh_pr_view() calls AND false CONTRACT_ON_OCC_MAIN hard-fails when local and OCC main spelled the same PR differently. Switched both keys to the normalized identity (repo, pr_number, sha). Regression tests: test_url_variants_dedupe_to_same_citation and test_url_variants_across_local_and_main_do_not_hard_fail. deploy-gate root cause: Action matcher src/*/nodes/**/*.py flagged the pure-logic gate service as a runtime path. Added dod-005 to contracts/OMN-10407.yaml documenting the deploy posture honestly: this slice is pure logic + Protocol probes, no container/topic touched, and concrete probe wiring + FSM integration ship in a follow-up ticket carrying its own deploy evidence. check_value contains "deploy" so the gate recognizes intent. NO skip token used. Verification: - 70/70 node_dod_verify unit tests green (was 67, +3 new). - Full uv run pytest tests/ -m "not kafka and not integration": 2786 passed / 0 failed (was 2783). - uv run ruff format/check, uv run mypy --strict on node_dod_verify/, all clean. - OMNIBASE_INFRA_PATH=... pre-commit run --all-files: all hooks pass. - Local replay of validate_pr_deploy_required.py against the three changed files now returns DEPLOY GATE PASSED. CR fingerprints (CodeRabbit auto-resolves on resolution): phantom:medusa:ocelot:97a9ca79-253e-49f4-bb67-27581b204d27 4e71b3a2
…mniNode-ai#467) * feat(OMN-10407): durable-evidence gate before Linear Done transition Adds a pure-logic DurableEvidenceGate service in node_dod_verify that refuses Linear Done when the durable evidence trail is local-only. OMN-9855 incident shape (2026-04-30): - Worker probed implementation already on omnibase_core/main (PR OmniNode-ai#949) - Generated DoD receipt LOCALLY at evidence/OMN-9855/dod_report.json, never committed - Updated Linear dod_evidence text to point at PR OmniNode-ai#949 - onex_change_control/main still had stale contract citing superseded OmniNode-ai#926 - Linear flipped Done; durable trail unverifiable from origin alone The gate enforces three checks before any Linear Done transition: 1. RECEIPT_TRACKED — evidence/<TICKET>/dod_report.json must be tracked on onex_change_control/main (not just present on disk). 2. CONTRACT_CITES_MERGE_COMMIT — every cited pr_url has gh-state MERGED and mergeCommit.oid matches the cited commit_sha. Closed/superseded PRs HARD FAIL. 3. CONTRACT_ON_OCC_MAIN — the contract version on OCC main contains the real merge-commit citation. Stale main HARD FAIL with explicit "open an OCC PR first" remediation hint. Failure raises DurableEvidenceGateError carrying the structured ModelDurableEvidenceGateResult; the first failed check supplies the remediation hint. Architecture: - Pure-logic gate + Protocol-typed probes (GitTrackedProbe, GhPrViewProbe, ContractOnRefLoader). Tests inject deterministic stubs. - New schema module model_durable_evidence_gate.py is pure (no I/O, no env, no time) per onex_change_control D-008 schema purity rules. - Production probe wiring (subprocess-backed git ls-tree / gh pr view / yaml load on git ref) is intentionally out of scope for this slice; follow-up ticket will wire the gate into the FSM phase ordering. DoD evidence: - 16 unit tests cover untracked receipt, superseded PR, wrong SHA, stale main, missing-on-main, structured error raise, citation extraction (top-level + nested + dedup + malformed), URL parser. - All 2783 non-kafka non-integration omnimarket tests pass. - mypy --strict clean on src/omnimarket/nodes/node_dod_verify/. - ruff format + check clean; pre-commit hooks all green. * fix(OMN-10407): address CR findings + deploy-gate (PR OmniNode-ai#467) CR Major OmniNode-ai#1 (durable_evidence_gate.py:~141-159): extract_cited_merge_commits accepted any string sha and let ModelCitedMergeCommit (min_length=7) raise ValidationError on malformed short SHAs — contract violation vs the docstring's "malformed citations are skipped" promise. Added _SHA_RE guard ([0-9a-fA-F]{7,40}) before parse_pr_url; malformed SHAs are now silently skipped. Regression test: test_extract_skips_malformed_short_sha. CR Major OmniNode-ai#2 (same file, dedupe + cross-set comparison): Citation dedupe and CONTRACT_ON_OCC_MAIN comparison were keyed on raw pr_url, so OmniNode-ai/pull/123 and OmniNode-ai/pull/123/files for the same PR were treated as distinct citations — caused double gh_pr_view() calls AND false CONTRACT_ON_OCC_MAIN hard-fails when local and OCC main spelled the same PR differently. Switched both keys to the normalized identity (repo, pr_number, sha). Regression tests: test_url_variants_dedupe_to_same_citation and test_url_variants_across_local_and_main_do_not_hard_fail. deploy-gate root cause: Action matcher src/*/nodes/**/*.py flagged the pure-logic gate service as a runtime path. Added dod-005 to contracts/OMN-10407.yaml documenting the deploy posture honestly: this slice is pure logic + Protocol probes, no container/topic touched, and concrete probe wiring + FSM integration ship in a follow-up ticket carrying its own deploy evidence. check_value contains "deploy" so the gate recognizes intent. NO skip token used. Verification: - 70/70 node_dod_verify unit tests green (was 67, +3 new). - Full uv run pytest tests/ -m "not kafka and not integration": 2786 passed / 0 failed (was 2783). - uv run ruff format/check, uv run mypy --strict on node_dod_verify/, all clean. - OMNIBASE_INFRA_PATH=... pre-commit run --all-files: all hooks pass. - Local replay of validate_pr_deploy_required.py against the three changed files now returns DEPLOY GATE PASSED. CR fingerprints (CodeRabbit auto-resolves on resolution): phantom:medusa:ocelot:97a9ca79-253e-49f4-bb67-27581b204d27 4e71b3a2
… ask; persist rejected candidate content (#1630) * fix(OMN-14004): code_generation quality gate assumes Python for every ask; persist rejected candidate content Two coupled defects found by the ratchet-canary (OMN-14000): 1. `_check_compiles_without_errors` unconditionally ran `ast.parse` on the candidate for ANY `code_generation` ask, regardless of what artifact language was actually requested. A YAML contract-fragment ask was false-rejected with "does not compile as Python: invalid syntax", then escalated through cheap_cloud -> cheap_frontier -> claude, burning cloud rate-limit budget for nothing. Fix: a fenced block's own language tag (```yaml``/```json``) now selects the parser for THAT block; an untagged fence or raw (non-fenced) content keeps the original Python-only behavior byte-for-byte unchanged, so the common Python case is unaffected. 2. `LocalDelegationDispatchPort`'s escalation loop logged only the gate's failure reasons on a quality-gate FAIL, never the rejected candidate's own text — so a false-reject (like #1) couldn't be debugged after the fact; the terminal payload's cumulative `content` field only ever carried the LAST attempt across an escalation chain. Fix: log the full rejected candidate content at the escalation point. The capture-file this logger writes to is already promoted to a content-addressed artifact by the CLI receipt layer (`receipt_mode.py`), so this makes the rejected text durable evidence without adding a new persistence surface. Verified live: `onex delegate "<YAML ask>" --task-type code_generation` was reproduced failing on origin/dev with `MALFORMED: response does not compile as Python: invalid syntax`; new unit tests lock in both fixes (8 new tests, all pass) plus regression guards for the untouched Python/raw-content paths. Full local gates green: ruff format/check clean, mypy --strict clean (2693 files), full pytest (no -k): 12104 passed, 5 failed + 1 error — identical pre-existing infra-dependent failures (Postgres/Kafka unavailable locally), same count/names as an unmodified origin/dev baseline run. Blocks scaling OMN-14000 (ratchet burn-down) past the `document` task-type workaround. * fix(OMN-14004): accept fence info strings in quality gate * style(OMN-14004): format quality gate tests
…ct schema (#1631) * fix(OMN-14000): migrate 6 nodes' handler_routing to the nested contract schema Ratchet burn-down batch #1 (missing_handler_routing, the ratchet-canary's #1 target). The arch_handler_contract_compliance gate (onex_change_control/src/onex_change_control/scanners/handler_contract_compliance.py) only recognizes the nested `handler_routing.handlers[].handler.{name,module}` schema; these 6 nodes were still on the legacy flat `handler_class`/ `handler_module` string schema and false-flagged as missing_handler_routing. Adds the nested `handler:` block (and `routing_strategy: operation_match` where absent) as a purely additive sibling to each existing handlers[] entry — every existing key/value/comment is preserved unchanged, nothing removed. Nodes: node_adr_canary_orchestrator, node_adr_document_ingestion_effect, node_kb_adr_publisher, node_kreuzberg_parse_effect, node_memory_lifecycle_orchestrator (3 handler entries), node_persona_lifecycle_orchestrator (2 handler entries). 4 of these 6 fixes (kb_adr_publisher, kreuzberg_parse_effect, memory_lifecycle_orchestrator, persona_lifecycle_orchestrator) were generated by the LOCAL delegate tier (Qwen3.6-35B-A3B via `.201`, `onex delegate --task-type document`) — 4/4 first-try, gate-verified, no cloud escalation. This is the flagship local-first proof for the ratchet burn-down lane (OMN-14000): mechanical, self-grading-oracle tech debt reduced by a local model, not a frontier one. Gate-verified: `python -m onex_change_control.validators.arch_handler_contract_compliance --repo-root omnimarket` — Compliant 336 -> 344 (+8 handler files), New violations 95 -> 87 (-8), Total handlers unchanged at 431. No new violations introduced. OMN-14000 * revert(OMN-14000): drop 4 nodes coupled to a separate state-coverage-baseline gate state-coverage-gate (scripts/validate_state_coverage.py, OMN-13781) grandfathers pre-existing uncovered declared-event-topic debt via scripts/validation/state_coverage_baseline.txt, but revokes the grandfather clause in --strict mode for any node DIRECTLY TOUCHED by a diff. All 4 of node_adr_canary_orchestrator, node_adr_document_ingestion_effect, node_memory_lifecycle_orchestrator, and node_persona_lifecycle_orchestrator are pre-listed in that baseline with genuinely uncovered output topics (unrelated to handler_routing) — touching their contract.yaml for the missing_handler_routing fix un-grandfathers that separate debt and fails CI. Fixing that properly means adding real test coverage for those declared topics — out of scope for a small, revertible ratchet-type batch (touches a DIFFERENT ratchet: declared-state test coverage, not handler_routing). Revert these 4 back to the legacy flat handler_routing schema for this PR; keep only node_kb_adr_publisher and node_kreuzberg_parse_effect, which have no state-coverage-baseline coupling and gate-verify clean on their own. Filed OMN-14009 to track the 4 deferred nodes + the general coupling risk for future missing_handler_routing batches.
…fix (local-first, offline) (#1626) * feat(OMN-13998): 2x2 SWE-discriminator harness + truncation-confound fix Promotes the OMN-13988 de-risk spike into a durable, tested harness and fixes the capability-vs-plumbing confound (blocker #2). Local-first, offline over captured artifacts (proof_class = offline; NOT a live runtime loop). New package omnimarket.delegation.swe_discriminator (extends graded_ladder): - 2x2 arm runner (monolith/decomposed x frontier/cost_routed), decomposer + per-slice worker + concat integration, balanced-bracket slice parser - offline blind deterministic floor grader (verifier != runner) - classify.py: detect_truncation + classify_run separate PASS/FAIL_WRONG (capability) from TRUNCATED/BLOCKED (plumbing/infra, excluded) so a token-cap truncation cannot deflate the hard-tier numbers (OMN-13335 hazard) - model_client: finish_reason capture, reasoning-aware budget (SWE_MAX_TOKENS, default 16384), 429 retry/backoff, env-overridable frontier; no hardcoded IPs - run_smoke: 2-phase RUN/GRADE orchestrator, pass^k repeats + PassKCell aggregate Evidence (docs/evidence/2026-07-05-swe-discriminator-smoke/): - grader calibration (selftest): real merged solutions PASS floor, pre-fix FAIL - 13 unit tests incl truncation detector proven on the EXACT live L3 repro - live pass^3 on L1 local arms (3/3); genuine decomposition (parsed slice + tax) - canonical 4-arm run: L1 local pass, L3 local correctly TRUNCATED, frontier BLOCKED Full 2x2 headline run remains gated on frontier reachability (blocker #1) and a larger hard-task budget/reasoning control (follow-ups). mypy --strict + ruff clean. * fix(OMN-13998): resolve tier endpoints from ladder_rungs config (url-authority gate) Removes the literal frontier URL const and the SWE_FRONTIER_URL env read that tripped the url-authority gate (2 violations). Endpoints now resolve from the committed graded_ladder ladder_rungs.yaml routing config (the same integration catalog the ladder recorder uses); rung selection is by id via SWE_FRONTIER_RUNG / SWE_COST_RUNG, keys/endpoints named by the rung, never a hardcoded URL. Verified: validator_url_authority reports 0 swe_discriminator violations; mypy --strict + ruff clean; 13 unit tests + grader selftest green; resolve_tier resolves frontier->glm-5.2 and cost_routed->local from config. * fix(OMN-13998): pass SWE discriminator runtime config explicitly * fix(OMN-13998): harden SWE smoke artifacts and parsing * ci(OMN-13998): refresh stuck checks * ci(OMN-13998): harden reviewer uv sync * ci(OMN-13998): extend hostile reviewer timeout * ci(OMN-13998): retrigger stuck hostile reviewer * fix(OMN-13998): handle malformed chat choices * test(OMN-13998): match urlopen timeout keyword * ci(OMN-13998): refresh cancelled check set * ci(OMN-13998): retrigger stale queued validators * ci(OMN-13998): refresh cancelled hostile reviewer * test(OMN-13998): accept ROI overlay in delegation ladder shim
…on-consumer honor path (ROI tier-flip enrichment) (#1636) * feat(OMN-14018): per-run backend pin on context-ROI runner → generation consumer honors it The context-ROI battery only ever routed to the contract-declared local tier, so context_roi_scores held a single tier (488 local rows, 100% success). To capture genuine per-tier graded outcomes (a real sub-floor cheap_cloud cohort for the demonstrable escalation-skip flip, OMN-14011 Path 1) a run must be pinnable to a chosen backend. Runner side: - ModelContextRoiRunRequest.target_endpoint (optional, default empty). - HandlerContextRoiRunner threads it onto every generation command as forced_endpoint_ref (only when non-empty; empty preserves the command shape). - run_context_roi_battery.py --target-endpoint flag → target_endpoint. Generation-consumer side (so the pin is REAL, not an inert field): - ModelNodeGenerationRequest.forced_endpoint_ref (optional, default empty). - handle() pins the STARTING route to the forced backend's tier via _forced_starting_route, which resolves the concrete model+endpoint through the SAME routing authority (delta with min_tier_name) the escalation path already uses. Fail-safe: an unresolvable ref keeps the contract-declared route (never crashes). Empty ref = byte-identical to prior behavior. Tests (5 new, all through the real routing authority — no mocked routing): - runner threads/omits forced_endpoint_ref by target_endpoint presence; - forced cloud pin makes generation attempt #1 post to the CLOUD tier's model+endpoint (vs contract-local); empty preserves local; bad ref falls back to local without crashing. 510 pass across gen-consumer + runner + scripts. - ruff/mypy --strict clean; check-no-faked-boundary + leaked-literals gates pass. NOTE (honest scope): this lands the enrichment MECHANISM. The live sub-floor capture (≥5 real cheap_cloud rows) + observed escalation-skip A/B is gated on OMN-14001/#1629 merged AND the stability-lane runtime redeployed with this code + OMNIDASH_ANALYTICS_DB_URL set (the ticket's own Blocked-by). No synthetic rows were authored. Refs OMN-14018, OMN-14011. * test(OMN-14018): assert runner terminal output topics by literal (state-coverage gate) Touching node_context_roi_runner activates the contract-state-coverage gate for it; the two declared terminal output states were asserted only by substring, so the gate saw them uncovered. Assert the full literal topic names (onex.evt.omnimarket.context-roi-run-{completed,failed}.v1) in the two emit tests -- a stronger assertion (exact topic, not substring) that covers both states. * fix(OMN-14018): forced route records BACKEND id as endpoint_class (overlay-countable capture) _forced_starting_route left endpoint_ref="" (mirroring escalated routes), so the attempt + run-level benchmark recorded the bare TIER NAME (e.g. 'cheap_cloud') as endpoint_class. The context_roi_scores row's endpoint_ref = that endpoint_class, and the ROI overlay maps a row to a tier via tier_for_backend(endpoint_ref) which resolves a BACKEND id ('cloud-glm' -> cheap_cloud) but returns None for a bare tier name -> every captured per-tier row would be silently DROPPED, yielding zero usable sub-floor rows for the OMN-14011 Path-1 flip. Fix: carry the forced backend id on endpoint_ref. Routing is unaffected — _call_llm posts to endpoint_url when authority_resolved is True (endpoint_ref is attribution-only there). Tests now assert the attempt AND run-level benchmark record the backend id as endpoint_class. Refs OMN-14018, OMN-14011. * ci(OMN-14018): refresh hostile reviewer context * test(OMN-14018): accept ROI overlay in delegation ladder shim
…schema (#1768) node_projection_baselines defined local models sharing almost no field names with the real producer contract (omnibase_infra's ModelBaselinesSnapshotEvent / ModelBaselinesComparisonRow / ModelBaselinesTrendRow / ModelBaselinesBreakdownRow): a required-no-default pattern_id the producer never sends, entirely fictional trend/breakdown concepts on the consumer side, and a snapshot write that bound columns (patterns_compared/patterns_recommended) that don't exist on baselines_snapshots. Live-checked on .201 stability-test before fixing: the projection_baselines consumer group had committed offset 2 with zero lag (both real events on the topic consumed) yet all four target tables held zero rows -- confirming failure mode #1 from the ticket (every real event silently crashed inside the runtime's catch-all dispatch boundary, never a partial degrade). Fix (both consumer paths declared in this node's contract, matching the OMN-14490/14506 precedent): - HandlerProjectionBaselines (the live-wired RuntimeLocal shim) and BaselinesProjectionRunner (the standalone Kafka->Postgres runner, still declared in docker-compose.projection.yml) both now validate the canonical omnibase_infra producer models directly instead of local slim copies, stripping transport-injected `_`-prefixed keys first. - New migration 0002 recreates baselines_comparisons/baselines_trend/ baselines_breakdown with the real producer row shapes (treatment_*/ control_*/roi_pct, cohort/session_count, pattern_id/pattern_label). All three tables held zero rows in every known environment, so there was no data to preserve across the shape change. - Wired the OMN-13548 DLQ path: BaselinesProjectionRunner declares poison_dlq_topics/publish_dlq (the base class's own POISON safety net routes an escaped ValidationError there); HandlerProjectionBaselines documents that a malformed payload propagates to the wiring layer's own dlq_topics route, matching the established RuntimeLocal-shim precedent. Two sibling nodes (node_projection_baselines_roi, node_projection_baselines_quality) imported this node's now-deleted fictional models. Gave each its own local, behavior-identical copy to avoid a compile break -- NOT a fix, tracked separately as OMN-14630 since their own aggregation semantics (token_delta, retry_delta, confidence tiers) have no producer analog and need their own redesign. Behavioral proof: tests/test_omn14513_baselines_seam.py drives the producer's real envelope through the real unwrap_envelope decode seam into BOTH consumer paths. Verified RED against the pre-fix models on this branch (pydantic ValidationError: comparisons.0.pattern_id Field required; runner silently degraded to a blank/'shadow'-keyed default row) and GREEN against the canonical ones. Closes OMN-14513
…rect its false provenance (#1999) * fix(OMN-15597): re-pin the AC5 census to the ticket's SHA, correct its provenance Merged #1998 (3cb2633) pinned the AC5 corpus at onex_change_control@9089ffa9 and asserted -- in the corpus module docstring and in TestAc5PinnedCorpusReCensus::test_pinned_corpus_is_the_census_the_ticket_names -- that its 59 values / 33 contracts were the ticket's census "reproduced exactly". That is false. It was a cardinality coincidence over a different contract set: OMN-15597's census point is d7b16271 (its own stated 32,593 scan total is exactly that SHA), and contracts/OMN-15239.yaml and contracts/OMN-15488.yaml do not exist at 9089ffa9 at all -- so the pinned corpus structurally could not contain OMN-15488, the live motivating case AC5 exists to protect. A green assertion encoding a false provenance claim is the same defect class #1998 was opened to close. Re-pinned to d7b16271 and regenerated with the oracle already committed in this repo (vendored pre-fix guard rejects AND current guard accepts), via the fixture's own documented git-archive procedure -- read-only, AC6 respected: OCC_CENSUS_SHA 9089ffa9 -> d7b16271 OCC_CENSUS_TOTAL_COMMAND_CHECK_VALUES 32564 -> 32593 class / contracts 59/33 -> 63/35 The new class is a strict superset of the old 59 (extras: OMN-15239.yaml#0, OMN-15488.yaml#0/#1/#2). The control at 9089ffa9 reproduces the outgoing snapshot exactly, so the divergence is the SHA, not the method. Provenance is now stated rather than implied: the docstring and the renamed test_pinned_corpus_is_the_ticket_census_point_under_the_committed_oracle say which oracle at which SHA, and record that the ticket's literal 59/33 was read with a different oracle (its unbalanced-$( rule, which yields 56/32 here) and is superseded, not reproduced. New test_pinned_corpus_covers_every_contract_the_ticket_enumerates makes the provenance claim falsifiable by containment of the ticket's 33 enumerated contract names -- something a cardinality coincidence cannot satisfy. Also corrects the docstring's "one day of OCC commits later" for 1e6b75f8: it is 76 minutes, same day. Findings by the #1998 adversarial verifier, independently re-measured and confirmed by lane G3 (Linear comment 921b6c7b on OMN-15597). Evidence-Ticket: OMN-15597 * chore(OMN-15597): mint a fresh head so ci.yml re-runs with OCC#5845 merged The first head ran ci.yml before the OCC companion existed, so occ-preflight / eligibility failed, OCC Preflight Dependency refused to start heavy CI, and CI Summary fail-closed -- every test-bearing job was skipped and this diff had zero Linux-runner proof. OCC#5845 merged 2026-08-01T13:24:08Z (born-admissible: app/onexbot-occ-writer, both commits authored AND committed by occ-companion-effect, single-parent, zero human commits) and occ-preflight / eligibility now passes on the existing head. No diff change. Evidence-Ticket: OMN-15597
…filename shape (#2200) * fix(OMN-16071): the OCC companion writer is add-only by path, not by filename shape Defect 1, the defect this ticket is titled after, is the last open half of OMN-16071 (Defect 2 landed in #2081/#2086). Its AC is literal: "the autobind path must be strictly add-only -- never open an existing receipt file for write." Every prior PR in this ticket's chain hardened the GUARD; none touched the WRITER, which is why the ticket carries a standing DO-NOT-FLIP. WHY THE EXISTING MECHANISM DID NOT CATCH IT. `assert_append_only_emissions` (OMN-15485) is the producer-side invariant meant to make the class unreachable from any emission site. It carried one exemption: if _SUPERSEDE_FILENAME_RE.search(filename): continue That asks "is this filename shaped like a correction?", never "is this path already merged?" -- and filename shape is not evidence of novelty. The merged path files every rebind at `<check_type>.supersede.<pr_number>.yaml`, keyed to the PRODUCT PR number, which is CONSTANT across every authoring pass for that same PR. So the 2nd, 3rd ... pass renders the identical paths and the guard waves each one through while they are already merged bytes on dev. It could not have known better. `ModelOccContractState` carried `existing_entry_ids` -- a set of DIRECTORIES -- and nothing about which FILES are merged, so the blanket exemption was the only way to let a legal first correction through. LIVE, from this ticket's own record: OCC#6616 (companion for omninode_infra#925) re-rendered eight `.supersede.925.yaml` files an earlier pass for that same PR#925 had merged, bumping `created_at` 07:19:22Z -> 17:14:25Z in place, alongside the base receipt. Reproductions #1-#4 in the ticket are the same shape through both producers. WHAT CHANGED * `ModelOccContractState.merged_receipt_paths` gives the pure planner file-level truth. Defaults to `()`, which reproduces the prior semantics byte-for-byte, so nothing in the merged corpus moves until the read-EFFECT populates it. * `assert_append_only_emissions` refuses any emission at a path in that set, BEFORE the directory rule and regardless of filename -- the same judgment `evaluate_append_only` makes off git status. The supersede exemption survives only for paths nothing has merged. * `_next_supersede_path` allocates a fresh generation keyed to the OCC companion PR authoring the rebind when the product-PR-keyed name is taken, and only for the keys that actually collide. The suffix stays a SINGLE integer because both this producer's `_SUPERSEDE_FILENAME_RE` and omnibase_core's `_SUPERSEDE_SUFFIX_RE` parse `.supersede.<NNNN>.yaml` strictly -- a compound `.supersede.925.6616.yaml` would be filed where nothing resolves it. The record's `replacement.pr_number` still names the PRODUCT PR, which is what `resolve_supersession`'s tier-1 selection keys on; the generation is provenance, never the binding. With no OCC PR number yet (pass 1) the emission is OMITTED and logged, not written -- the merged record already binds this same product PR with the same commit_sha / branch / probe, so suppressing the re-mint loses no evidence while opening it for write is the defect. * Born path (`OccCompanionEmitter`): the slot receipt's skip was keyed to `contract_path.is_file()` -- a question about `contracts/<ticket>.yaml` used to decide whether to open a receipt under `drift/dod_receipts/`. Two different files; its siblings in the same loop already ask the direct question. It now also reads the receipt path itself. The contract half is RETAINED deliberately: minting a slot receipt into a pre-existing contract that does not declare that item would trade an append-only violation for an orphan receipt. * The read-EFFECT lists the merged receipt files, scoped to `existing_entry_ids` (1-10 dirs in practice, not the ticket's whole tree). FAILS SOFT to `()`: a listing hiccup degrades to the shipped semantics rather than refusing a mint, and the write-EFFECT's pre-push `_assert_append_only` plus the hosted gate remain fail-closed behind it. PROOF. Two new gates, 13 cases, RED-proven against dev tip b7eef36. The control is stated precisely because a sloppier one would prove nothing: `merged_receipt_paths` is a NEW member of a `extra="forbid"` model, so all 10 compute legs against a plain `git checkout origin/dev -- src/` fail on `extra_forbidden` -- a schema error, which says nothing about the guard. Isolating control, measured: dev `src/` with ONLY `events/occ_companion.py` restored from this commit (the field, none of the logic) -> **6 failed / 4 passed**. The 6 fail behaviourally on the shipped producer: it re-emits the already-merged paths. The 4 controls stay green under that same control (empty `merged_receipt_paths` is a no-op, a first supersede is still legal, a plain receipt in a frozen dir is still refused), so the fix is not passing by refusing everything. The born-path leg needs no such isolation -- it drives `_emit_companion_sync` directly. Against a plain dev `src/`: **1 failed / 2 passed**, the failure being the writer rewriting a merged receipt's bytes, and the 2 passes being controls that hold both directions (contract + receipt present still skips per OMN-15785; a fresh ticket still mints, so the fix cannot pass by never writing and trading the violation for a MISSING_RECEIPT companion). The consumer leg writes both passes' real output to one tree and asks omnibase_core's `resolve_supersession` the question occ-preflight asks. All 24 cases in the three touched gate files pass on this commit. `test_occ_append_only_producer_omn_15485.py`'s falsifier docstring claimed "filename shape is what decides" for the whole invariant; it only ever exercised the directory rule. Corrected in place rather than left to be read at face value -- that reading is how the exemption went unexamined. Re-measured after the rebase onto b7eef36, not carried over: 746 passed / 30 skipped across node_occ_companion_compute + node_pr_lifecycle_fix_effect + node_occ_companion_effect + node_occ_state_effect with OCC_REPO_DIR set. 6178 passed / 38 skipped / 1 xfailed on the full unit suite. mypy --strict clean across 3077 source files, ruff clean, pre-commit run --all-files clean. KNOWN INERTNESS, stated rather than implied: node_occ_companion_compute and the born-path emitter run in the .201 dev-lane effects runtime, not product-repo CI. This code takes live effect at the next dev-lane refresh, so the live-corpus proof for this fix is ARMED, not yet observed. Closes OMN-16116 as a duplicate of this defect. * fix(OMN-16071): the supersede-skip log names which of its two causes fired CodeRabbit, correctly, on omnimarket#2200: `_next_supersede_path` returns `None` for two distinct reasons and the log asserted the first one unconditionally. * Pass 1 has no `occ_pr_number` to key a fresh generation with, and pass 2 of the same mint WILL file it. * A key whose product-PR-keyed AND OCC-PR-keyed generations are both already merged has nothing left to add, and no later pass will ever emit it. The operator responses differ. Claiming "no OCC PR number is available yet" in the second case sends a reader hunting for a number that is set, and promises a pass that will never come. RED-proven, not assumed: the new case fails against the shipped log text, and its captured output is the false claim itself -- skipping supersession for OMN-15922/dod-deploy-assessment - its product-PR-keyed path is already merged and no OCC PR number is available yet ... (emitted with occ_pr_number=6616 SET) Both directions are pinned: the exhausted-generations branch must say "ALREADY merged" / "no later pass will emit it" and must NOT say "no OCC PR number is available yet"; the pass-1 branch must say the inverse. A one-directional assertion would pass on a log that always said both. 747 passed / 30 skipped across the four affected node suites. ruff, mypy --strict (3077 files) clean.
Summary
FailureCategoryenum with 11 standardized failure category stringsFailureHistoryEntryandFailureHistorySummarydataclasses for cross-run trackingTrack A-resolvefor MERGEABLE+BLOCKED+GREEN PRs (unresolved review threads)is_merge_ready()to exclude BLOCKED PRs so they fall through to A-resolveTest plan
Summary by CodeRabbit