Skip to content

feat(merge-sweep): add failure history tracking and Track A-resolve [OMN-7573] - #1

Merged
jonahgabriel merged 1 commit into
mainfrom
jonah/merge-sweep-failure-tracking
Apr 5, 2026
Merged

jonahgabriel merged 1 commit into
mainfrom
jonah/merge-sweep-failure-tracking

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Apr 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add FailureCategory enum with 11 standardized failure category strings
  • Add FailureHistoryEntry and FailureHistorySummary dataclasses for cross-run tracking
  • Add Track A-resolve for MERGEABLE+BLOCKED+GREEN PRs (unresolved review threads)
  • Fix is_merge_ready() to exclude BLOCKED PRs so they fall through to A-resolve
  • Add escalation logic: CHRONIC (≥5 consecutive failures) and RECIDIVIST (≥3 polishes still failing) PRs skip polish
  • Bump contract version to 1.1.0
  • Add 6 new tests covering failure history, escalation, and Track A-resolve

Test plan

  • All 14 unit tests pass locally
  • CI passes

Summary by CodeRabbit

  • New Features
    • Added a new thread-resolution queue track for managing PRs requiring resolved review threads.
    • Introduced cross-run failure history tracking with escalation thresholds (STUCK, CHRONIC, RECIDIVIST).
    • Enhanced PR failure categorization to identify root causes (branch staleness, unresolved threads, conflicts, CI failures).
    • Implemented escalation-based skip logic to prevent repeated polish cycles on chronic failures.

…OMN-7573]

Add FailureCategory enum, FailureHistoryEntry/Summary dataclasses, and
cross-run failure escalation logic to NodeMergeSweep. CHRONIC (>=5 consecutive
failures) and RECIDIVIST (>=3 polishes still failing) PRs skip polish automatically.
Add Track A-resolve for MERGEABLE+BLOCKED+GREEN PRs (thread resolution). Fix
is_merge_ready to exclude BLOCKED PRs so they fall through to A-resolve. Bump
contract version to 1.1.0. Add 6 new tests covering failure history escalation.
@linear

linear Bot commented Apr 5, 2026

Copy link
Copy Markdown

OMN-7573

@coderabbitai

coderabbitai Bot commented Apr 5, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

The changes introduce cross-run failure tracking and a new PR classification track (A-resolve) for thread resolution. The contract version incremented to 1.1.0, failure history models were added, and MergeSweepRequest/Result extended to support failure categorization and escalation logic based on historical failure patterns.

Changes

Cohort / File(s) Summary
Contract Definition
src/omnimarket/nodes/node_merge_sweep/contract.yaml
Version bump from 1.0.0 to 1.1.0 for both contract_version and node_version. Description expanded to include thread-resolution queue (Track A-resolve) and failure tracking with escalation thresholds (STUCK, CHRONIC, RECIDIVIST).
Handler Implementation
src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py
Added new A_RESOLVE track for PRs blocked but mergeable with passing checks. Introduced failure history models (FailureCategory enum, FailureHistoryEntry, FailureHistorySummary). Extended MergeSweepRequest with failure_history dict and run_id. Extended MergeSweepResult with failure_history_summary and track_a_resolve property. Implemented _needs_thread_resolution(), updated _classify_pr() to track per-PR failure categories, added _should_skip_polish() escalation logic, and _compute_failure_summary() for cross-run history aggregation.
Test Coverage
tests/test_golden_chain_merge_sweep.py
Expanded golden-chain assertions for PR failure-category attribution across Track A and B outcomes. Added new TestFailureHistory suite validating failure-history input, escalation logic (chronic/recidivist/stuck patterns triggering skip decisions), and failure_history_summary computation. Verified behavior with omitted/empty failure history.

Sequence Diagram

sequenceDiagram
    participant Req as MergeSweepRequest
    participant Handler as Handler
    participant Classify as Classifier
    participant History as FailureHistory
    participant Result as MergeSweepResult

    Req->>Handler: failure_history, run_id, PRs
    Handler->>Classify: _classify_pr() with history context
    Classify->>History: check failure_categories
    History-->>Classify: STUCK, CHRONIC, RECIDIVIST status
    Classify-->>Handler: (track, reason, failure_categories)
    
    alt Track A-resolve
        Handler->>Handler: _needs_thread_resolution()
        Handler->>Handler: mark A_RESOLVE track
    else Track B_POLISH
        Handler->>Handler: _should_skip_polish() via history
        Handler->>Handler: escalate to SKIP if threshold met
    else Track A
        Handler->>Handler: mergeable, proceed
    end
    
    Handler->>Handler: _compute_failure_summary()
    Handler->>Result: failure_history_summary
    Result-->>Req: MergeSweepResult
Loading

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 munching carrot thoughtfully

A-resolve hops into sight,
With failure trails tracked through the night,
Threads untangled, stuck patterns named,
My sweep grows wise, cross-run tamed! 🧵✨


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py (1)

84-92: cleaned_merged is never populated.

FailureHistorySummary exposes this field, but _compute_failure_summary() has no current-run input and always returns the default 0. Either derive it from classified + history, or leave it out of the 1.1.0 response until it is actually wired up.

Also applies to: 307-329

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py` around
lines 84 - 92, FailureHistorySummary exposes cleaned_merged but it is never
populated; update _compute_failure_summary to compute cleaned_merged (or remove
it) — locate the FailureHistorySummary dataclass and the
_compute_failure_summary function and either: 1) derive cleaned_merged from the
current-run classified results combined with history (e.g., count items in
classified where label/state indicates a cleaned+merged outcome and/or where
history shows previously cleaned then merged), or 2) remove cleaned_merged from
FailureHistorySummary and any response construction until you wire it up; ensure
updates reference the FailureHistorySummary class and the
_compute_failure_summary method and adjust any callers that expect
cleaned_merged accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py`:
- Around line 219-240: The current ordering calls self._needs_thread_resolution
before self._needs_polish so PRs with review_decision == "CHANGES_REQUESTED" are
routed to PRTrack.A_RESOLVE instead of PRTrack.B_POLISH; fix by either moving
the polish block (the if self._needs_polish(pr, require_approval): ... that
returns PRTrack.B_POLISH) above the thread-resolution block (the if
self._needs_thread_resolution(pr): ... that returns PRTrack.A_RESOLVE) or adjust
self._needs_thread_resolution to explicitly ignore PRs where pr.review_decision
== "CHANGES_REQUESTED" so that CHANGES_REQUESTED PRs are evaluated by
self._needs_polish and routed to PRTrack.B_POLISH.

---

Nitpick comments:
In `@src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py`:
- Around line 84-92: FailureHistorySummary exposes cleaned_merged but it is
never populated; update _compute_failure_summary to compute cleaned_merged (or
remove it) — locate the FailureHistorySummary dataclass and the
_compute_failure_summary function and either: 1) derive cleaned_merged from the
current-run classified results combined with history (e.g., count items in
classified where label/state indicates a cleaned+merged outcome and/or where
history shows previously cleaned then merged), or 2) remove cleaned_merged from
FailureHistorySummary and any response construction until you wire it up; ensure
updates reference the FailureHistorySummary class and the
_compute_failure_summary method and adjust any callers that expect
cleaned_merged accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Lite

Run ID: 3d0d63a3-9ce5-48a9-87d4-04528eea5851

📥 Commits

Reviewing files that changed from the base of the PR and between c74a41d and 5d82d90.

📒 Files selected for processing (3)
  • src/omnimarket/nodes/node_merge_sweep/contract.yaml
  • src/omnimarket/nodes/node_merge_sweep/handlers/handler_merge_sweep.py
  • tests/test_golden_chain_merge_sweep.py

@jonahgabriel
jonahgabriel merged commit 060a7d8 into main Apr 5, 2026
1 check passed
jonahgabriel added a commit that referenced this pull request Apr 20, 2026
…orts

Adds a new `pr_checks_live` check dimension to HandlerOverseerVerifier that
shells out to `gh pr checks <num> --repo <owner>/<name> --json bucket,state,
conclusion,name,completedAt` for each PR an agent self-reports as green.
Previously the verifier ran five pure-Python checks and rubber-stamped PR
completion claims without ever contacting GitHub — the #1 leverage failure
mode in overnight dispatch ("agents lie").

Changes:
- ModelClaimedPr(pr_number, repo) model for (PR, repo) claim tuples
- ModelVerifierRequest gains optional `claimed_prs: list[ModelClaimedPr]`
- _check_pr_checks_live: per-PR subprocess shell-out with 30s timeout, per-PR
  error isolation, `bucket=="pass"` + conclusion whitelist gating; any red
  check surfaces PR + failing check name in the failure reason
- Follows canonical shell-out pattern from node_pr_snapshot_effect (subprocess
  .run, capture_output=True, text=True, isolated try/except for timeout / OSError
  / JSONDecodeError)
- `pr_checks_live` inserted at priority slot 4 (above allowed_action_scope):
  a broken PR is stronger evidence of agent lying than a schema quibble
- Skipped with a note when `claimed_prs` is empty, preserving zero-I/O
  semantics for non-PR verifications

Tests (6 new + 19 existing overseer tests pass):
- pr_checks_live skipped when no PRs claimed (subprocess never invoked)
- two green PRs → verdict PASS, pr_checks_live passed
- one green + one red → verdict FAIL, failing PR + check name in message
- subprocess timeout → verdict FAIL, descriptive timeout message
- gh non-zero exit → verdict FAIL with exit code + stderr
- malformed JSON → verdict FAIL with parse-error message
- existing consumer golden-chain test updated for 6-dim expectation

`uv run ruff format/check` clean, `uv run mypy src/ --strict` clean (878 files).
github-merge-queue Bot pushed a commit that referenced this pull request Apr 20, 2026
…orts (#354)

* OMN-9273: wire overseer-verify to gh pr checks against agent self-reports

Adds a new `pr_checks_live` check dimension to HandlerOverseerVerifier that
shells out to `gh pr checks <num> --repo <owner>/<name> --json bucket,state,
conclusion,name,completedAt` for each PR an agent self-reports as green.
Previously the verifier ran five pure-Python checks and rubber-stamped PR
completion claims without ever contacting GitHub — the #1 leverage failure
mode in overnight dispatch ("agents lie").

Changes:
- ModelClaimedPr(pr_number, repo) model for (PR, repo) claim tuples
- ModelVerifierRequest gains optional `claimed_prs: list[ModelClaimedPr]`
- _check_pr_checks_live: per-PR subprocess shell-out with 30s timeout, per-PR
  error isolation, `bucket=="pass"` + conclusion whitelist gating; any red
  check surfaces PR + failing check name in the failure reason
- Follows canonical shell-out pattern from node_pr_snapshot_effect (subprocess
  .run, capture_output=True, text=True, isolated try/except for timeout / OSError
  / JSONDecodeError)
- `pr_checks_live` inserted at priority slot 4 (above allowed_action_scope):
  a broken PR is stronger evidence of agent lying than a schema quibble
- Skipped with a note when `claimed_prs` is empty, preserving zero-I/O
  semantics for non-PR verifications

Tests (6 new + 19 existing overseer tests pass):
- pr_checks_live skipped when no PRs claimed (subprocess never invoked)
- two green PRs → verdict PASS, pr_checks_live passed
- one green + one red → verdict FAIL, failing PR + check name in message
- subprocess timeout → verdict FAIL, descriptive timeout message
- gh non-zero exit → verdict FAIL with exit code + stderr
- malformed JSON → verdict FAIL with parse-error message
- existing consumer golden-chain test updated for 6-dim expectation

`uv run ruff format/check` clean, `uv run mypy src/ --strict` clean (878 files).

* OMN-9273: address CodeRabbit review — cap fan-out, guard JSON shape

Two mechanical hardening fixes raised by CodeRabbit:

1. **Fan-out cap** — Add `_MAX_CLAIMED_PRS = 20` constant. `_check_pr_checks_live`
   fails fast with a descriptive message when `claimed_prs` exceeds the cap,
   preventing pathological N-PR invocations from stalling verification up to
   N×30s. 20 keeps the worst-case at ~10min, well above any realistic
   agent self-report size.

2. **JSON shape guard** — Previously `raw: list[dict[str, Any]] = json.loads(...)`
   trusted gh's output shape. A valid non-list JSON (e.g. top-level object) would
   iterate over dict keys (strings) and `AttributeError` on `.get()`, breaking
   per-PR error isolation and crashing the verifier. Now explicitly assert
   top-level list + per-item dict, returning a `JSON shape error` message
   matching the existing parse-error envelope shape.

Tests added (3):
- too many claimed PRs → fail fast, subprocess.run never invoked
- non-list JSON (top-level object) → JSON shape error, no crash
- list with non-dict items → JSON shape error, no crash

Full suite: 2228 passed (3 new, 2225 prior). ruff + mypy --strict clean.

* OMN-9273: coerce check name to str to prevent TypeError in sort/join

CodeRabbit flagged that red_checks sourced from untyped JSON could contain
None or int values for `name`, causing TypeError when sort/join runs. Coerce
via `str(item.get('name') or '<unnamed>')`. New test covers None + int + str
mixed rows, asserts sorted/joined output contains each coerced value.
jonahgabriel added a commit that referenced this pull request Apr 30, 2026
CR Major #1 (durable_evidence_gate.py:~141-159):
extract_cited_merge_commits accepted any string sha and let
ModelCitedMergeCommit (min_length=7) raise ValidationError on
malformed short SHAs — contract violation vs the docstring's
"malformed citations are skipped" promise. Added _SHA_RE guard
([0-9a-fA-F]{7,40}) before parse_pr_url; malformed SHAs are now
silently skipped. Regression test:
test_extract_skips_malformed_short_sha.

CR Major #2 (same file, dedupe + cross-set comparison):
Citation dedupe and CONTRACT_ON_OCC_MAIN comparison were keyed on
raw pr_url, so /pull/123 and /pull/123/files for the same PR were
treated as distinct citations — caused double gh_pr_view() calls
AND false CONTRACT_ON_OCC_MAIN hard-fails when local and OCC main
spelled the same PR differently. Switched both keys to the
normalized identity (repo, pr_number, sha). Regression tests:
test_url_variants_dedupe_to_same_citation and
test_url_variants_across_local_and_main_do_not_hard_fail.

deploy-gate root cause:
Action matcher src/*/nodes/**/*.py flagged the pure-logic gate
service as a runtime path. Added dod-005 to contracts/OMN-10407.yaml
documenting the deploy posture honestly: this slice is pure logic
+ Protocol probes, no container/topic touched, and concrete probe
wiring + FSM integration ship in a follow-up ticket carrying its
own deploy evidence. check_value contains "deploy" so the gate
recognizes intent. NO skip token used.

Verification:
- 70/70 node_dod_verify unit tests green (was 67, +3 new).
- Full uv run pytest tests/ -m "not kafka and not integration":
  2786 passed / 0 failed (was 2783).
- uv run ruff format/check, uv run mypy --strict on
  node_dod_verify/, all clean.
- OMNIBASE_INFRA_PATH=... pre-commit run --all-files: all hooks
  pass.
- Local replay of validate_pr_deploy_required.py against the
  three changed files now returns DEPLOY GATE PASSED.

CR fingerprints (CodeRabbit auto-resolves on resolution):
phantom:medusa:ocelot:97a9ca79-253e-49f4-bb67-27581b204d27
4e71b3a2
andywu42 pushed a commit to andywu42/omnimarket that referenced this pull request May 12, 2026
…mniNode-ai#467)

* feat(OMN-10407): durable-evidence gate before Linear Done transition

Adds a pure-logic DurableEvidenceGate service in node_dod_verify that
refuses Linear Done when the durable evidence trail is local-only.

OMN-9855 incident shape (2026-04-30):
- Worker probed implementation already on omnibase_core/main (PR OmniNode-ai#949)
- Generated DoD receipt LOCALLY at evidence/OMN-9855/dod_report.json,
  never committed
- Updated Linear dod_evidence text to point at PR OmniNode-ai#949
- onex_change_control/main still had stale contract citing superseded OmniNode-ai#926
- Linear flipped Done; durable trail unverifiable from origin alone

The gate enforces three checks before any Linear Done transition:

1. RECEIPT_TRACKED — evidence/<TICKET>/dod_report.json must be tracked
   on onex_change_control/main (not just present on disk).
2. CONTRACT_CITES_MERGE_COMMIT — every cited pr_url has gh-state MERGED
   and mergeCommit.oid matches the cited commit_sha. Closed/superseded
   PRs HARD FAIL.
3. CONTRACT_ON_OCC_MAIN — the contract version on OCC main contains the
   real merge-commit citation. Stale main HARD FAIL with explicit "open
   an OCC PR first" remediation hint.

Failure raises DurableEvidenceGateError carrying the structured
ModelDurableEvidenceGateResult; the first failed check supplies the
remediation hint.

Architecture:
- Pure-logic gate + Protocol-typed probes (GitTrackedProbe,
  GhPrViewProbe, ContractOnRefLoader). Tests inject deterministic stubs.
- New schema module model_durable_evidence_gate.py is pure (no I/O,
  no env, no time) per onex_change_control D-008 schema purity rules.
- Production probe wiring (subprocess-backed git ls-tree / gh pr view /
  yaml load on git ref) is intentionally out of scope for this slice;
  follow-up ticket will wire the gate into the FSM phase ordering.

DoD evidence:
- 16 unit tests cover untracked receipt, superseded PR, wrong SHA,
  stale main, missing-on-main, structured error raise, citation
  extraction (top-level + nested + dedup + malformed), URL parser.
- All 2783 non-kafka non-integration omnimarket tests pass.
- mypy --strict clean on src/omnimarket/nodes/node_dod_verify/.
- ruff format + check clean; pre-commit hooks all green.

* fix(OMN-10407): address CR findings + deploy-gate (PR OmniNode-ai#467)

CR Major OmniNode-ai#1 (durable_evidence_gate.py:~141-159):
extract_cited_merge_commits accepted any string sha and let
ModelCitedMergeCommit (min_length=7) raise ValidationError on
malformed short SHAs — contract violation vs the docstring's
"malformed citations are skipped" promise. Added _SHA_RE guard
([0-9a-fA-F]{7,40}) before parse_pr_url; malformed SHAs are now
silently skipped. Regression test:
test_extract_skips_malformed_short_sha.

CR Major OmniNode-ai#2 (same file, dedupe + cross-set comparison):
Citation dedupe and CONTRACT_ON_OCC_MAIN comparison were keyed on
raw pr_url, so OmniNode-ai/pull/123 and OmniNode-ai/pull/123/files for the same PR were
treated as distinct citations — caused double gh_pr_view() calls
AND false CONTRACT_ON_OCC_MAIN hard-fails when local and OCC main
spelled the same PR differently. Switched both keys to the
normalized identity (repo, pr_number, sha). Regression tests:
test_url_variants_dedupe_to_same_citation and
test_url_variants_across_local_and_main_do_not_hard_fail.

deploy-gate root cause:
Action matcher src/*/nodes/**/*.py flagged the pure-logic gate
service as a runtime path. Added dod-005 to contracts/OMN-10407.yaml
documenting the deploy posture honestly: this slice is pure logic
+ Protocol probes, no container/topic touched, and concrete probe
wiring + FSM integration ship in a follow-up ticket carrying its
own deploy evidence. check_value contains "deploy" so the gate
recognizes intent. NO skip token used.

Verification:
- 70/70 node_dod_verify unit tests green (was 67, +3 new).
- Full uv run pytest tests/ -m "not kafka and not integration":
  2786 passed / 0 failed (was 2783).
- uv run ruff format/check, uv run mypy --strict on
  node_dod_verify/, all clean.
- OMNIBASE_INFRA_PATH=... pre-commit run --all-files: all hooks
  pass.
- Local replay of validate_pr_deploy_required.py against the
  three changed files now returns DEPLOY GATE PASSED.

CR fingerprints (CodeRabbit auto-resolves on resolution):
phantom:medusa:ocelot:97a9ca79-253e-49f4-bb67-27581b204d27
4e71b3a2
jonahgabriel added a commit that referenced this pull request Jul 7, 2026
… ask; persist rejected candidate content (#1630)

* fix(OMN-14004): code_generation quality gate assumes Python for every ask; persist rejected candidate content

Two coupled defects found by the ratchet-canary (OMN-14000):

1. `_check_compiles_without_errors` unconditionally ran `ast.parse` on the
   candidate for ANY `code_generation` ask, regardless of what artifact
   language was actually requested. A YAML contract-fragment ask was
   false-rejected with "does not compile as Python: invalid syntax", then
   escalated through cheap_cloud -> cheap_frontier -> claude, burning cloud
   rate-limit budget for nothing. Fix: a fenced block's own language tag
   (```yaml``/```json``) now selects the parser for THAT block; an untagged
   fence or raw (non-fenced) content keeps the original Python-only behavior
   byte-for-byte unchanged, so the common Python case is unaffected.

2. `LocalDelegationDispatchPort`'s escalation loop logged only the gate's
   failure reasons on a quality-gate FAIL, never the rejected candidate's own
   text — so a false-reject (like #1) couldn't be debugged after the fact;
   the terminal payload's cumulative `content` field only ever carried the
   LAST attempt across an escalation chain. Fix: log the full rejected
   candidate content at the escalation point. The capture-file this logger
   writes to is already promoted to a content-addressed artifact by the CLI
   receipt layer (`receipt_mode.py`), so this makes the rejected text durable
   evidence without adding a new persistence surface.

Verified live: `onex delegate "<YAML ask>" --task-type code_generation` was
reproduced failing on origin/dev with `MALFORMED: response does not compile
as Python: invalid syntax`; new unit tests lock in both fixes (8 new tests,
all pass) plus regression guards for the untouched Python/raw-content paths.
Full local gates green: ruff format/check clean, mypy --strict clean (2693
files), full pytest (no -k): 12104 passed, 5 failed + 1 error — identical
pre-existing infra-dependent failures (Postgres/Kafka unavailable locally),
same count/names as an unmodified origin/dev baseline run.

Blocks scaling OMN-14000 (ratchet burn-down) past the `document` task-type
workaround.

* fix(OMN-14004): accept fence info strings in quality gate

* style(OMN-14004): format quality gate tests
jonahgabriel added a commit that referenced this pull request Jul 7, 2026
…ct schema (#1631)

* fix(OMN-14000): migrate 6 nodes' handler_routing to the nested contract schema

Ratchet burn-down batch #1 (missing_handler_routing, the ratchet-canary's #1
target). The arch_handler_contract_compliance gate
(onex_change_control/src/onex_change_control/scanners/handler_contract_compliance.py)
only recognizes the nested `handler_routing.handlers[].handler.{name,module}`
schema; these 6 nodes were still on the legacy flat `handler_class`/
`handler_module` string schema and false-flagged as missing_handler_routing.

Adds the nested `handler:` block (and `routing_strategy: operation_match`
where absent) as a purely additive sibling to each existing handlers[] entry
— every existing key/value/comment is preserved unchanged, nothing removed.

Nodes: node_adr_canary_orchestrator, node_adr_document_ingestion_effect,
node_kb_adr_publisher, node_kreuzberg_parse_effect,
node_memory_lifecycle_orchestrator (3 handler entries),
node_persona_lifecycle_orchestrator (2 handler entries).

4 of these 6 fixes (kb_adr_publisher, kreuzberg_parse_effect,
memory_lifecycle_orchestrator, persona_lifecycle_orchestrator) were generated
by the LOCAL delegate tier (Qwen3.6-35B-A3B via `.201`, `onex delegate
--task-type document`) — 4/4 first-try, gate-verified, no cloud escalation.
This is the flagship local-first proof for the ratchet burn-down lane
(OMN-14000): mechanical, self-grading-oracle tech debt reduced by a local
model, not a frontier one.

Gate-verified: `python -m
onex_change_control.validators.arch_handler_contract_compliance --repo-root
omnimarket` — Compliant 336 -> 344 (+8 handler files), New violations 95 -> 87
(-8), Total handlers unchanged at 431. No new violations introduced.

OMN-14000

* revert(OMN-14000): drop 4 nodes coupled to a separate state-coverage-baseline gate

state-coverage-gate (scripts/validate_state_coverage.py, OMN-13781) grandfathers
pre-existing uncovered declared-event-topic debt via
scripts/validation/state_coverage_baseline.txt, but revokes the grandfather
clause in --strict mode for any node DIRECTLY TOUCHED by a diff. All 4 of
node_adr_canary_orchestrator, node_adr_document_ingestion_effect,
node_memory_lifecycle_orchestrator, and node_persona_lifecycle_orchestrator
are pre-listed in that baseline with genuinely uncovered output topics
(unrelated to handler_routing) — touching their contract.yaml for the
missing_handler_routing fix un-grandfathers that separate debt and fails CI.

Fixing that properly means adding real test coverage for those declared
topics — out of scope for a small, revertible ratchet-type batch (touches a
DIFFERENT ratchet: declared-state test coverage, not handler_routing). Revert
these 4 back to the legacy flat handler_routing schema for this PR; keep only
node_kb_adr_publisher and node_kreuzberg_parse_effect, which have no
state-coverage-baseline coupling and gate-verify clean on their own.

Filed OMN-14009 to track the 4 deferred nodes + the general coupling risk for
future missing_handler_routing batches.
jonahgabriel added a commit that referenced this pull request Jul 7, 2026
…fix (local-first, offline) (#1626)

* feat(OMN-13998): 2x2 SWE-discriminator harness + truncation-confound fix

Promotes the OMN-13988 de-risk spike into a durable, tested harness and fixes
the capability-vs-plumbing confound (blocker #2). Local-first, offline over
captured artifacts (proof_class = offline; NOT a live runtime loop).

New package omnimarket.delegation.swe_discriminator (extends graded_ladder):
- 2x2 arm runner (monolith/decomposed x frontier/cost_routed), decomposer +
  per-slice worker + concat integration, balanced-bracket slice parser
- offline blind deterministic floor grader (verifier != runner)
- classify.py: detect_truncation + classify_run separate PASS/FAIL_WRONG
  (capability) from TRUNCATED/BLOCKED (plumbing/infra, excluded) so a token-cap
  truncation cannot deflate the hard-tier numbers (OMN-13335 hazard)
- model_client: finish_reason capture, reasoning-aware budget (SWE_MAX_TOKENS,
  default 16384), 429 retry/backoff, env-overridable frontier; no hardcoded IPs
- run_smoke: 2-phase RUN/GRADE orchestrator, pass^k repeats + PassKCell aggregate

Evidence (docs/evidence/2026-07-05-swe-discriminator-smoke/):
- grader calibration (selftest): real merged solutions PASS floor, pre-fix FAIL
- 13 unit tests incl truncation detector proven on the EXACT live L3 repro
- live pass^3 on L1 local arms (3/3); genuine decomposition (parsed slice + tax)
- canonical 4-arm run: L1 local pass, L3 local correctly TRUNCATED, frontier BLOCKED

Full 2x2 headline run remains gated on frontier reachability (blocker #1) and a
larger hard-task budget/reasoning control (follow-ups). mypy --strict + ruff clean.

* fix(OMN-13998): resolve tier endpoints from ladder_rungs config (url-authority gate)

Removes the literal frontier URL const and the SWE_FRONTIER_URL env read that
tripped the url-authority gate (2 violations). Endpoints now resolve from the
committed graded_ladder ladder_rungs.yaml routing config (the same integration
catalog the ladder recorder uses); rung selection is by id via SWE_FRONTIER_RUNG
/ SWE_COST_RUNG, keys/endpoints named by the rung, never a hardcoded URL.
Verified: validator_url_authority reports 0 swe_discriminator violations; mypy
--strict + ruff clean; 13 unit tests + grader selftest green; resolve_tier
resolves frontier->glm-5.2 and cost_routed->local from config.

* fix(OMN-13998): pass SWE discriminator runtime config explicitly

* fix(OMN-13998): harden SWE smoke artifacts and parsing

* ci(OMN-13998): refresh stuck checks

* ci(OMN-13998): harden reviewer uv sync

* ci(OMN-13998): extend hostile reviewer timeout

* ci(OMN-13998): retrigger stuck hostile reviewer

* fix(OMN-13998): handle malformed chat choices

* test(OMN-13998): match urlopen timeout keyword

* ci(OMN-13998): refresh cancelled check set

* ci(OMN-13998): retrigger stale queued validators

* ci(OMN-13998): refresh cancelled hostile reviewer

* test(OMN-13998): accept ROI overlay in delegation ladder shim
jonahgabriel added a commit that referenced this pull request Jul 7, 2026
…on-consumer honor path (ROI tier-flip enrichment) (#1636)

* feat(OMN-14018): per-run backend pin on context-ROI runner → generation consumer honors it

The context-ROI battery only ever routed to the contract-declared local tier, so
context_roi_scores held a single tier (488 local rows, 100% success). To capture
genuine per-tier graded outcomes (a real sub-floor cheap_cloud cohort for the
demonstrable escalation-skip flip, OMN-14011 Path 1) a run must be pinnable to a
chosen backend.

Runner side:
- ModelContextRoiRunRequest.target_endpoint (optional, default empty).
- HandlerContextRoiRunner threads it onto every generation command as
  forced_endpoint_ref (only when non-empty; empty preserves the command shape).
- run_context_roi_battery.py --target-endpoint flag → target_endpoint.

Generation-consumer side (so the pin is REAL, not an inert field):
- ModelNodeGenerationRequest.forced_endpoint_ref (optional, default empty).
- handle() pins the STARTING route to the forced backend's tier via
  _forced_starting_route, which resolves the concrete model+endpoint through the
  SAME routing authority (delta with min_tier_name) the escalation path already
  uses. Fail-safe: an unresolvable ref keeps the contract-declared route (never
  crashes). Empty ref = byte-identical to prior behavior.

Tests (5 new, all through the real routing authority — no mocked routing):
- runner threads/omits forced_endpoint_ref by target_endpoint presence;
- forced cloud pin makes generation attempt #1 post to the CLOUD tier's
  model+endpoint (vs contract-local); empty preserves local; bad ref falls back
  to local without crashing. 510 pass across gen-consumer + runner + scripts.
- ruff/mypy --strict clean; check-no-faked-boundary + leaked-literals gates pass.

NOTE (honest scope): this lands the enrichment MECHANISM. The live sub-floor
capture (≥5 real cheap_cloud rows) + observed escalation-skip A/B is gated on
OMN-14001/#1629 merged AND the stability-lane runtime redeployed with this code +
OMNIDASH_ANALYTICS_DB_URL set (the ticket's own Blocked-by). No synthetic rows
were authored.

Refs OMN-14018, OMN-14011.

* test(OMN-14018): assert runner terminal output topics by literal (state-coverage gate)

Touching node_context_roi_runner activates the contract-state-coverage gate for
it; the two declared terminal output states were asserted only by substring, so
the gate saw them uncovered. Assert the full literal topic names
(onex.evt.omnimarket.context-roi-run-{completed,failed}.v1) in the two emit tests
-- a stronger assertion (exact topic, not substring) that covers both states.

* fix(OMN-14018): forced route records BACKEND id as endpoint_class (overlay-countable capture)

_forced_starting_route left endpoint_ref="" (mirroring escalated routes), so the
attempt + run-level benchmark recorded the bare TIER NAME (e.g. 'cheap_cloud') as
endpoint_class. The context_roi_scores row's endpoint_ref = that endpoint_class,
and the ROI overlay maps a row to a tier via tier_for_backend(endpoint_ref) which
resolves a BACKEND id ('cloud-glm' -> cheap_cloud) but returns None for a bare
tier name -> every captured per-tier row would be silently DROPPED, yielding zero
usable sub-floor rows for the OMN-14011 Path-1 flip.

Fix: carry the forced backend id on endpoint_ref. Routing is unaffected —
_call_llm posts to endpoint_url when authority_resolved is True (endpoint_ref is
attribution-only there). Tests now assert the attempt AND run-level benchmark
record the backend id as endpoint_class.

Refs OMN-14018, OMN-14011.

* ci(OMN-14018): refresh hostile reviewer context

* test(OMN-14018): accept ROI overlay in delegation ladder shim
jonahgabriel added a commit that referenced this pull request Jul 15, 2026
…schema (#1768)

node_projection_baselines defined local models sharing almost no field
names with the real producer contract (omnibase_infra's
ModelBaselinesSnapshotEvent / ModelBaselinesComparisonRow /
ModelBaselinesTrendRow / ModelBaselinesBreakdownRow): a required-no-default
pattern_id the producer never sends, entirely fictional trend/breakdown
concepts on the consumer side, and a snapshot write that bound columns
(patterns_compared/patterns_recommended) that don't exist on
baselines_snapshots.

Live-checked on .201 stability-test before fixing: the projection_baselines
consumer group had committed offset 2 with zero lag (both real events on
the topic consumed) yet all four target tables held zero rows -- confirming
failure mode #1 from the ticket (every real event silently crashed inside
the runtime's catch-all dispatch boundary, never a partial degrade).

Fix (both consumer paths declared in this node's contract, matching the
OMN-14490/14506 precedent):
- HandlerProjectionBaselines (the live-wired RuntimeLocal shim) and
  BaselinesProjectionRunner (the standalone Kafka->Postgres runner, still
  declared in docker-compose.projection.yml) both now validate the
  canonical omnibase_infra producer models directly instead of local slim
  copies, stripping transport-injected `_`-prefixed keys first.
- New migration 0002 recreates baselines_comparisons/baselines_trend/
  baselines_breakdown with the real producer row shapes (treatment_*/
  control_*/roi_pct, cohort/session_count, pattern_id/pattern_label). All
  three tables held zero rows in every known environment, so there was no
  data to preserve across the shape change.
- Wired the OMN-13548 DLQ path: BaselinesProjectionRunner declares
  poison_dlq_topics/publish_dlq (the base class's own POISON safety net
  routes an escaped ValidationError there); HandlerProjectionBaselines
  documents that a malformed payload propagates to the wiring layer's own
  dlq_topics route, matching the established RuntimeLocal-shim precedent.

Two sibling nodes (node_projection_baselines_roi,
node_projection_baselines_quality) imported this node's now-deleted
fictional models. Gave each its own local, behavior-identical copy to avoid
a compile break -- NOT a fix, tracked separately as OMN-14630 since their
own aggregation semantics (token_delta, retry_delta, confidence tiers) have
no producer analog and need their own redesign.

Behavioral proof: tests/test_omn14513_baselines_seam.py drives the
producer's real envelope through the real unwrap_envelope decode seam into
BOTH consumer paths. Verified RED against the pre-fix models on this branch
(pydantic ValidationError: comparisons.0.pattern_id Field required; runner
silently degraded to a blank/'shadow'-keyed default row) and GREEN against
the canonical ones.

Closes OMN-14513
jonahgabriel added a commit that referenced this pull request Aug 1, 2026
…rect its false provenance (#1999)

* fix(OMN-15597): re-pin the AC5 census to the ticket's SHA, correct its provenance

Merged #1998 (3cb2633) pinned the AC5 corpus at onex_change_control@9089ffa9
and asserted -- in the corpus module docstring and in
TestAc5PinnedCorpusReCensus::test_pinned_corpus_is_the_census_the_ticket_names
-- that its 59 values / 33 contracts were the ticket's census "reproduced
exactly". That is false. It was a cardinality coincidence over a different
contract set: OMN-15597's census point is d7b16271 (its own stated 32,593 scan
total is exactly that SHA), and contracts/OMN-15239.yaml and
contracts/OMN-15488.yaml do not exist at 9089ffa9 at all -- so the pinned
corpus structurally could not contain OMN-15488, the live motivating case AC5
exists to protect. A green assertion encoding a false provenance claim is the
same defect class #1998 was opened to close.

Re-pinned to d7b16271 and regenerated with the oracle already committed in this
repo (vendored pre-fix guard rejects AND current guard accepts), via the
fixture's own documented git-archive procedure -- read-only, AC6 respected:

  OCC_CENSUS_SHA                        9089ffa9 -> d7b16271
  OCC_CENSUS_TOTAL_COMMAND_CHECK_VALUES    32564 -> 32593
  class / contracts                        59/33 -> 63/35

The new class is a strict superset of the old 59 (extras: OMN-15239.yaml#0,
OMN-15488.yaml#0/#1/#2). The control at 9089ffa9 reproduces the outgoing
snapshot exactly, so the divergence is the SHA, not the method.

Provenance is now stated rather than implied: the docstring and the renamed
test_pinned_corpus_is_the_ticket_census_point_under_the_committed_oracle say
which oracle at which SHA, and record that the ticket's literal 59/33 was read
with a different oracle (its unbalanced-$( rule, which yields 56/32 here) and
is superseded, not reproduced. New
test_pinned_corpus_covers_every_contract_the_ticket_enumerates makes the
provenance claim falsifiable by containment of the ticket's 33 enumerated
contract names -- something a cardinality coincidence cannot satisfy.

Also corrects the docstring's "one day of OCC commits later" for 1e6b75f8: it
is 76 minutes, same day.

Findings by the #1998 adversarial verifier, independently re-measured and
confirmed by lane G3 (Linear comment 921b6c7b on OMN-15597).

Evidence-Ticket: OMN-15597

* chore(OMN-15597): mint a fresh head so ci.yml re-runs with OCC#5845 merged

The first head ran ci.yml before the OCC companion existed, so
occ-preflight / eligibility failed, OCC Preflight Dependency refused to start
heavy CI, and CI Summary fail-closed -- every test-bearing job was skipped and
this diff had zero Linux-runner proof. OCC#5845 merged 2026-08-01T13:24:08Z
(born-admissible: app/onexbot-occ-writer, both commits authored AND committed
by occ-companion-effect, single-parent, zero human commits) and
occ-preflight / eligibility now passes on the existing head. No diff change.

Evidence-Ticket: OMN-15597
@jonahgabriel
jonahgabriel deleted the jonah/merge-sweep-failure-tracking branch August 25, 2026 18:14
jonahgabriel added a commit that referenced this pull request Aug 29, 2026
…filename shape (#2200)

* fix(OMN-16071): the OCC companion writer is add-only by path, not by filename shape

Defect 1, the defect this ticket is titled after, is the last open half of
OMN-16071 (Defect 2 landed in #2081/#2086). Its AC is literal: "the autobind
path must be strictly add-only -- never open an existing receipt file for
write." Every prior PR in this ticket's chain hardened the GUARD; none touched
the WRITER, which is why the ticket carries a standing DO-NOT-FLIP.

WHY THE EXISTING MECHANISM DID NOT CATCH IT. `assert_append_only_emissions`
(OMN-15485) is the producer-side invariant meant to make the class unreachable
from any emission site. It carried one exemption:

    if _SUPERSEDE_FILENAME_RE.search(filename):
        continue

That asks "is this filename shaped like a correction?", never "is this path
already merged?" -- and filename shape is not evidence of novelty. The merged
path files every rebind at `<check_type>.supersede.<pr_number>.yaml`, keyed to
the PRODUCT PR number, which is CONSTANT across every authoring pass for that
same PR. So the 2nd, 3rd ... pass renders the identical paths and the guard
waves each one through while they are already merged bytes on dev.

It could not have known better. `ModelOccContractState` carried
`existing_entry_ids` -- a set of DIRECTORIES -- and nothing about which FILES
are merged, so the blanket exemption was the only way to let a legal first
correction through.

LIVE, from this ticket's own record: OCC#6616 (companion for
omninode_infra#925) re-rendered eight `.supersede.925.yaml` files an earlier
pass for that same PR#925 had merged, bumping `created_at` 07:19:22Z ->
17:14:25Z in place, alongside the base receipt. Reproductions #1-#4 in the
ticket are the same shape through both producers.

WHAT CHANGED

* `ModelOccContractState.merged_receipt_paths` gives the pure planner
  file-level truth. Defaults to `()`, which reproduces the prior semantics
  byte-for-byte, so nothing in the merged corpus moves until the read-EFFECT
  populates it.
* `assert_append_only_emissions` refuses any emission at a path in that set,
  BEFORE the directory rule and regardless of filename -- the same judgment
  `evaluate_append_only` makes off git status. The supersede exemption survives
  only for paths nothing has merged.
* `_next_supersede_path` allocates a fresh generation keyed to the OCC
  companion PR authoring the rebind when the product-PR-keyed name is taken,
  and only for the keys that actually collide. The suffix stays a SINGLE
  integer because both this producer's `_SUPERSEDE_FILENAME_RE` and
  omnibase_core's `_SUPERSEDE_SUFFIX_RE` parse `.supersede.<NNNN>.yaml`
  strictly -- a compound `.supersede.925.6616.yaml` would be filed where
  nothing resolves it. The record's `replacement.pr_number` still names the
  PRODUCT PR, which is what `resolve_supersession`'s tier-1 selection keys on;
  the generation is provenance, never the binding. With no OCC PR number yet
  (pass 1) the emission is OMITTED and logged, not written -- the merged record
  already binds this same product PR with the same commit_sha / branch / probe,
  so suppressing the re-mint loses no evidence while opening it for write is
  the defect.
* Born path (`OccCompanionEmitter`): the slot receipt's skip was keyed to
  `contract_path.is_file()` -- a question about `contracts/<ticket>.yaml` used
  to decide whether to open a receipt under `drift/dod_receipts/`. Two
  different files; its siblings in the same loop already ask the direct
  question. It now also reads the receipt path itself. The contract half is
  RETAINED deliberately: minting a slot receipt into a pre-existing contract
  that does not declare that item would trade an append-only violation for an
  orphan receipt.
* The read-EFFECT lists the merged receipt files, scoped to `existing_entry_ids`
  (1-10 dirs in practice, not the ticket's whole tree). FAILS SOFT to `()`:
  a listing hiccup degrades to the shipped semantics rather than refusing a
  mint, and the write-EFFECT's pre-push `_assert_append_only` plus the hosted
  gate remain fail-closed behind it.

PROOF. Two new gates, 13 cases, RED-proven against dev tip b7eef36. The control
is stated precisely because a sloppier one would prove nothing: `merged_receipt_paths`
is a NEW member of a `extra="forbid"` model, so all 10 compute legs against a plain
`git checkout origin/dev -- src/` fail on `extra_forbidden` -- a schema error, which
says nothing about the guard. Isolating control, measured: dev `src/` with ONLY
`events/occ_companion.py` restored from this commit (the field, none of the logic)
-> **6 failed / 4 passed**. The 6 fail behaviourally on the shipped producer: it
re-emits the already-merged paths. The 4 controls stay green under that same
control (empty `merged_receipt_paths` is a no-op, a first supersede is still legal,
a plain receipt in a frozen dir is still refused), so the fix is not passing by
refusing everything.

The born-path leg needs no such isolation -- it drives `_emit_companion_sync`
directly. Against a plain dev `src/`: **1 failed / 2 passed**, the failure being
the writer rewriting a merged receipt's bytes, and the 2 passes being controls
that hold both directions (contract + receipt present still skips per OMN-15785;
a fresh ticket still mints, so the fix cannot pass by never writing and trading
the violation for a MISSING_RECEIPT companion). The consumer leg writes both
passes' real output to one tree and asks omnibase_core's `resolve_supersession`
the question occ-preflight asks. All 24 cases in the three touched gate files pass
on this commit.

`test_occ_append_only_producer_omn_15485.py`'s falsifier docstring claimed
"filename shape is what decides" for the whole invariant; it only ever
exercised the directory rule. Corrected in place rather than left to be read at
face value -- that reading is how the exemption went unexamined.

Re-measured after the rebase onto b7eef36, not carried over: 746 passed /
30 skipped across node_occ_companion_compute + node_pr_lifecycle_fix_effect +
node_occ_companion_effect + node_occ_state_effect with OCC_REPO_DIR set.
6178 passed / 38 skipped / 1 xfailed on the full unit suite. mypy --strict clean
across 3077 source files, ruff clean, pre-commit run --all-files clean.

KNOWN INERTNESS, stated rather than implied: node_occ_companion_compute and the
born-path emitter run in the .201 dev-lane effects runtime, not product-repo
CI. This code takes live effect at the next dev-lane refresh, so the live-corpus
proof for this fix is ARMED, not yet observed.

Closes OMN-16116 as a duplicate of this defect.

* fix(OMN-16071): the supersede-skip log names which of its two causes fired

CodeRabbit, correctly, on omnimarket#2200: `_next_supersede_path` returns
`None` for two distinct reasons and the log asserted the first one
unconditionally.

* Pass 1 has no `occ_pr_number` to key a fresh generation with, and pass 2 of
  the same mint WILL file it.
* A key whose product-PR-keyed AND OCC-PR-keyed generations are both already
  merged has nothing left to add, and no later pass will ever emit it.

The operator responses differ. Claiming "no OCC PR number is available yet" in
the second case sends a reader hunting for a number that is set, and promises a
pass that will never come.

RED-proven, not assumed: the new case fails against the shipped log text, and
its captured output is the false claim itself --

  skipping supersession for OMN-15922/dod-deploy-assessment - its
  product-PR-keyed path is already merged and no OCC PR number is available yet
  ... (emitted with occ_pr_number=6616 SET)

Both directions are pinned: the exhausted-generations branch must say "ALREADY
merged" / "no later pass will emit it" and must NOT say "no OCC PR number is
available yet"; the pass-1 branch must say the inverse. A one-directional
assertion would pass on a log that always said both.

747 passed / 30 skipped across the four affected node suites. ruff, mypy
--strict (3077 files) clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant