feat(OMN-16774): wire event-chain tests as a required per-PR CI gate

[jonahgabriel]

Problem

OMN-16767 killed the entire delegation chain on the .201 dev lane, and every repo's CI was green the whole time.

HandlerRoutingIntent was handed the projection arm's raw input_data dict instead of a validated ModelRoutingIntent, raised AttributeError: 'dict' object has no attribute 'payload' on its first dereference, the projection arm swallowed it, and every routing request went to the platform quarantine sink. Zero routing decisions were produced; every delegation timed out.

Nothing caught it because nothing drives the real dispatch path per PR:

Existing coverage	Why it was green during a total outage
Handler-isolation tests	Construct the handler and call handle() directly — they never see wiring.
tests/unit/nodes/node_coding_agent/test_real_dispatch_multitopic_routing.py	The closest "real dispatch" suite, but it imports _make_dispatch_callback directly — hand-building the callback and bypassing _prepare_handler_wiring, the exact arm-selection function the defect lived in.
omnimarket delegation-regression-nightly.yml	Drives a real chain, but is nightly and needs a live Redpanda + Postgres lane.
omnibase_core onex-run harness	A different path, already proven, and not where the defect was.

Doctrine rule 5: detection not wired as a pre-merge gate is advisory and gets ignored.

What this adds

tests/integration/chains/ — a parameterized suite that drives whole chains through the real seam with zero infrastructure:

raw JSON wire bytes
  -> EventBusInmemory                       (real, default local transport)
  -> EventBusSubcontractWiring callback     (real consumer)
  -> _deserialize_to_envelope               (real deserializer)
  -> _prepare_handler_wiring                (real ARM SELECTION  <-- the seam)
  -> MessageDispatchEngine.dispatch         (real engine)
  -> handler
  -> DispatchResultApplier.apply            (real terminal publish)
  -> terminal event on the bus

It asserts the two things the outage needed and nobody was checking:

1. the terminal event actually lands, and
2. nothing reaches the quarantine sink.

New chains are added as ChainCase rows. The only non-production seam is _import_handler_class, patched so the resolver returns the in-module handler rather than importing a package path.

Negative control (would_have_caught)

Run locally against this exact suite, toggling only handler_wiring.py:

handler_wiring.py	Result
dev @ e2110e1ae (pre-fix)	2 failed, 2 passed
+ the version from PR #2937 (fix)	4 passed

The failure reproduces the production log lines verbatim:

Projection handler error: handler=HandlerMirrorRoutingIntent
  topic=onex.cmd.omnibase-infra.delegation-routing-request.v1
  error_type=AttributeError error='dict' object has no attribute 'payload'

Projection handler HandlerMirrorRoutingIntent has NO DLQ topic declared in
  contract.event_bus.dlq_topics — routing malformed/erroring event to the
  platform quarantine sink onex.dlq.omnibase-infra.quarantine.v1

The no-db_io control row passes in both configurations, which localizes the failure to db_io arm selection rather than to the handler, the bus, or the engine.

Why the db_io row is xfail(strict=True)

The fix (#2937) has not merged, so that row is genuinely RED on dev today. Rather than block this gate behind another PR, the row carries broken_by="OMN-16767" → xfail(strict=True), the same "known drift tracked in an open ticket" pattern kafka-boundary-compat already uses in this file.

Consequences, both verified locally:

• Today: the gate lands green (2 passed, 2 xfailed) and actively blocks every other chain — rule 5 is satisfied now, not later.
• When fix(OMN-16767): db_io must not steal a typed def-B handler into the projection arm #2937 lands: the row XPASSes, and because the marker is strict CI goes RED, forcing the marker's deletion. It is self-retiring; it cannot rot into a permanent silencer.

broken_by is documented as usable only for a ticket whose fix is already in flight — never to silence a new failure, since a chain that starts failing is a dead chain, which is the whole point of the gate.

Gate wiring (same PR, per rule 5)

• ci.yml: new unconditional event-chain-gate job, display name Event Chain Gate. No needs:/if:, so it is always present.
• scripts/ci/ci_summary_gate.py: "Event Chain Gate" added to STRICT_GATE_JOBS.

Both halves matter. The default-deny sweep already fails CI Summary on any non-allowlisted job failure; the STRICT_GATE_JOBS entry additionally fails closed when the job is skipped or absent, so the gate cannot be silently retired by deleting it.

Required contexts — what blocks today

omnibase_infra's dev requires exactly one context:

$ gh api repos/OmniNode-ai/omnibase_infra/branches/dev/protection/required_status_checks --jq '.contexts'
["CI Summary"]

Because CI Summary is already required and its poller is default-deny over the run's jobs, this gate blocks merge the day it lands — no branch-protection edit is needed and none was made.

Constraints honored

• No Kafka, no Postgres, no service container in the job — in-memory transport only.
• No bypass flags; no --no-verify. Full pre-commit + the governed pre-push selector ran green (the selector escalated to the full suite because the diff touches .github/workflows/, cf. OMN-16745; the escalation was honored, not overridden).
• No branch-protection mutations.
• Does not duplicate fix(OMN-16767): db_io must not steal a typed def-B handler into the projection arm #2937's seam regression — that test asserts arm selection for one handler; this generalizes it to a whole-chain terminal + no-quarantine gate.

Ticket

OMN-16774 (AC1, AC3, AC4). Related: OMN-16767, OMN-16769, OMN-16773.

dod_evidence: negative-control matrix above (2 failed→4 passed on the fix toggle), strict-xfail flip verified in both directions, tests/ci/test_ci_summary_gate.py 81 passed, check_topic_literals.py clean, ruff + mypy --strict clean on the new suite.

Evidence-Ticket: OMN-16774
Evidence-Source: OCC#7324

[coderabbitai]

Warning

Review limit reached

Next included review available in 35 minutes.

View limit details

Limit details: You’ve used the included review currently available. Your 131 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2b6f900b-c771-4e5d-8dab-59874d70468a

📥 Commits

Reviewing files that changed from the base of the PR and between f0ab966 and 049cee2.

📒 Files selected for processing (4)

• .github/workflows/ci.yml
• scripts/ci/ci_summary_gate.py
• tests/integration/chains/__init__.py
• tests/integration/chains/test_event_chain_gate.py

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8468/OMN-8524) and do not block merge. Error: all review endpoints [192.168.86.201:8000 192.168.86.201:8000 ] unreachable — preflight short-circuit (no models available)

---

Gate semantics (pilot phase)

Verdict	Meaning	Blocks merge?
passed	No critical findings	No
blocked	CRITICAL findings found	Yes
degraded	All models unavailable (infra)	No (pilot)

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review (OMN-8468/OMN-8524)

[jonahgabriel]

Correction to this PR body (post-merge, lane chain-ci-gate-2)

The section "Why the db_io row is xfail(strict=True)" above describes an approach that was removed before merge and is not what landed. The body was not re-edited after the final commit 049cee2a7 ("drop the env-dependent xfail and make the db_io row non-vacuous"). Recording the correction rather than leaving the merged record wrong.

What actually landed on dev (verified against origin/dev @ 6efddd855):

• No row carries a non-empty broken_by — git show origin/dev:tests/integration/chains/test_event_chain_gate.py | grep -nE '^\s*broken_by=' returns nothing. The broken_by/xfail(strict=True) machinery still exists on ChainCase as an unused escape hatch, but zero rows use it.
• The db_io row is instead guarded by an explicit non-vacuity precondition (line ~424): a row declaring db_tables must find db_io.db_tables on the parsed contract or it fails loudly. The row can never go green while exercising nothing.

Why the xfail was dropped: #2937 merged first, so the row was no longer red; and a strict=True marker turns the resulting XPASS into a CI failure. Worse, the pass/fail split proved to be environment-dependent — the local uv-synced venv selected _make_projection_dispatch_callback and the chain died, while the CI shared env did not and the chain terminalized. An environment-dependent expectation is not a gate.

Negative control re-run against the LANDED suite

The matrix in the body above was measured against the pre-amend suite. Re-run in a throwaway worktree at origin/dev @ 6efddd855, toggling only src/omnibase_infra/runtime/auto_wiring/handler_wiring.py:

handler_wiring.py	Result
dev @ 6efddd855 (fix present)	4 passed
swapped to 16d61691e (pre-fix, parent of #2937's merge)	2 failed, 2 passed

Failing rows: test_chain_reaches_terminal_and_never_quarantines[delegation-routing-with-db-io] and test_typed_def_b_handler_is_never_wired_to_the_projection_arm[delegation-routing-with-db-io]. The delegation-routing-no-db-io control row passes in both configurations, localizing the failure to db_io arm selection rather than the handler, bus, or engine.

Production signature reproduced verbatim:

Projection handler error: handler=HandlerMirrorRoutingIntent
  topic=onex.cmd.omnibase-infra.delegation-routing-request.v1
  error_type=AttributeError error='dict' object has no attribute 'payload'
Projection handler HandlerMirrorRoutingIntent has NO DLQ topic declared in
  contract.event_bus.dlq_topics — routing malformed/erroring event to the
  platform quarantine sink onex.dlq.omnibase-infra.quarantine.v1

The conclusion in the body is unchanged; only the mechanism description was stale.

Gate confirmed live

Event Chain Gate ran completed/success on this PR's merge head (run 33110270490), is unconditional in dev's ci.yml, and is registered in scripts/ci/ci_summary_gate.py::STRICT_GATE_JOBS on dev (line 127). dev's only required context is ["CI Summary"], so it blocks today. No branch-protection edit was made.

Companion half: omnimarket#2167 merged cff0d765 (OCC companion #7325 merged first, after an add/add union on contracts/OMN-16774.yaml).