fix(OMN-16678): one deterministic event-bus resolution authority (explicit > ONEX_EVENT_BUS_TYPE > probe)

[jonahgabriel]

Summary

Fixes OMN-16678. Two related defects meant nothing decided the event-bus transport repeatably. Both were proven with measurements captured in OmniNode-ai/knowledge-base#59.

Defect 1 — the override was honoured by one path and ignored by the other.
backends/auto_configure.py::select_event_bus read ONEX_EVENT_BUS_TYPE; cli/cli_delegate.py::resolve_default_bus — the function onex delegate actually calls when --bus is omitted — never did. Setting ONEX_EVENT_BUS_TYPE=inmemory to pin onex delegate to the in-process bus did nothing, even though resolve_default_bus's own docstring claimed it "mirrors the SAME probe-then-select precedence."

Defect 2 — the probe tier was a coin flip.
probe_kafka Stage 2 wraps AdminClient.list_topics(timeout=2.0) in a bare except Exception that degrades any failure — including a plain metadata timeout against a healthy broker — to REACHABLE. resolve_default_bus mapped REACHABLE to inmemory; select_event_bus mapped the identical state to kafka. Measured over 20 consecutive resolve_default_bus() calls with an unchanged environment and a healthy broker:

Resolved bus	Count
kafka	14
inmemory	6

70/30 with zero input variation. A delegation that silently lands on inmemory never reaches the deployed node_delegate_skill_orchestrator consumer and never lands a projection in the shared delegation_events table — the OMN-14376 data-loss class, reintroduced through the indeterminate branch.

What changed

backends/auto_configure.py::resolve_bus_type is now the single authority both call sites route through. select_event_bus owns construction only; resolve_default_bus is a thin call into it.

Resolution order (one order, every caller)

1. Explicit argument — --bus at the CLI, the new bus_type= on select_event_bus. Never second-guessed, never probed against.
2. ONEX_EVENT_BUS_TYPE — inmemory / kafka / cloud, case- and whitespace-insensitive. Empty or unset falls through. cloud resolves to the Kafka transport because service_kernel.py already classifies it as broker-backed (_broker_required_types).
3. Live broker probe — mapped totally over EnumProbeState, no implicit default branch.

No new env-var surface: ONEX_EVENT_BUS_TYPE already existed, and its read stays in auto_configure.py, the file that already owned it. cli_delegate.py still reads no environment variable directly (check-env-reads).

Indeterminate-probe decision (recorded per the ticket's DoD)

Chosen behavior: fail fast on REACHABLE, raising EventBusResolutionAmbiguousError.

The probe tier is now total, and each state is classified by whether its verdict is conclusive:

Probe state	Outcome	Why
AUTHORITATIVE / HEALTHY	kafka	Determinate positive — the broker answered.
DISCOVERED	inmemory	Determinate negative. Reached only by branches that concluded something definite: no bootstrap configured, unparseable broker address, refused TCP connect. Repeatable, so it stays a fallback.
REACHABLE	raise	Indeterminate. TCP is up but the metadata call did not complete (timeout, auth failure, missing client library), so whether this broker will serve this caller is genuinely unknown.

Rationale, per feedback_no_defensive_no_defaults: when the underlying signal is not repeatable, refusing is the only outcome that is. Both prior mappings of REACHABLE were defensible in isolation and contradicted each other in practice, and which one you got was decided by transient network timing rather than by configuration. The error names the ambiguity (probe state + probe reason) and both deterministic remedies (--bus kafka|inmemory, or ONEX_EVENT_BUS_TYPE=kafka|inmemory), so an operator who wants the old "try Kafka anyway" behavior states it explicitly and gets it every time.

An unrecognised ONEX_EVENT_BUS_TYPE value likewise raises rather than degrading to "probe and hope" — a typo in an override must not read as "no override".

Behavior change to call out: select_event_bus previously built a Kafka bus on REACHABLE "despite the probe result". That branch is gone. The runtime kernel path reaches it only when no override is set; ONEX_EVENT_BUS_TYPE=kafka restores the old outcome deterministically.

Tests (TDD, probe always mocked — no live broker in the unit suite)

New tests/unit/backends/test_bus_resolution_order.py:

• each tier of the order, and that a higher tier does not probe at all (asserted on a recording stub, so a regression to probe-first cannot pass silently)
• cloud alias, case/whitespace normalisation, empty-string-is-unset, unrecognised value raises
• the total probe mapping, including the DISCOVERED-is-determinate distinction
• REACHABLE raises, the error names the ambiguity and both remedies, and the named remedy actually works
• repeatability: 20 consecutive calls on identical mocked input yield exactly one distinct answer — the direct counterpart to the measured 14/6 split

Updated:

• tests/unit/cli/test_cli_delegate.py — stub target moved to auto_configure.probe_kafka (cli_delegate no longer imports the probe, so patching the old name would be a silent no-op); added regressions for the override reaching the delegate path (probe stubbed to the opposite answer) and for REACHABLE refusing; autouse fixture clears ambient ONEX_EVENT_BUS_TYPE.
• tests/unit/runtime/test_kernel_registry.py — test_reachable_with_explicit_servers_uses_kafka inverted to ..._refuses_to_guess, with the contradiction it encoded documented in the docstring; added override-recovery and explicit-bus_type coverage.

Verification

• uv run pytest tests/unit — 23814 passed, 40 skipped, full unit suite green.
• uv run mypy src/omnibase_infra/backends/auto_configure.py src/omnibase_infra/cli/cli_delegate.py --strict — clean.
• pre-commit run --all-files — clean (exit 0), including check-env-reads.
• Governed impacted-test selector ran on pre-push; no escalation ordered.

Follow-up

The knowledge-base "combining deployment tiers" guide honestly documents the coin-flip third outcome today. A small KB PR updates that section to the shipped behavior once this merges — the guide stays true to what actually ships.

Evidence-Ticket: OMN-16678
Evidence-Source: OCC#7227

[coderabbitai]

Warning

Review limit reached

Next included review available in 1 minute.

View limit details

Limit details: You’ve used the included review currently available. Your 125 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e4104eab-68aa-4755-a086-5192a78f6c6e

📥 Commits

Reviewing files that changed from the base of the PR and between 800e11a and dfad019.

📒 Files selected for processing (5)

• src/omnibase_infra/backends/auto_configure.py
• src/omnibase_infra/cli/cli_delegate.py
• tests/unit/backends/test_bus_resolution_order.py
• tests/unit/cli/test_cli_delegate.py
• tests/unit/runtime/test_kernel_registry.py

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8468/OMN-8524) and do not block merge. Error: all review endpoints [192.168.86.201:8000 192.168.86.201:8000 ] unreachable — preflight short-circuit (no models available)

---

Gate semantics (pilot phase)

Verdict	Meaning	Blocks merge?
passed	No critical findings	No
blocked	CRITICAL findings found	Yes
degraded	All models unavailable (infra)	No (pilot)

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review (OMN-8468/OMN-8524)