fix(OMN-15395): RF policy must provision, not refuse — managed floor + fail-closed re-raise at every call site

[jonahgabriel]

Remediates the five adversarial-review findings against OMN-15395 as landed in
#2543 (squash-merged to dev as b2ca4faa). The first revision is already live on
dev, and finding 1 below is a live regression there: against a managed (MSK)
cluster the provisioner currently refuses to create every topic. This PR is the
fix, not a follow-up nicety.

Ticket: OMN-15395 — fix: managed-staging topic provisioner must be contract-driven and reject RF1

---

Finding-by-finding

1. HIGH · correctness/false-claim — refusing everything is not fail-closed, it is off

Verified claim, reproduced before changing anything, against the real
contracts_root (src/omnibase_infra/nodes) with ModelTopicProvisioningPolicy.managed():

total specs: 168
with declared RF: 0
RF distribution: {None: 168}
resolved: 0   refused: 168

The PR that landed deleted the only eleven replication_factor declarations in the
tree and added none, so on MSK the provisioner could create nothing at all.

I also measured why it could not simply be fixed by declaring RF everywhere.
Across all 62 contract files:

declaration shape	count
distinct provisioned topics	168
topics with no producing declaration anywhere in this repo	75
producing declaration sites that are bare strings (no place for topic_config)	95
producing sites already carrying a topic_config block	11

Those 75 appear only in event_bus.subscribe_topics — they are produced by
omniclaude / omnimarket / CLI relays. A literal refuse-on-undeclared policy makes
provisioning a permanent no-op for them with no in-repo contract that could ever
be fixed.

Fix. The managed profile now resolves an undeclared RF to
MANAGED_MINIMUM_REPLICATION_FACTOR (2) instead of refusing.

This is not the deleted defect reintroduced. The defect was a module-level constant of
1 that overrode the broker's own RF2 default, environment-blind, everywhere. The new
value is profile-scoped, equal to the MSK broker's own default, and is the same constant
the managed-staging namespace catalog already binds
(managed_staging_canary_catalog_namespace.yaml → default_replication_factor, wired to
this constant in model_canary_namespace.py). Acceptance criterion (c) names the
divergence between that catalog's RF2 and the manager's RF1 as the defect; converging
both on one constant is the fix. An RF1 topic is unreachable through this path.

What stays fail-closed: a declared RF below the floor still aborts the entire pass
with zero CreateTopics issued. That path is now batch-scoped, not per-topic — the
per-topic "refuse and carry on" branch (and the refused result key) is gone, along with
the §"Honest note on a scoping decision" reasoning that justified it on the false premise
that compliant sibling topics existed.

The eleven deleted declarations are restored at RF2, and the ratchet
(test_no_contract_declares_rf1.py) gained a second assertion: the declarations must
still exist. The previous revision checked only "none below the floor", so deleting
every declaration satisfied it — which is exactly what happened.

2. HIGH · correctness — undisclosed hard boot regression on the DLQ path

kernel_glue._provision_dlq_topics (kernel_glue.py:208) is the one caller with no
try/except, and derived DLQ names are absent from the contract-derived spec
registry. Verified: onex.dlq.omnibase-infra.example-event.v1 → IN SPEC REGISTRY: False,
and resolve_replication_factor(declared=None) raised. That propagated uncaught through
build_and_start_core_runtime (service_kernel.py:3268), so on MSK any DLQ topic not
already on the broker refused to start the S6 dispatch loop.

Resolved by finding 1's default and covered by two tests, one of which drives the real
_provision_dlq_topics helper rather than a surrogate.

3. MEDIUM · claim-vs-code — the fail-closed re-raise was never implemented

error_topic_provisioning.py:14-17 claimed the distinct class existed "so the
provisioning call sites can re-raise it past their best-effort except Exception
boundaries". It was re-raised only inside service_topic_manager itself. Implemented at
all four external call sites:

call site	shape
runtime/auto_wiring/handler_wiring.py:5107	per-contract boot interleave (the actual call shape)
runtime/service_kernel.py:1321	universe-warm boundary
runtime/service_kernel.py:1389	post-validation auto-create
runtime/runtime_host_process.py:3697	live-contract pre-provision

Plus a static guard (test_every_provisioning_call_site_reraises_the_policy_error)
that reads the shipped source and fails if any ensure_topic_exists /
ensure_provisioned_topics_exist call sits behind a bare except Exception without
re-raising first — so a new best-effort call site cannot silently reintroduce the
swallow. A docstring promise is not a mechanism.

Note on blast radius, stated rather than buried: _interleave_contract runs under
asyncio.gather(...) without return_exceptions=True, so this re-raise aborts the boot
subscribe pass. That is the intent for a contract declaring RF1 against MSK, and the
static ratchet in test_no_contract_declares_rf1.py prevents such a contract from
landing.

4. LOW · consistency — resolver output discarded at one creation site

service_topic_manager.py:755 called resolve_replication_factor(...) for its floor
side-effect and then passed config.replication_factor to NewTopic. It now passes the
resolver's return value, and records a ModelTopicSpec for the readiness path.
"There is one resolver" only holds if every creation site uses what it returns.

5. LOW · dead-surface — refresh_existing_topics

Deleted (zero callers, zero coverage). The snapshot lifecycle is now documented where the
cache is declared: re-fetched at the top of every ensure_provisioned_topics_exist pass,
fetched lazily once for ensure_topic_exists, folded forward on each create. The only
staleness a live instance can observe is an out-of-band deletion, whose cost is a skipped
create the next full pass repairs.

---

Additional defect found while remediating (not in the review)

Restoring the eleven RF2 contract declarations broke
tests/ci/test_topic_config_contract_parity.py, which asserts field-by-field parity
between each migrated topic's contract topic_config and its transitional
ModelTopicSpec entry in platform_topic_suffixes.py. Nine registry entries still said
replication_factor=None. Both sides now bind MANAGED_MINIMUM_REPLICATION_FACTOR, so
the seam cannot drift by editing one side.

This was caught only by the full unit suite on .200, not by the focused run — the
seam-matching rule earning its keep.

---

Seam definition — topic-spec fields end to end

The replication factor crosses five boundaries. Field-by-field:

boundary	carrier	field	undeclared representation
contract YAML	published_events[].topic_config	replication_factor: int	key absent
extractor	ModelContractTopicEntry	replication_factor: int | None	None
spec	ModelTopicSpec	replication_factor: int | None	None = contract declared nothing (never 1)
policy	ModelTopicProvisioningPolicy	minimum / default / capacity _replication_factor	resolved to default, or refused if the profile has none
broker	aiokafka.admin.NewTopic	replication_factor: int	never None — resolution precedes construction
readback	describe_topics → evaluate_topic_readiness	len(partition["replicas"])	None spec ⇒ no assertion

The same ModelTopicSpec object flows creation → _created_specs → readiness, so a
freshly created topic is confirmed against the spec it was created with. Pre-existing
topics are deliberately not spec-gated (the 519 legacy RF1 topics would flip healthy
topics to NOT-READY and block consumer attach); their drift is reported by
ensure_provisioned_topics_exist and repaired by the operator-gated WS-M lane.

New: self-hosted capacity ceiling

capacity_replication_factor is what makes a contract-declared RF2 landable at all. A
single-node broker (local Redpanda, CI sandboxes, the .201 lanes) rejects RF > broker count with INVALID_REPLICATION_FACTOR, so without a ceiling all eleven restored
declarations would fail CreateTopics on every non-managed lane. The ceiling only ever
reduces, and a model validator refuses a ceiling below the profile's durability floor —
otherwise a capacity reduction would be a bypass of the whole RF1 rejection, since
reduction happens before the floor check. Managed carries no ceiling.

---

Acceptance-criteria mapping

AC	Status	Where
(a) explicit + contract-driven RF, no module-level RF1 fallback	Met, with one recorded deviation	DEFAULT_EVENT_TOPIC_REPLICATION_FACTOR deleted; eleven contract declarations at RF2; single resolver. Deviation: an undeclared RF resolves to the managed floor rather than refusing — justified by the 75-orphan-topic readback above and documented in the model_topic_provisioning_policy module docstring, not hidden.
(b) RF1 in managed staging REJECTED fail-closed	Met	Batch-abort before any CreateTopics; re-raised past all four best-effort call sites; static guard keeps it that way. node_remote_agent_invoke_effect/contract.yaml fixed (RF1 → RF2), not exempted.
(c) spec passes through every creation/readiness path	Met	Creation, ensure_topic_exists (spec / bare-name / snapshot-config), readiness, canary catalog, managed_staging_topic_checker all resolve through the one policy; the snapshot-config site now uses the resolver's output (finding 4).
(d) list/diff before CreateTopics	Met (unchanged from #2543)	One describe_topics per pass; only missing topics created; existing-topic drift reported, never mutated.
(e) ONEX_BOOT_UNIVERSE_PROVISION stays 0	Met	Untouched except that the warm's error boundary now re-raises a durability violation.
(f) RED-before / GREEN-after on the real path	Met against two baselines	Against dev: RF1 created silently, RF defaulted to 1, ~1,280 blind authorizations. Against revision 1: 0/168 resolvable, DLQ boot refusal, four swallowing call sites, RF2 unprovisionable on single-broker. Enumerated per-test in the test module docstring.

---

Gates — run on .200 (stickybeatz-studio), patch-transfer verified

Per rule 11a. Edits were authored locally, transferred by git bundle, and verified
identical before any gate ran — a gate on an unedited copy is a vacuous green.

• Tree identity: .200 HEAD 746865732c45107f57ec876b717f035db195af20 == local HEAD.
• Per-file: shasum -a 256 on all 17 changed files, local == .200, byte-identical.
• uv run mypy src/omnibase_infra/ — Success, 2628 source files.
• uv run ruff format --check + ruff check src/ tests/ — clean.
• pre-commit — full hook set ran on commit (no --no-verify, no skip tokens). Two
  findings were fixed rather than suppressed: a bare MagicMock() on a transport surface
  (now spec=ProtocolEventBusLike / spec=ProtocolDispatchEngine) and a ruff import-sort
  fix.
• Full unit suite, no -k narrowing: 23575 passed, 46 skipped, 4 failed.

The 4 remaining failures are pre-existing full-suite env pollution, not this change

test_dependency_materializer::test_from_env,
test_kafka_bootstrap_no_localhost_fallback::test_from_env_uses_env_var_when_set, and two
test_model_kafka_producer_config_max_request_size::test_from_env*. Evidence they are not
mine:

• tests/unit/runtime/ alone at this HEAD: 5291 passed, zero failures.
• My new/changed test files run immediately before the four victims
  (tests/unit/event_bus/ tests/unit/topics/ + the three victim modules): 886 passed.
• They are from_env config tests with no relationship to topic provisioning; the
  *_default_when_unset case failing means an earlier test left the var set in
  os.environ.

Confirmed pre-existing by a dev-baseline run, executed in the same .200 worktree
and venv (detached checkout of b2ca4faa, then restored):

=== DEV BASELINE HEAD: b2ca4faa94c2b5700f288f462a4a4a0f12b77ab3 ===
FAILED tests/unit/runtime/test_dependency_materializer.py::TestModelMaterializerConfig::test_from_env
FAILED tests/unit/runtime/test_kafka_bootstrap_no_localhost_fallback.py::...::test_from_env_uses_env_var_when_set
FAILED tests/unit/runtime/test_model_kafka_producer_config_max_request_size.py::...::test_from_env_reads_kafka_max_request_size
FAILED tests/unit/runtime/test_model_kafka_producer_config_max_request_size.py::...::test_from_env_default_when_unset
4 failed, 23562 passed, 46 skipped, 3879 deselected

The same four, at dev, without this PR. Per the "second failure of the same check is a
bug, not a flake" rule this is not being written off as transient — it is a real
cross-directory os.environ pollution bug, filed as OMN-15432 rather than folded in
here, because fixing it means finding and repairing an unrelated test's teardown.

Also filed from this lane: OMN-15433 — two files on dev carry a 2026 SPDX year
against a validator pinned to 2025, making pre-commit run --all-files unconditionally
red on a clean checkout. Neither file is in this diff.

proof_class

receipt-bound (code + executed tests). Live MSK readback belongs to the separate,
operator-gated WS-M repair legs, per the ticket's own scope note. No AWS mutation here.

Evidence-Source: OCC#5511
Evidence-Ticket: OMN-15395

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 10 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6c0a44c2-f962-4e98-9d42-7368ca3144ec

📥 Commits

Reviewing files that changed from the base of the PR and between b2ca4fa and 7468657.

📒 Files selected for processing (17)

• src/omnibase_infra/event_bus/service_topic_manager.py
• src/omnibase_infra/nodes/node_llm_inference_effect/contract.yaml
• src/omnibase_infra/nodes/node_registration_orchestrator/contract.yaml
• src/omnibase_infra/nodes/node_remote_agent_invoke_effect/contract.yaml
• src/omnibase_infra/nodes/node_row_count_diagnostic_effect/contract.yaml
• src/omnibase_infra/nodes/node_runner_fleet_health_compute/contract.yaml
• src/omnibase_infra/nodes/node_runner_fleet_maintain_orchestrator/contract.yaml
• src/omnibase_infra/nodes/node_runner_health_snapshot_effect/contract.yaml
• src/omnibase_infra/runtime/auto_wiring/handler_wiring.py
• src/omnibase_infra/runtime/runtime_host_process.py
• src/omnibase_infra/runtime/service_kernel.py
• src/omnibase_infra/topics/model_topic_provisioning_policy.py
• src/omnibase_infra/topics/platform_topic_suffixes.py
• tests/unit/event_bus/test_service_topic_manager.py
• tests/unit/event_bus/test_topic_provisioner_rf_policy_omn15395.py
• tests/unit/topics/test_no_contract_declares_rf1.py
• tests/unit/topics/test_topic_provisioning_policy.py

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8468/OMN-8524) and do not block merge. Error: all review endpoints [192.168.86.201:8000 192.168.86.201:8001 ] unreachable — preflight short-circuit (no models available)

---

Gate semantics (pilot phase)

Verdict	Meaning	Blocks merge?
passed	No critical findings	No
blocked	CRITICAL findings found	Yes
degraded	All models unavailable (infra)	No (pilot)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8468/OMN-8524)