[jonahgabriel]

Summary

Fixes runtime-effects health on the dev lane following the abandoned dispatch incident (OMN-19355). Previously, an abandoned dispatch kept the event bus in a DEGRADED state for its entire lifetime. Because the container healthcheck uses --degraded-policy fail, this caused the container to remain unhealthy until it was replaced.

Design Decision

The abandon path intentionally avoids cancellation. A to_thread worker cannot be killed, and cancelling the awaiter would release the projection gate slot that the thread still occupies.

Chosen Shape:

• The orphan remains tracked (gate-slot accounting is unchanged and it still counts toward consumer_dispatch_orphan_limit).
• The orphan stops counting toward DEGRADED status after a bounded grace period.
• The orphan limit still flips the bus to UNHEALTHY regardless of orphan age.

Changes

• New Config Field: consumer_dispatch_orphan_degraded_grace_seconds

  • Added to ModelKafkaEventBusConfig next to consumer_dispatch_orphan_limit.
  • Typed pydantic field.
  • Default: 300.0.
  • Constraints: gt 0, le 86400.
  • No env fallback.
  • Grace runs from the time of abandonment (OrphanedDispatch.abandoned_at).

• Status Model: ModelDispatchDeadlineStatus

  • Gains degrading_orphans.
  • Status is unhealthy if orphaned_dispatches >= orphan_limit.
  • Status is degraded if degrading_orphans > 0.
  • Status is healthy otherwise.

Tests

• New File: tests/unit/event_bus/test_omn20464_orphan_degraded_grace.py (3 tests)
  1. Old orphan stays tracked but bus reports healthy.
  2. Limit still flips unhealthy after grace.
  3. Field typed with default 300.0.
• Regression: Existing test_omn19355_dispatch_deadline.py and test_runtime_health_dispatch_deadline_dimension_omn19355.py still pass.
• Total: 19 passed across the three files.
• Linting: ruff and mypy clean on touched modules.
• AC3 (Gate-slot accounting unchanged): No change to the abandon/cancel path; the orphan task is not cancelled and remains in _orphaned_dispatches until it settles. Asserted by the test (orphaned_dispatches stays 1 after grace).

Lab

Lab: host=h201 lane=headless remote lane command="uv run pytest tests/unit/event_bus/test_omn20464_orphan_degraded_grace.py tests/unit/event_bus/test_omn19355_dispatch_deadline.py tests/unit/services/test_runtime_health_dispatch_deadline_dimension_omn19355.py -q" head=250d3bb533 (branch OMN-20464-stuck-dispatch-degraded-grace) observed=19 passed

AC4 Status

NOT done. No dev-lane runtime-effects readback was performed. The docker shim on this host refuses mutating compose operations on the declared dev lane project, and no sanctioned runtime warm-redeploy path was available to this headless lane (the hotpatch-pr skill patches the plugin cache, not the runtime). Stopped instead of improvising. AC4 remains open.

Evidence-Ticket: OMN-20464
Evidence-Source: OCC#12687

[github-actions]

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 4
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 4 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

• [CRITICAL] src/omnibase_infra/event_bus/event_bus_kafka.py (gpt-oss-review) — New required field 'abandoned_at' breaks existing OrphanedDispatch constructions | The OrphanedDispatch dataclass now requires the 'abandoned_at' attribute. Only the code path in _await_dispatch_withi
• [MAJOR] src/omnibase_infra/models/health/model_dispatch_deadline_status.py (gpt-oss-review) — ModelDispatchDeadlineStatus validation fails for callers not supplying 'degrading_orphans' | The ModelDispatchDeadlineStatus schema now requires the 'degrading_orphans' field. Existing code that build
• [MINOR] src/omnibase_infra/event_bus/event_bus_kafka.py (gpt-oss-review) — Linear scan to compute degrading orphan count on each status request | dispatch_deadline_status computes 'degrading' by iterating over all orphaned dispatches each call, resulting in O(n) work where n
• [MINOR] src/omnibase_infra/event_bus/event_bus_kafka.py (gpt-oss-review) — Potential exposure of internal loop time via health model | The 'abandoned_at' timestamp is derived from loop.time() and later exposed indirectly through ModelDispatchDeadlineStatus fields, which may

[github-actions]

✅ Hostile Reviewer — REVIEWED

Critical findings: 3
Major findings: 2
Total findings: 11
Models succeeded: qwen3-review,gpt-oss-review
Models unavailable: none

Action required: findings were posted as review threads. Address or reject each one, then resolve its thread — the Hostile Review Thread Gate fails while hostile-reviewer threads are unresolved (OMN-17492).

---

Semantics (OMN-17492 — the model finds, thread resolution gates)

Surface	Meaning	Blocks merge?
Review threads	Per-finding, posted by the reviewer	No (informational)
Hostile Review Thread Gate	Deterministic: unresolved hostile-reviewer threads exist	Fails until resolved (not yet a required context)
degraded verdict	Fewer than 2 models succeeded (infra)	Fails this job with a named reason

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review (Qwen3.8-27B), gpt-oss-review (gpt-oss-120b) (OMN-8468/OMN-8524/OMN-17492)

[github-actions]

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 6
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 5
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 5 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

• [CRITICAL] src/omnibase_infra/models/health/model_dispatch_deadline_status.py (gpt-oss-review) — ModelDispatchDeadlineStatus new required field breaks existing callers | The Pydantic model ModelDispatchDeadlineStatus now requires the field degrading_orphans without a default. Code elsewhere t
• [MAJOR] src/omnibase_infra/event_bus/event_bus_kafka.py (qwen3-review) — Race condition in degrading_orphans calculation | The degrading_orphans count is computed by iterating over orphans and checking now - orphan.abandoned_at < grace. However, orphans is a snapsh
• [MAJOR] tests/unit/event_bus/test_omn20464_orphan_degraded_grace.py (qwen3-review) — Missing test for edge case where grace is zero or very small | The configuration allows consumer_dispatch_orphan_degraded_grace_seconds to be any value greater than 0.0. The tests use `GRACE_SECONDS
• [MINOR] src/omnibase_infra/services/service_runtime_health_monitor.py (qwen3-review) — Documentation inconsistency in health monitor docstring | The docstring for evaluate_dispatch_deadline in service_runtime_health_monitor.py is updated to mention the grace window, but the descript
• [MINOR] src/omnibase_infra/event_bus/event_bus_kafka.py (gpt-oss-review) — Inconsistent time sources for grace calculation | abandoned_at is recorded using loop.time() while the grace comparison uses time.monotonic(). Although both are monotonic, they are not guarantee

Findings demoted from threads (anchor rejected)

• [CRITICAL] hostile-reviewer (qwen3-review)

  Time source mismatch between abandoned_at and dispatch_deadline_status | The abandoned_at field is populated with loop.time() in _await_dispatch_within_deadline, but dispatch_deadline_status computes now using time.monotonic() inside a try/except RuntimeError block. loop.time() and time.monotonic() are not guaranteed to share the same epoch or tick rate. If the event loop's clock is not backed by time.monotonic() (e.g., in certain embedded or custom loop implementations), the subtraction

  Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

• [CRITICAL] hostile-reviewer (gpt-oss-review)

  OrphanedDispatch missing default for new field | The dataclass OrphanedDispatch now includes a required field abandoned_at but only the construction in _await_dispatch_within_deadline supplies a value. Any other code path that instantiates OrphanedDispatch without providing abandoned_at will raise a TypeError at runtime, breaking dispatch tracking. | Evidence: Added field abandoned_at: float to OrphanedDispatch and only set it in _await_dispatch_within_deadline via abandoned_at=loop.time(). No d

  Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

• [MINOR] hostile-reviewer (qwen3-review)

  Integration test relies on real-time sleep, which is flaky | The integration test test_bus_health_recovers_after_grace_while_orphan_stays_tracked uses await asyncio.sleep(GRACE_SECONDS + 0.2) to wait for the grace period to elapse. This is a real-time sleep, which can be flaky in CI environments where the system is under load. The test should use a mock or fake clock to control time, or use a condition variable to wait for the health status to change, rather than relying on a fixed sleep duration. | Evi

  Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

• [MINOR] hostile-reviewer (qwen3-review)

  O(n) scan of orphans on every health check | The dispatch_deadline_status method now performs an O(n) scan of all orphans to compute degrading_orphans. If the number of orphans is large (up to orphan_limit, which is at most 16), this is not a significant performance concern. However, if the health check is called frequently (e.g., every second by a health monitor), the repeated O(n) scan could add up. Consider maintaining a counter of degrading orphans that is updated when orphans are added or when th

  Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

• [MINOR] hostile-reviewer (gpt-oss-review)

  Linear scan for degrading orphans on each health check | dispatch_deadline_status computes degrading by iterating over the entire _orphaned_dispatches collection on every call. With a large number of orphans this introduces O(n) work for each health check, potentially increasing latency of health endpoints. | Evidence: degrading = sum(1 for orphan in orphans if now - orphan.abandoned_at < grace) performed each invocation. | Fix: Maintain a separate counter that increments when an orphan enters the g

  Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

• [MINOR] hostile-reviewer (gpt-oss-review)

  Flaky integration test due to real-time sleep | The new integration test test_bus_health_recovers_after_grace_while_orphan_stays_tracked relies on asyncio.sleep(GRACE_SECONDS + 0.2). Variability in CI environments can cause the grace window to be missed, leading to nondeterministic failures. | Evidence: Test uses actual sleep to wait for grace expiration instead of mocking time. | Fix: Replace real sleeps with a mockable time source or use freezegun/asyncio clock manipulation to deterministically ad

  Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).