Skip to content

fix(OMN-20359): C16 probe scans until window end or terminal found - #4464

Merged
jonahgabriel merged 1 commit into
devfrom
OMN-20359-c16-probe-race
Oct 2, 2026
Merged

jonahgabriel merged 1 commit into
devfrom
OMN-20359-c16-probe-race

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

OMN-20359: C16 Receipt Identity Probe Terminal Scan and Correlation ID

Summary

This change modifies scripts/ci/c16_receipt_identity_probe.py to fix a race condition in terminal scanning and adds healthy_correlation_id to the C16 record.

  1. Terminal Scan Logic: _scan_terminal now keeps reading until the window ends or this run's own correlation-id terminal is found. Catching up to the end offsets captured at scan start is no longer a stop condition.
    • At window end, if every partition read to the end, the terminal is established as absent (empty TerminalObservation, grades FAIL as before).
    • Otherwise, the unobserved error is raised (grades SKIP, still red).
    • The bar is unchanged: a truly missing terminal still fails.
    • grade_identity, grade_skip_never_pass, and grade_typed_death are untouched.
  2. Record Field: The C16 record now carries healthy_correlation_id (the healthy run's correlation id; neither a secret nor a tenant value). This is threaded from _submit through observe_live, Observations, and grade into record_dict. Old replay files without the key read as an empty string. RECORD_VERSION stays 1 (every sibling probe keeps 1; the field is additive).
  3. Documentation: Module docstring and the comment above the scan loop were updated. The workflow header is unchanged as its text does not state the old scan behaviour.

Red first

The following output was generated on the pre-fix source with the new tests.

Command:

uv run pytest tests/ci/test_c16_receipt_identity_probe_omn19181.py -q -k "late_terminal or absent_terminal or correlation_id"

Output:

E   AttributeError: 'Observations' object has no attribute 'healthy_correlation_id'
E   AttributeError: 'Observations' object has no attribute 'healthy_correlation_id'
E   assert 0.0004249290213920176 >= 0.3
E   AssertionError: assert None == {'correlation_id': 'cid-1', 'provider': 'local'}
E    +  where None = TerminalObservation(payload=None, error=None).payload
FAILED tests/ci/test_c16_receipt_identity_probe_omn19181.py::test_the_healthy_correlation_id_survives_live_observation_and_recording
FAILED tests/ci/test_c16_receipt_identity_probe_omn19181.py::test_replay_healthy_correlation_id_is_optional_and_must_be_a_string
FAILED tests/ci/test_c16_receipt_identity_probe_omn19181.py::test_absent_terminal_is_established_only_after_the_window_ends
FAILED tests/ci/test_c16_receipt_identity_probe_omn19181.py::test_a_late_terminal_after_the_watermark_is_found_inside_the_window
4 failed, 1 passed, 32 deselected in 0.20s

Lab proof

  • Workflow: chain-canary-c16-receipt-identity.yml dispatched once against branch OMN-20359-c16-probe-race at head 8b758fefefdcd55053969120bdc8fd26a2c4a753 (dev lane).
  • Run Details:
    • Run id: 37004933472
    • Conclusion: success
    • Record as_of: 2026-10-02T12:08:54Z
    • Verdict: pass
    • Grades: R-DELEG-11 PASS, R-DELEG-12 PASS, R-DELEG-26 PASS
    • Record version: 1
    • healthy_correlation_id: ba67a0b2-ddbd-40cf-ad46-307c3ab5113f
  • Coverage Note: A late terminal cannot be forced on the live lane, so the live run proves the record field and the unchanged green path; the red-first unit test proves the late case.

Verification

  • Focused Tests:
    • Command: uv run pytest tests/ci/test_c16_receipt_identity_probe_omn19181.py -q
    • Result: 37 passed
    • Environment: Lab host h202 at 8b758fefefdcd55053969120bdc8fd26a2c4a753, exit 0.
  • Linting and Type Checking:
    • ruff check and mypy --strict on the two changed files: clean.
    • pre-commit run --all-files: clean (shellcheck installed on h202 via uv tool to satisfy the shell-hygiene hook).

Notes

  • Evidence for the defect: From the ledger friction row 2026-10-02T11:27:16Z (not re-verified here): run 37000178114 graded R-DELEG-12 FAIL while a completed terminal sat on the terminal topic after the probe's scan ended; rerun 37000848440 was green. The correlation-id match is not established by that record.
  • Tests Updated: tests/ci/test_c16_receipt_identity_probe_omn19181.py includes updates for late_terminal, absent_terminal, and correlation_id (two tests). The early-return test was rewritten to the new contract (absent terminal still grades FAIL, and only after the window ends).
  • Ticket: OMN-20359

Evidence-Ticket: OMN-20359
Evidence-Source: OCC#12328

_scan_terminal now reads until window end or this run's correlation-id terminal,
instead of stopping when partitions reach start-captured end offsets.
At window end: all partitions read to end => terminal absent (FAIL); otherwise
unobserved error (SKIP). Missing terminal still fails.
C16 record now carries healthy_correlation_id, threaded through to record_dict;
replay files without the key read as empty string. RECORD_VERSION stays 1.
Updated module docstring, scan loop comment, and tests (late_terminal,
absent_terminal, correlation_id; early-return test rewritten). No change to
grade_identity, grade_skip_never_pass, grade_typed_death.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 13
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 13 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

  • [MAJOR] scripts/ci/c16_receipt_identity_probe.py, _scan_terminal (qwen3-review) — caught_up check runs only after the wait loop exits, not per-iteration | The diff moves the caught_up computation out of the per-iteration loop body and into a single check after the loop. The loop's
  • [MAJOR] scripts/ci/c16_receipt_identity_probe.py, _scan_terminal (qwen3-review) — caught_up is computed against stale end offsets captured before the scan began | The ends dict is captured once at the start of the scan (the high watermarks at that instant). After the loop, the code
  • [MAJOR] scripts/ci/c16_receipt_identity_probe.py:685-698 (gpt-oss-review) — _scan_terminal may return before the configured wait_seconds elapse | The function returns as soon as the caught_up condition is satisfied, without ensuring that the total wait_seconds timeout has bee
  • [MAJOR] scripts/ci/c16_receipt_identity_probe.py:179-184, 433-440, 808-815 (gpt-oss-review) — Schema change without versioning may break downstream consumers | Adding healthy_correlation_id to Observations and Record alters the public data contract. Existing consumers that deserialize th
  • [MAJOR] scripts/ci/c16_receipt_identity_probe.py:748-756 (gpt-oss-review) — Potential AttributeError when healthy run receipt is missing | observe_live accesses healthy.correlation_id unconditionally when constructing Observations. If _observe_run returns a `RunObserv
  • [MINOR] scripts/ci/c16_receipt_identity_probe.py, module docstring (qwen3-review) — correlation_id written to record artifact; threat model justification is thin | The diff adds healthy_correlation_id to the Record dataclass and to record_dict, which serializes it into an uploaded ar
  • [MINOR] tests/ci/test_c16_receipt_identity_probe_omn19181.py (qwen3-review) — No test for the case where the terminal is found on the first poll | The new test test_a_late_terminal_after_the_watermark_is_found_inside_the_window covers a terminal arriving in a later batch. There
  • [MINOR] tests/ci/test_c16_receipt_identity_probe_omn19181.py, test_absent_terminal_is_established_only_after_the_window_ends (qwen3-review) — test_absent_terminal asserts elapsed >= 0.3, coupling test to wall-clock timing | The test asserts elapsed >= 0.3 where wait_seconds=0.3. This is a wall-clock assertion that will be flaky on heavily l
  • [MINOR] scripts/ci/c16_receipt_identity_probe.py, observations_from_replay (qwen3-review) — observations_from_replay defaults healthy_correlation_id to empty string with no validation of consistency | In observations_from_replay, if the key is absent, healthy_correlation_id defaults to ''. T
  • [MINOR] scripts/ci/c16_receipt_identity_probe.py, _scan_terminal (qwen3-review) — Full wait_seconds is now the minimum scan time in the absent case | As a consequence of moving the caught_up check outside the loop, the _scan_terminal function now always runs for the full wait_secon
  • [MINOR] scripts/ci/c16_receipt_identity_probe.py:808-815 (gpt-oss-review) — Healthy correlation ID is emitted in the public record | The healthy_correlation_id field is added to the Record output JSON. Although the comment states it is not a secret, exposing internal corr
  • [MINOR] scripts/ci/c16_receipt_identity_probe.py:433-440, 808-815 (gpt-oss-review) — Additional field increases serialization overhead | Including healthy_correlation_id in every Record adds extra bytes to the JSON payload and marginally increases memory usage during processing, w
  • [MINOR] tests/ci/test_c16_receipt_identity_probe_omn19181.py:306-322 (gpt-oss-review) — Time‑based test introduces flakiness on slow CI runners | The test test_absent_terminal_is_established_only_after_the_window_ends asserts that the scan duration is at least the supplied `wait_second

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

✅ Hostile Reviewer — REVIEWED

Critical findings: 0
Major findings: 5
Total findings: 13
Models succeeded: qwen3-review,gpt-oss-review
Models unavailable: none

Action required: findings were posted as review threads. Address or reject each one, then resolve its thread — the Hostile Review Thread Gate fails while hostile-reviewer threads are unresolved (OMN-17492).


Semantics (OMN-17492 — the model finds, thread resolution gates)

Surface Meaning Blocks merge?
Review threads Per-finding, posted by the reviewer No (informational)
Hostile Review Thread Gate Deterministic: unresolved hostile-reviewer threads exist Fails until resolved (not yet a required context)
degraded verdict Fewer than 2 models succeeded (infra) Fails this job with a named reason

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review (Qwen3.8-27B), gpt-oss-review (gpt-oss-120b) (OMN-8468/OMN-8524/OMN-17492)

@jonahgabriel
jonahgabriel added this pull request to the merge queue Oct 2, 2026
Merged via the queue into dev with commit 4897e56 Oct 2, 2026
228 of 239 checks passed
@jonahgabriel
jonahgabriel deleted the OMN-20359-c16-probe-race branch October 2, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant