Skip to content

feat(OMN-18949): gate census drift in CI, and report live lane drift to a person - #3913

Merged
jonahgabriel merged 1 commit into
devfrom
jonah/omn-18949-census-drift-gate-v2
Sep 21, 2026
Merged

jonahgabriel merged 1 commit into
devfrom
jonah/omn-18949-census-drift-gate-v2

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Closes OMN-18949 (under OMN-18276).

The gap

Nothing refused on census drift, and nothing reported it. The committed snapshot carried a drift count of five at severity warning; the only surface anywhere that read it rendered the prose string 5 drift item(s) — see census into a doctrine file, where the next refresh normalised it away.

A second, quieter half turned up on the way in. The table generator's own docstring claimed CI fails when the doctrine file's lane block drifts from what it would produce. No such check existed. The workflow invoked the generator with neither a comparison flag nor an output path, purely to prove it does not crash, and the claim stood for months.

What landed

Two halves, because they answer two different questions.

In CI, a new drift gate wired into the existing lane-census job: do the two committed files still describe the same world?

finding condition
stale_unexpected_container a finding reports a container as undeclared, and the manifest now declares it
stale_container_absent the mirror: a finding outlived the declaration it was about
snapshot_lane_unknown a lane one file knows and the other does not
finding_count_mismatch the snapshot's own count disagrees with its own findings
generator_lane_undeclared a manifest lane with no explicit entry in the generator's static maps
generator_map_orphan a static-map entry naming a lane the manifest has retired

It deliberately does not require the drift count to be zero. Live drift on a mutable lane is a normal, reportable state, and a gate refusing every pull request over it would be switched off within a day. A test pins that negative so a later change has to argue with it.

On the lab host, a census row in the system report: has the live topology moved? CI cannot see that host's docker daemon at all. The row reuses the existing poster, per-key de-duplication, hysteresis and cron. No new cron unit, no new credential, no new probe script — it drives the same checker CI runs, pointed at the live snapshot, so the two verdicts cannot diverge.

"More than one tick" is not implemented in the probe. The shared alert machinery already requires a key to hold its status for two consecutive ticks before it pages; a private counter would be a second answer that disagrees the first time either default changed. A test pins the reuse.

Three defects found and fixed on the way

  • The docstring claim above. Corrected in place, naming why the comparison was never built: the block lives in another repository this job does not check out.
  • Two lanes were absent from the generator's static maps — the dogfood lane from both, the fleet CI bus from the port map. Each rendered an em-dash for that reason alone, indistinguishable from a lane that genuinely has none. Both are declared now, and differently: the bus declares no runtime ports, the dogfood lane's have simply never been recorded, and the rendered table now says which. Inventing the dogfood ports from an unverified compose read would have been the wrong repair.
  • The probe's first draft discarded the checker's stderr, turning a missing YAML dependency into the row exit 1. Captured now, so the row carries the cause.

Incident replay, as the default-deny coverage gate requires. Both artifacts are read out of the object store at 2e4eec05c, where the lab prod lane had been removed from the manifest and both static maps still carried its rows. They stayed six days, invisible because the generator iterates the manifest, and were removed only when the pre-PR pool claimed the exact ports the dead row still named. The discriminator requires the same functions to return nothing against the current tree, so a guard refusing everything could not satisfy it, and a third case requires the finding to name only the retired lane.

RED/GREEN on the undeclared-lane finding, by removing the two entries this PR adds:

RED   FAIL: 2 disagreement(s) ... [generator_lane_undeclared] lane=dogfood / lane=ci-bus
GREEN OK: the committed census and the manifest agree.

Focused suites: 76 passed across the new drift tests, the new reporter-sink tests, the incident replay, and the two pre-existing census modules. The whole reporter suite (62 cases) re-run green after its harness gained the new probe's off-switch. Lint, format and strict type-check clean on both scripts. The coverage gate reports 62 guard(s) covered, 159 wired, 0 pending.

An unreadable or empty input exits distinctly rather than reporting agreement, and every run prints the input sizes it derived its verdict from, so a zero is never reported bare.

Residual, stated rather than implied

--check <path> now implements the comparison the docstring claimed, ignoring only the provenance timestamp. It is not run against the registry repository's doctrine file in this job: that is a different repository this CI does not check out, and making a path-filtered gate depend on a cross-repo fetch trades one silent failure for an availability risk on unrelated pull requests. Closing it needs a read token and belongs on the six-hourly refresh workflow, which already mints one. The residual is written into the workflow beside the step.

The reporter change reaches the lab host through the normal maintenance sync, so the census row appears on the first tick after that sync, not on merge.

Enforcement path

The job is path-filtered and is not itself a required context. A red check-run on a head is caught by the default-deny external sweep under the required summary umbrella, which landed in the commit this branch is based on. No branch-protection change is requested, and none should be made: adding a path-filtered job to the required set wedges every pull request that does not touch those paths.

DoD

Fresh-event: the first pull request touching the census paths after merge runs the gate, and the first host tick after the maintenance sync emits a census row. Deliberately not self-authored.

Operator action

None required. No branch-protection change, no credential change, no version bump (no source package change).

Why this replaces an earlier pull request

omnibase_infra#3909 carried the same change and was cut before two things landed on dev:

  • OMN-18944 added a structurally identical reporter sink in the same four places this one touches. Both changes are the same shape by design, so they collided at every one of them.
  • OMN-18980 fixed the census refresh test's end-to-end case, which this branch had also touched. Its fix is more thorough than the one here was, and is taken whole. Two fixes for one defect in one file is how the weaker one survives.

This branch is cut at current dev with both already present and carries only this sink on top. #3909 should be closed unmerged.

The day-three time bomb, and what this gate does about it

The failure that held #3909 is worth recording, because it is this ticket's own subject pointed the wrong way. The refresh test read the live committed census's age, so every open pull request in the repository went red on the day that file turned three days old — which is exactly the day the refresh bot opens its pull request. The three-day threshold exists to leave four days of margin before the seven-day age gate; the test made that margin unusable, because nothing could merge during it, including the refresh itself.

The gate added here does not have that property, deliberately. It asserts that the two committed files agree with each other, never that either is recent. Age is a separate, existing check with its own threshold. A gate whose verdict changes because the calendar advanced is a gate that reddens the fleet on a schedule, and this one cannot: every finding it produces is a disagreement between two artifacts, resolvable by editing one of them.

The refresh path never depends on the thing it refreshes. Verified on the live event rather than asserted: the bot's refresh #3907 was authored and merged by the bot app at 2026-09-21T09:00:58Z, merge commit d0fe11626320, with no human in the loop. Its single summary run polled for twelve and a half minutes and passed. A report reached this lane that the summary had raced and failed closed after thirteen seconds; that does not reproduce. The pull request had one head, one summary run, and it succeeded.

Evidence-Ticket: OMN-18949
Evidence-Source: OCC#10652

…to a person

Nothing refused on census drift and nothing reported it. The committed snapshot
carried findings at severity warning; the only surface that read it rendered a
prose drift count into a doctrine file, where the next refresh normalised it
away.

Two halves, because they are two questions.

CI (scripts/check_lane_census_drift.py, wired into the lane-census-staleness
job): do the two COMMITTED files still describe the same world? A finding
reporting a container as undeclared that the manifest now declares, a snapshot
that miscounts its own findings, a lane one file knows and the other does not,
and a manifest lane the table generator has no entry for are each a hard
failure. It deliberately does NOT require drift_count to be zero: live drift on
a mutable lane is a normal reportable state, and a gate refusing every PR over
it would be switched off within a day.

The .201 reporter (a census row): has the LIVE topology moved? CI cannot see
the lab host's docker daemon at all. The row reuses the existing Slack poster,
per-key de-duplication, hysteresis and the existing cron -- no new cron unit,
no new credential, and no new probe script, because it drives the same checker
CI runs. More than one tick is the shared CONFIRM_TICKS default and is
deliberately NOT reimplemented in the probe, where it would be a second answer
that could disagree.

Ships with its incident replay, as the default-deny coverage gate requires. The
artifacts are the census manifest and the table generator read out of the
object store at 2e4eec0, where OMN-18320 had removed the lab prod lane and
both of the generator static maps still carried its rows. They stayed six days,
invisible because the generator iterates the manifest, and were removed only
when the pre-PR pool claimed the exact ports the dead row named. The
discriminator requires the same functions to return nothing against the current
tree, so a guard that refused everything could not satisfy it.

Three defects found and fixed on the way. The generator docstring claimed CI
failed when the doctrine file lane block drifted; no such check existed, and
the workflow ran the generator against nothing as a smoke test. The dogfood
lane was absent from both static maps and the fleet CI bus from the port map,
so each rendered an em-dash for that reason alone, indistinguishable from a
declared absence; both are declared now, and differently, behind explicit
sentinels. And the probe first draft discarded the checker stderr, turning a
missing PyYAML into a bare exit code with no cause.

The --check flag implements the comparison the docstring claimed, ignoring only
the provenance timestamp. It is NOT run against the registry repo doctrine file
in CI: that is a different repository this job does not check out, and the
residual is stated in the workflow rather than papered over.

Cut fresh at current dev. The first branch carried the same change, but
OMN-18944 landed a structurally identical sink in the same four places of the
reporter while it was open, and OMN-18980 fixed a census test this branch had
also touched. Both of those are taken as they stand on dev; only this sink is
added on top.
@jonahgabriel
jonahgabriel enabled auto-merge (squash) September 21, 2026 10:39
jonahgabriel pushed a commit to OmniNode-ai/onex_change_control that referenced this pull request Sep 21, 2026
#10652)

* evidence: OCC companion pass 1 for OmniNode-ai/omnibase_infra#3913

* evidence: OCC companion self-bind for #10652

---------

Co-authored-by: node-occ-companion-effect <occ-companion-effect@omninode.ai>
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Hostile Reviewer — DEGRADED (informational)

Critical findings: 0
Major findings: 0
Total findings: 0
Models succeeded: glm-review

Note: Fewer than 2 reviewer models succeeded. Degraded results are informational (OMN-8468/OMN-8524) and do not block merge. Error: cli_review exit 2 (fewer than 2 models succeeded — partial/total outage)


Semantics (OMN-17492 — the model finds, thread resolution gates)

Surface Meaning Blocks merge?
Review threads Per-finding, posted by the reviewer No (informational)
Hostile Review Thread Gate Deterministic: unresolved hostile-reviewer threads exist Fails until resolved (not yet a required context)
degraded verdict Fewer than 2 models succeeded (infra) No

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review, qwen3-review-b, glm-review (OMN-8468/OMN-8524/OMN-17492)

@jonahgabriel
jonahgabriel merged commit 77df826 into dev Sep 21, 2026
223 of 231 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-18949-census-drift-gate-v2 branch September 21, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant