Skip to content

feat(OMN-18868): a wire-compatibility gate replays every changed payload through the last released consumer - #2751

Merged
jonahgabriel merged 4 commits into
devfrom
jonah/omn-18868-wire-compatibility-gate
Sep 22, 2026
Merged

jonahgabriel merged 4 commits into
devfrom
jonah/omn-18868-wire-compatibility-gate

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

The seam nothing in this repository can see

In a test the producer and the consumer are the same commit. Version skew is therefore inexpressible as an in-repository test, and version skew is the entire defect.

omnimarket#2692 added published_at to ModelDelegateSkillRequest. Every check passed. The deployed consumer was an older released wheel whose model forbids extras, so it refused the payload at the decode boundary and dead-lettered every delegate (OMN-18852). 771 golden-chain files did not catch it and could not have.

A version comparison could not have caught it either, and this repository already has one. The postmortem records producer and consumer both reporting 0.4.134, because the consumer image had been baked fifteen minutes earlier from a different tree. Only decoding the new shape with the old model answers the question.

What the gate does

For every wire model module a pull request changes:

  1. resolve the last released version from the release tags this tree descends from;
  2. read that release's src/ out of git with git archive, no checkout, no second clone;
  3. ask the working tree, in its own subprocess, which keys each model can emit;
  4. push those keys through the released model's own model_validate, in a second subprocess whose sys.path is rooted in the released tree;
  5. fail on that validator's own refusal, naming the field, the model and the release.

Two versions of one package cannot share an interpreter, which is why each tree gets its own process and the two exchange JSON.

Proof against real history, not only fixtures

Anchored at v0.4.133, the last release before published_at landed, against today's tree:

WIRE_COMPAT_REFUSED: the last released consumer (v0.4.133, version 0.4.133) cannot
decode this tree's wire payloads.
  src/omnimarket/models/delegation/wire/model_delegate_skill_request.py: the released
  consumer ModelDelegateSkillRequest at v0.4.133 REFUSES the key 'published_at':
  Extra inputs are not permitted

That last clause is the live terminal's own words from 2026-09-20T01:43:38Z, reproduced rather than paraphrased.

Positive control, same command against the current release: every wire model in the tree passes, WIRE_COMPAT_PASS on v0.4.166. Both are tests, not one-off runs.

Two decisions worth reviewing

It grades keys, not values. Every key carries one placeholder and only extra_forbidden and missing are findings. Synthesising a type-correct value per annotation means shipping a factory library, and every gap in it reddens a change that broke nothing, which is how a gate gets switched off. The two graded error types are exactly the two a placeholder cannot manufacture: one fires on a key being present, the other on it being absent, and neither reads the value.

It does NOT take the release-identity gate's shallow-clone fallback, deliberately. There, widening to the full tag list is the stricter answer. Here the arithmetic inverts: fewer reachable tags resolve an older consumer that knows fewer fields, so --merged is the strict direction and the full list is the permissive one. git tag --merged is used unconditionally and an unresolvable anchor raises.

Controls

Fourteen, split so neither half can stand alone. A gate with only falsifiers is a blanket freeze; a gate with only positive controls has never been proven able to fail.

  • Falsifiers — added field the release refuses; dropped field the release still requires; and the one that separates this from a same-commit test, a tree whose own model tolerates the payload while the release does not, with a first assertion proving the tree really is blind.
  • Positive controls — a field the release already declares; a release that tolerates extras; a change outside the wire package; a module the release does not contain; and the current release decoding today's wire models.
  • Fail-closed arms — no release tag, a release carrying no source tree, a released module that will not import, and the guard against a probe resolving the working tree as the release.
  • Drift guard — the declared wire-model set is asserted against every wire/ package in the tree, so a new one turns a test red until someone decides in writing whether it is graded.

Enforcement

Registered in the strict gate-job set, so it blocks through the existing required CI Summary context. No branch-protection change is required and none is requested, on the same terms as the release-identity gate: that context is already required on dev and main, so a pull request whose wire payload no released consumer can decode cannot be required-green.

Limits, stated rather than left to be found

  • Top-level keys only. A nested model's shape change is graded when that model's own module is in the changed set, not transitively through its parent. A nested type living outside the wire package is out of scope.
  • No waiver list. One was considered and rejected: a waiver is the hole the gate exists to close, and the sanctioned escape is the release order it is enforcing.
  • It grades the maximal shape the producer can emit, so a new optional field is a finding even where the producer omits it while unset. That is the rule working, not over-strictness: published_at is excluded when None today and is still refused the moment anything stamps it, which is the reason the field exists.

Ticket: OMN-18868. Parent: OMN-18864. Cause: OMN-18852, OMN-18633.

Evidence-Ticket: OMN-18868
Evidence-Source: OCC#10788

The change-control citation above was moved from the earlier companion to OCC#10788, which adds a record binding the wire-compatibility proof to the current head d5d5ece of this pull request. The earlier companion records describe superseded heads.

@jonahgabriel
jonahgabriel enabled auto-merge (squash) September 21, 2026 15:31
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

⚠️ Architectural Review — WARNINGS

Errors: 0
Warnings: 1

What this checks

Rule Description
ARCH-TOPIC-001 No hardcoded Kafka topic strings in handler code
ARCH-DI-001 No event_bus=None bypass
ARCH-DI-002 No direct Handler instantiation outside workflow_runner/adapters
ARCH-DI-003 No reinvented DI containers
ARCH-TOPIC-002 contract.yaml topics follow `onex.{cmd

ARCH-DI-002: Direct Handler instantiation (warning)\n\n```\nsrc/omnimarket/nodes/node_delegate_skill_orchestrator/ports/port_local_delegation_dispatch.py:796: self._effect_handler = effect_handler or HandlerLlmDelegationCall()

src/omnimarket/nodes/node_delegate_skill_orchestrator/ports/port_local_delegation_dispatch.py:797: self._projection_handler = projection_handler or HandlerProjectionDelegation()
src/omnimarket/nodes/node_delegate_skill_orchestrator/ports/port_local_delegation_dispatch.py:817: self._judge = judge if judge is not None else HandlerJudgeAdequacy()\n```\n\nHandler classes should be wired only in workflow_runner.py or adapter modules. Direct instantiation elsewhere bypasses the contract system.\n\n

Static architectural lint — no model inference (OMN-14176).

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

✅ Hostile Reviewer — PASSED

Blocking findings (critical/error): 0
Total findings: 15
Models succeeded: qwen3-review,qwen3-review-b


Gate semantics

Verdict Meaning Blocks merge?
passed >=2 models succeeded, no critical/error findings No
blocked CRITICAL or ERROR findings found Yes
degraded Fewer than 2 models succeeded (infra unavailable/timeout) Yes (OMN-15110)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524)

onexbot-occ-writer Bot added a commit to OmniNode-ai/onex_change_control that referenced this pull request Sep 21, 2026
…nimarket#2751

The declared checks were executed for real in the OmniNode-ai/omnimarket checkout at the PR head by the product-repo OCC receipt runner, and the results written append-only into this companion. Producer-minted PENDING receipts are superseded by a net-new record; no existing receipt was edited.

Ticket: OMN-16859 (AC3b)
Onex-Workflow: OCC Receipt Runner
Onex-Run: https://github.com/OmniNode-ai/omnimarket/actions/runs/35620477201
onexbot-occ-writer Bot added a commit to OmniNode-ai/onex_change_control that referenced this pull request Sep 21, 2026
…nimarket#2751

The declared checks were executed for real in the OmniNode-ai/omnimarket checkout at the PR head by the product-repo OCC receipt runner, and the results written append-only into this companion. Producer-minted PENDING receipts are superseded by a net-new record; no existing receipt was edited.

Ticket: OMN-16859 (AC3b)
Onex-Workflow: OCC Receipt Runner
Onex-Run: https://github.com/OmniNode-ai/omnimarket/actions/runs/35654233846
@jonahgabriel

Copy link
Copy Markdown
Contributor Author

Pre-PR proof countersign: receipt sha256=9f422e1dc77b995c2d5a46efbdfd77d245f443d7f2698c6d6f85b633318ef5af actor=prepr-countersign verdict=COUNTERSIGNED-WITH-LIMITS

Third-actor confirmation of the OMN-19050 latch fix on this chain. The digest above is the attempt-scoped supersede record in the companion. The fresh-event claim holds and the resolver behaves exactly as the semantics require. I also found one thing the lane did not ask me to look for, at the end, and it is the reason this is a limits verdict rather than a clean one.

This was a fresh event, not a replay, on six independent discriminators. Any one could have failed and none did.

the FAIL the PASS
Actions run 35620477201 35654233846
run attempt 1 1
head commit bab99887 b5c6f7a4
created 15:39:58Z 20:57:04Z
duration 15607 ms 16885 ms
tests 1 failed, 12 passed, 1 skipped 18 passed, 2 skipped

Two run identifiers with both attempts reading one rules out a re-run, which mints a second attempt on the first identifier rather than a new run. The test totals differ, fourteen against twenty, so the two executions did not see the same tree. The timing is tight in the right direction: the merge commit was authored at 20:56:58Z and the run created six seconds later.

It executed merged code. The omnimarket half merged at 20:50:18Z and its merge commit is an ancestor of this head with nothing behind. The other two merged at 20:12:35Z and 20:35:00Z, both before the run was created. I re-asserted the merged-code point the way the lane asked, but by content rather than by git diff: both modules I ran were staged as immutable archives addressed by their merge commits and their digests compared against the authoritative blobs at those same commits.

I ran the merged resolver rather than reading it, against the three real record files, unmodified.

A positive control: FAIL record alone        -> FAIL  from ...supersede.2751.yaml
B real chain, records unmodified             -> PASS  from ...supersede.2751.0002.yaml
C negative control: PASS at the FAIL head    -> FAIL  from ...supersede.2751.yaml
D positive control: PASS at any other sha    -> PASS  from ...supersede.2751.0002.yaml

Case B is the selection this confirmation is about, and it matches what the lane reported. Case C is the negative control the semantics require: the only field differing from B is the passing record's commit identifier, set to the failing record's own head, and the resolver returns the failing record. A pass re-filed at the head that already failed does not clear it, so the chain cannot be used as a retry-until-green channel. Case D exists because C alone would not separate the rule from the mutation; moving the identifier anywhere else restores the pass.

Worth noting as correct rather than incidental: the guard compares against the latest prior failure and then stops, so an old failure at an unrelated head cannot be resurrected by a later pass that happens to share its commit.

The finding. The two halves of this fix disagree on the negative control, and I measured it. The gate helper the third pull request changed is not a caller of the resolver; it is a second, independent implementation of the same selection, and the guard was added to only one of them. Running the merged gate helper over the identical four cases:

case merged resolver merged gate helper
A FAIL alone the FAIL the FAIL
B real chain the PASS the PASS
C PASS at the FAIL head the FAIL the PASS
D PASS at another sha the PASS the PASS

The gate helper ranks purely by dotted-numeric sequence and applies no commit-identity guard, so on case C it selects the record the resolver refuses. On the live chain this changes nothing, because the real records carry different commits and both halves agree on case B, which is why this is a limits verdict and not a refusal.

I am not calling it an open hole, because I did not establish which of the two binds the evidence outcome. The gate's own documentation names the resolver as the authority for the active receipt and casts the gate as validating the replacement, which if correct makes this a defence-in-depth gap rather than a bypass. That is the question for the owners, and the answer decides the severity. The durable fix is the same either way: have the gate call the resolver rather than reimplement its ranking, so a future rule cannot be added to one and missed by the other. One rule with two implementations is how this class of divergence recurs.

Not proven, and not substituted for. This pull request has not merged and its preflight is still red, which the lane stated plainly and which I am not treating as part of the claim. The cause is a formatting hook in the companion wanting to re-wrap the failing record's captured output scalar, red since before any of this work existed and handed to another lane. What is countersigned here is the resolution behaviour on the live chain, not mergeability today.

jonahgabriel added a commit to OmniNode-ai/onex_change_control that referenced this pull request Sep 21, 2026
…imarket#2751 (#10688)

* evidence(OMN-18868): author OCC companion for OmniNode-ai/omnimarket#2751

OCC companion by node_pr_lifecycle_fix_effect (OMN-13317 F1 / OMN-13990 / OMN-14285). Product PR head bab9988752f031628bf02fbecae852bf6fa0acee.

* evidence(OMN-18868): self-bind OCC#10688 + rebind contract_sha256

* evidence(OMN-16859): executed test_passes receipts for OmniNode-ai/omnimarket#2751

The declared checks were executed for real in the OmniNode-ai/omnimarket checkout at the PR head by the product-repo OCC receipt runner, and the results written append-only into this companion. Producer-minted PENDING receipts are superseded by a net-new record; no existing receipt was edited.

Ticket: OMN-16859 (AC3b)
Onex-Workflow: OCC Receipt Runner
Onex-Run: https://github.com/OmniNode-ai/omnimarket/actions/runs/35620477201

* evidence(OMN-16859): executed test_passes receipts for OmniNode-ai/omnimarket#2751

The declared checks were executed for real in the OmniNode-ai/omnimarket checkout at the PR head by the product-repo OCC receipt runner, and the results written append-only into this companion. Producer-minted PENDING receipts are superseded by a net-new record; no existing receipt was edited.

Ticket: OMN-16859 (AC3b)
Onex-Workflow: OCC Receipt Runner
Onex-Run: https://github.com/OmniNode-ai/omnimarket/actions/runs/35654233846

* style(OMN-18868): yamlfmt the FAIL supersede record so the companion can merge

Pre-commit has been red on every head of this companion since 15:42Z, on one
file and one hook: yamlfmt reports "files were modified by this hook" against
test_passes.supersede.2751.yaml, the FAIL record the old runner wrote. It is
not the supersede latch, which is fixed and merged, and it predates that work.

Only the formatter's line wrapping changes. Verified rather than assumed
before committing, because OMN-15479 exists for the case where running this
formatter DESTROYS a value:

  - the parsed documents are equal before and after;
  - probe_stdout is byte-identical, 2699 characters both sides;
  - the replacement record still reads status FAIL;
  - the OMN-15479 contamination ratchet passes on the reformatted file,
    with no line-marker sentinel in any parsed value and no new folded
    block scalar.

So this is the opposite of the corruption case: the rewrite only moves where
the line-continuation backslashes sit inside one long double-quoted scalar.

The append-only invariant is untouched. This record exists on no other branch,
absent from both main and dev, so nothing merged is being edited, and the PASS
record beside it, test_passes.supersede.2751.0002.yaml, is not touched at all.

WHY IT ESCAPED THE EXISTING HANDLING, worth a follow-up rather than a silent
fix: the runner carries yamlfmt-stability handling from OMN-17794 that covers
a plain or single-quoted multi-line scalar and a literal block with the keep
indicator. A long DOUBLE-QUOTED scalar with backslash continuations is a third
shape, and it is the shape a FAIL record produces, because a failing check
captures a lot of stdout. The PASS record written beside this one was not
flagged, because its stdout is one short line. The trigger is therefore a FAIL
record with long captured output, and it will recur on the next one.

Diagnosed by the peer lane that fixed the latch; reproduced, re-verified and
committed here as the owner of OMN-18868.

Onex-Lane: omn18868-67-seam-gates-restart
Onex-Session: 01403a8ae0e24ab2a42c076e81600810

---------

Co-authored-by: omnimarket-bot <bot@omninode.ai>
Co-authored-by: onexbot-occ-writer[bot] <307849072+onexbot-occ-writer[bot]@users.noreply.github.com>
Co-authored-by: Jonah Gray <jonah@omninode.ai>
…eleased consumer

A test cannot see a version skew, because in a test the producer and the
consumer are the same commit. omnimarket#2692 added published_at to
ModelDelegateSkillRequest, passed every check in this repository, and was then
refused at the decode boundary by the deployed extra-forbid consumer, which
dead-lettered every delegate (OMN-18852). 771 golden-chain files could not have
caught it, and neither could a version comparison: the postmortem records
producer and consumer both reporting 0.4.134.

The gate resolves the last release from the tags this tree descends from, reads
that release's src/ out of git with no checkout, and pushes the working tree's
emitted key set through the released model's own model_validate in a subprocess
rooted in the released tree. Two versions of one package cannot share an
interpreter, so each tree gets its own process and they exchange JSON.

Only extra_forbidden and missing are findings; every key carries one placeholder
and type errors are discarded, because synthesising a value per annotation would
mean shipping a factory library whose every gap reddens a change that broke
nothing. Unlike the release-identity gate this one does NOT widen the tag set on
a shallow clone: there the wider set is stricter, here it resolves a NEWER
consumer and is the permissive direction.

Proven against real history: anchored at v0.4.133 the gate reproduces the live
terminal verbatim, published_at Extra inputs are not permitted, and against the
current release every wire model in the tree passes. Twelve fixture controls
cover the added-field and dropped-field arms, the compatible-tree with
incompatible-release case that separates this from a same-commit test, two
positive controls, three fail-closed arms and the guard against a probe
resolving the tree as the release.

Registered in the strict gate-job set, so it blocks through the existing
required CI Summary context and needs no branch-protection change.

Claude-Session: https://claude.ai/code/session_011WaS5jrEdJDstEEtiJuRvW
Onex-Lane: omn18868-67-seam-gates-restart
Onex-Session: 69ca97601b144645814a468e142f629b
The module docstring underlined its section headings with rows of equals
signs, which the strict ai-slop sweep classes as a markdown separator in a
docstring and refuses. The headings stay; only the rules under them go.

Claude-Session: https://claude.ai/code/session_011WaS5jrEdJDstEEtiJuRvW
Onex-Lane: omn18868-67-seam-gates-restart
Onex-Session: 69ca97601b144645814a468e142f629b
…xists

The gate was sound and its own controls refused the pull request. The
occ-receipt-runner executed this file in a depth-1 checkout with no tags
(run 35620477201) and test_the_current_release_decodes_todays_wire_models
failed with WIRE_COMPAT_RELEASE_UNRESOLVABLE -- the correct fail-closed answer
from the gate, and the wrong reading for a control, because it says nothing
about whether dev is compatible. The sibling anchored replay already skipped
on that same precondition, so one file handled the same missing history two
different ways.

Both now share one precondition helper and skip with a reason that names where
they DO run. Both are recorded in config/skip_count_baseline.yaml, max_skips
259 to 261, with provenance stating why the shard job can never execute them.

They are not unexecuted work. The Wire Compatibility Gate job checks out at
fetch-depth 0, so it is the one checkout that carries the tags and the
ancestry, and it now runs them with -m integration and then reads the JUnit
report -- because two SKIPPED controls also exit zero, and trusting the exit
status would turn this into a gate that prints and passes.

test_the_gate_job_runs_the_real_history_controls asserts that arrangement
against the parsed workflow tree rather than against a string, covering the
five ways the gate could exist and enforce nothing: job absent, name outside
STRICT_GATE_JOBS, job made conditional, checkout thinned, report unread. Five
negative controls mutate the live workflow and assert each is caught. Unlike
the two controls it runs in every checkout.

Claude-Session: https://claude.ai/code/session_01EYUMfw1eY4umCKvhQvS8ZF
Onex-Lane: omn18868-67-seam-gates-restart
Onex-Session: fe0fe067f2f64ea5bd614e79e36ac3dd
…ix is in force

OMN-19050 closed the FAIL-supersede latch in omnibase_core and merged to
that repo's dev. It is inert on the one path that decides pull requests
here, because the Receipt Gate this repo runs is pinned to a commit that
predates it.

The pin was 76532c20, version 0.47.19, an ancestor of the fix a03b1072 by
exactly 11 commits. At that sha the resolver's supersede-suffix pattern is
still the dot-excluding form and neither _sequence_key nor _guarded_winner
exists, so an attempt-scoped record is not outranked there -- it is
invisible. Live consequence on #2751: occ-preflight passes while the
Receipt Gate reports a non-pass receipt for a key whose merged companion
already carries a PASS, and the gate's receipt_ids omits the key entirely,
which is the signature of an unseen record rather than a rejected one.

Proven with a negative control, run over the same three merged receipt
files from onex_change_control dev both times:

  core at 76532c20, the old pin   -> test_passes.supersede.2751.yaml
                                     FAIL at bab9988
  core at a03b1072, the new pin   -> test_passes.supersede.2751.0002.yaml
                                     PASS at b5c6f7a

Same input, opposite verdicts, and the pin is the only variable.

No published release carries the fix. v0.47.20 is the latest, tagged
3afa3528 on 2026-09-20, and the fix merged 2026-09-21; git tag --contains
on the fix returns empty. A released version would be preferable for a
required-gate pin, so this pins the merge sha and the comment says it
should be re-pointed at the first release that carries it.

Blast radius, measured rather than asserted. receipt-gate.yml is
byte-identical across the move, blob cb0c2b54 at both shas, so the gate's
own shape does not change. The validation surface moves 18 files,
+893/-220, and the two gate-relevant validators besides the resolver,
validator_occ_append_only.py and validator_receipt_diff_consistency.py,
both come from one commit, omnibase_core#1730 (OMN-18996).

One line plus its comment. No other file changes.

Evidence-Ticket OMN-19149

Onex-Lane: omn18868-67-seam-gates-restart
Onex-Session: 69ca97601b144645814a468e142f629b
@jonahgabriel
jonahgabriel force-pushed the jonah/omn-18868-wire-compatibility-gate branch from b5c6f7a to d5d5ece Compare September 22, 2026 07:48
@jonahgabriel
jonahgabriel merged commit a035e53 into dev Sep 22, 2026
164 of 166 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-18868-wire-compatibility-gate branch September 22, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant