Repository navigation
evidence(OMN-15253): falsifiable deploy-scope probes for the slice-1 product diffs - #5164
Merged
Merged
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 44 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds two verified command-based evidence entries for OMN-15253: one validating deploy-staging readiness at a pinned commit and one validating OCC self-bind PR state, with corresponding passing receipt artifacts. ChangesEvidence records
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
added 2 commits
July 27, 2026 11:43
…product diffs The autobind companion (#5152) bound Evidence-Source but declared no falsifiable deploy probe, so deploy-gate correctly failed omnimarket#1920: "PR touches runtime paths but no cited ticket has deploy DoD evidence." Slice 1 is source-only by design (proof_class: code-only) — the readiness verdict only becomes live-proven in slice 2, when the collect EFFECT captures a real snapshot from the canonical dev cluster and its output is compared against the bash verifiers. So these are the hermetic content probes the gate accepts for compute-only tickets: they read the deployed-source surface at the pinned product commits, never a receipt or contract this ticket's own PRs author. They go RED on real regressions: the def-B signature or the pure/compute descriptor changing; ModelEventEnvelope/ModelHandlerOutput reappearing in the handler; the composition document losing its LIVE-verified broker instance class or migration digest; or scripts/required-configmap-vars.txt being re-added after this change deletes it. Refs: omnimarket#1920, omninode_infra#702, OMN-14724.
jonahgabriel
force-pushed
the
jonah/omn-15253-deploy-evidence
branch
from
July 27, 2026 18:45
23a87f1 to
6c0d382
Compare
… token cannot read that repo) The composition-document probe returned HTTP 404 from the Contract Compliance runner: the OCC CI token has no read access to OmniNode-ai/omninode_infra, so that probe is unrunnable in the environment the gate executes it in. An evidence item the gate can never run is worse than no item — it would sit RED forever and teach everyone to ignore the gate. The omninode_infra half is covered where it can actually be executed: tests/k8s/test_staging_composition_contract.py in that repo asserts the document's self-hash, the LIVE-verified broker instance class, the pinned migration digest, and manifest agreement, and runs in its own CI on every PR touching k8s/. The omnimarket deploy probe (def-B signature, no envelope shape, pure COMPUTE descriptor at product commit 3096f242) is retained and PASSes in-gate.
jonahgabriel
force-pushed
the
jonah/omn-15253-deploy-evidence
branch
from
July 27, 2026 19:19
e92c373 to
c642080
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@contracts/OMN-15253.yaml`:
- Around line 38-54: Make every copy of the OMN-15253 deploy probe fail closed
and validate structure precisely: in contracts/OMN-15253.yaml lines 38-54,
require successful API responses before evaluating results and replace
line-based grep with AST-aware handler assertions and YAML-aware contract
assertions for the request signature, forbidden envelope types, and pure COMPUTE
declaration. Regenerate
drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml
lines 6-10 and 18-22 so both check_value and probe_command contain the identical
corrected probe.
In
`@drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml`:
- Around line 1-2: Rename both DoD receipt files to timestamped paths using the
<run_timestamp>.yaml naming convention:
drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml
(lines 1-2) and drift/dod_receipts/OMN-15253/occ-self-bind-pr-5164/command.yaml
(lines 1-2) require the same rename; no content change is needed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1eae1121-91e0-42b2-812f-cf3e2a50069b
📒 Files selected for processing (3)
contracts/OMN-15253.yamldrift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yamldrift/dod_receipts/OMN-15253/occ-self-bind-pr-5164/command.yaml
Comment on lines
+38
to
+54
| Deploy-scope falsifiable probe for the OMN-15253 slice-1 omnimarket diff at product commit 75ec54c3. | ||
| Slice 1 is source-only and ships no live deployment (proof_class: code-only; the verdict only becomes | ||
| live-proven in slice 2 when the collect EFFECT captures a real snapshot), so this is the hermetic | ||
| content probe the deploy gate accepts for compute-only tickets. It reads the deployed-source surface | ||
| at the product commit, never a receipt or contract this ticket's own PRs author, and goes RED if | ||
| the def-B signature regresses, if the pre-def-B envelope shape reappears, or if the node stops declaring | ||
| itself a pure COMPUTE. | ||
| source: "manual" | ||
| status: "verified" | ||
| checks: | ||
| - check_type: "command" | ||
| check_value: >- | ||
| gh api 'repos/OmniNode-ai/omnimarket/contents/src/omnimarket/nodes/node_staging_readiness_compute/handlers/handler_staging_readiness_compute.py?ref=75ec54c36035d4bfdfe0d6fc7f2a94b8ea1c8389' | ||
| --jq .content | base64 -d | grep -c 'request: ModelStagingReadinessRequest' && ! gh api 'repos/OmniNode-ai/omnimarket/contents/src/omnimarket/nodes/node_staging_readiness_compute/handlers/handler_staging_readiness_compute.py?ref=75ec54c36035d4bfdfe0d6fc7f2a94b8ea1c8389' | ||
| --jq .content | base64 -d | grep -qE '(import|from).*Model(EventEnvelope|HandlerOutput)' && | ||
| gh api 'repos/OmniNode-ai/omnimarket/contents/src/omnimarket/nodes/node_staging_readiness_compute/contract.yaml?ref=75ec54c36035d4bfdfe0d6fc7f2a94b8ea1c8389' | ||
| --jq .content | base64 -d | grep -c 'purity: pure' |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Make every copy of the deploy probe fail closed and structurally precise.
The contract and both receipt command fields currently allow an API failure to satisfy the inverted negative pipeline, and raw line-oriented grep does not prove the required handler or descriptor structure.
contracts/OMN-15253.yaml#L38-L54: replace the probe with explicit API-status checks and AST/YAML-aware assertions.drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml#L6-L10: regeneratecheck_valuefrom the corrected probe.drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml#L18-L22: regenerateprobe_commandidentically.
📍 Affects 2 files
contracts/OMN-15253.yaml#L38-L54(this comment)drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml#L6-L10drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml#L18-L22
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@contracts/OMN-15253.yaml` around lines 38 - 54, Make every copy of the
OMN-15253 deploy probe fail closed and validate structure precisely: in
contracts/OMN-15253.yaml lines 38-54, require successful API responses before
evaluating results and replace line-based grep with AST-aware handler assertions
and YAML-aware contract assertions for the request signature, forbidden envelope
types, and pure COMPUTE declaration. Regenerate
drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml
lines 6-10 and 18-22 so both check_value and probe_command contain the identical
corrected probe.
Comment on lines
+1
to
+2
| --- | ||
| schema_version: 1.0.0 |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Move both receipts to timestamped paths.
drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml#L1-L2: rename the receipt to<run_timestamp>.yaml.drift/dod_receipts/OMN-15253/occ-self-bind-pr-5164/command.yaml#L1-L2: rename the receipt to<run_timestamp>.yaml.
As per coding guidelines, DoD receipts must be stored at the timestamped path.
📍 Affects 2 files
drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml#L1-L2(this comment)drift/dod_receipts/OMN-15253/occ-self-bind-pr-5164/command.yaml#L1-L2
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml`
around lines 1 - 2, Rename both DoD receipt files to timestamped paths using the
<run_timestamp>.yaml naming convention:
drift/dod_receipts/OMN-15253/dod-deploy-staging-readiness-node-75ec54c3/command.yaml
(lines 1-2) and drift/dod_receipts/OMN-15253/occ-self-bind-pr-5164/command.yaml
(lines 1-2) require the same rename; no content change is needed.
Source: Coding guidelines
jonahgabriel
force-pushed
the
jonah/omn-15253-deploy-evidence
branch
from
July 27, 2026 19:35
c642080 to
38f3bae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OMN-15253 — deploy-scope evidence for the slice-1 product PRs
Product PRs: OmniNode-ai/omnimarket#1920, OmniNode-ai/omninode_infra#702
Ticket: https://linear.app/omninode/issue/OMN-15253
Why this PR exists
The autobind companion (#5152) bound
Evidence-Sourcebut declared no falsifiable deploy probe, so deploy-gate correctly failed omnimarket#1920:That is the gate working. This PR supplies the missing evidence rather than routing around it.
Why hermetic content probes and not a live deploy probe
Slice 1 is source-only by design —
proof_class: code-only, stated in the ticket and in both product PR bodies. The readiness verdict does not become live-proven until slice 2, when the collect EFFECT captures a real snapshot from the canonical dev cluster and its output is compared against the bash verifiers it is meant to retire. Claiming a live-readback proof class here would be exactly the false-Done this ticket exists to prevent.So these are the hermetic
gh api contentsprobes the gate explicitly accepts for compute-only tickets (OMN-14051 guard). They read the deployed-source surface at the pinned product commits — never a receipt or contract authored by this ticket's own PRs, which is the "greps text you wrote and can never go RED" pattern the gate rejects.What goes RED
def handle(+request: ModelStagingReadinessRequestat omnimarket3096f242! grep -qE 'ModelEventEnvelope|ModelHandlerOutput'purity: purein the node contractinstance_class: kafka.m5.largeat omninode_infra9b919595t3.small; live readback said otherwise)sha256:6a433490…! gh api …/scripts/required-configmap-vars.txtNet-new evidence entries only; the two autobound entries are untouched.
Summary by CodeRabbit