Skip to content

evidence(OMN-12661): add clean on-off experiment receipts - #2293

Merged
jonahgabriel merged 3 commits into
devfrom
jonah/omn-12661-clean-evidence
Jun 7, 2026
Merged

jonahgabriel merged 3 commits into
devfrom
jonah/omn-12661-clean-evidence

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Jun 7, 2026 •

Copy link
Copy Markdown
Contributor

Evidence-Ticket: OMN-12661

Summary

Verification

  • uv run pre-commit run yamlfmt --files contracts/OMN-12661.yaml drift/dod_receipts/OMN-12661/*/command.yaml
  • uv run python scripts/lint_contract_check_values.py contracts/OMN-12661.yaml
  • uv run python scripts/check_dod_evidence_present.py contracts/OMN-12661.yaml
  • local validator_occ_merge_eligibility for omnimarket PR evidence(OMN-11011): bind package migration classification #1072 -> eligible

Downstream

Summary by CodeRabbit

  • Tests

    • Added verification records for the ON-vs-OFF experiment validation, including test execution results and deployment evidence checks.
  • Chores

    • Added contract definition and configuration files for experiment evidence tracking and validation.

@coderabbitai

coderabbitai Bot commented Jun 7, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds a new evidence bundle contract for ticket OMN-12661 (WS-F bounded ON-vs-OFF experiment) along with five supporting evidence receipt files documenting test execution, deployment verification, PR state checks, cost analysis, and proof classification outcomes, all verified with PASS status.

Changes

Evidence Bundle for ON-vs-OFF Experiment (OMN-12661)

Layer / File(s) Summary
Contract definition and evidence references
contracts/OMN-12661.yaml
Schema version, ticket metadata, interface configuration, and emergency_bypass disabled. Defines five dod_evidence items each with grep-based shell checks to assert status: PASS in receipt command.yaml files under $CONTRACT_REPO_DIR/drift/dod_receipts/OMN-12661/, including an extra deploy-verified assertion for the deploy evidence.
Evidence receipt records
drift/dod_receipts/OMN-12661/dod-cost-delta-bundle/command.yaml, drift/dod_receipts/OMN-12661/dod-deploy-evidence/command.yaml, drift/dod_receipts/OMN-12661/dod-occ-pr-evidence/command.yaml, drift/dod_receipts/OMN-12661/dod-on-off-run-evidence/command.yaml, drift/dod_receipts/OMN-12661/dod-proof-classification/command.yaml
Five evidence records with schema version, ticket/evidence identifiers, command/check metadata, contract SHA-256 hash, PASS status, run timestamp, commit/branch/runner/verifier context, captured probe_stdout and actual_output text, exit code, PR number, and working directory. Cost-delta bundle documents pytest invocation; deploy evidence verifies importability of node_on_vs_off_experiment_compute; OCC PR evidence runs gh pr view query; ON-vs-OFF evidence documents 38/38 test passes; proof classification documents pytest execution.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Poem

🐰 A bundle of proof, tied up with care,
Five receipts showing all that's fair—
Tests pass, deploys shine, PR state gleams,
Evidence woven through OMN-12661 dreams! 📋✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding clean, valid evidence receipts for the OMN-12661 on-off experiment, replacing placeholder stubs from a superseded PR.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-12661-clean-evidence

Comment @coderabbitai help to get the list of available commands and usage tips.

@jonahgabriel
jonahgabriel added this pull request to the merge queue Jun 7, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@drift/dod_receipts/OMN-12661/dod-occ-pr-evidence/command.yaml`:
- Around line 1-23: The receipt in
drift/dod_receipts/OMN-12661/dod-occ-pr-evidence/command.yaml is inconsistent
with the contract contracts/OMN-12661.yaml: the contract describes
dod-occ-pr-evidence as verifying the OCC PR that carries the central contract
and PASS receipts, but this receipt's check_value/probe_command target
omnimarket PR `#1072` while pr_number is 2293; align them by either (A) updating
check_value and probe_command to target PR `#2293` (and ensure
probe_stdout/actual_output reflect that PR) or (B) update
contracts/OMN-12661.yaml (the dod-occ-pr-evidence description) to state it
verifies PR `#1072`; modify the unique fields check_value, probe_command,
pr_number, and any related probe_stdout/actual_output so all three consistently
reference the same PR number and evidence.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 92ebcc71-4806-4e95-837f-48e21da36c13

📥 Commits

Reviewing files that changed from the base of the PR and between bb13786 and 2c500c3.

📒 Files selected for processing (6)
  • contracts/OMN-12661.yaml
  • drift/dod_receipts/OMN-12661/dod-cost-delta-bundle/command.yaml
  • drift/dod_receipts/OMN-12661/dod-deploy-evidence/command.yaml
  • drift/dod_receipts/OMN-12661/dod-occ-pr-evidence/command.yaml
  • drift/dod_receipts/OMN-12661/dod-on-off-run-evidence/command.yaml
  • drift/dod_receipts/OMN-12661/dod-proof-classification/command.yaml

Comment thread drift/dod_receipts/OMN-12661/dod-occ-pr-evidence/command.yaml
Merged via the queue into dev with commit 91e08ab Jun 7, 2026
41 of 42 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-12661-clean-evidence branch June 7, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant