Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions contracts/OMN-19997.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,91 @@ dod_evidence:
check_value: |-
uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py -q
cwd: "${OMNI_HOME}/omninode_infra"
# OMN-19997 / 2026-09-29 -- BIND EACH CRITERION TO THE TEST THAT DECIDES IT.
#
# Autobind bound AC1, AC2 and AC3 to dod-OmniNode-ai-omninode_infra-pr-1742, whose
# whole check is `gh pr view 1742 --json number,state`. That succeeds for as long
# as the PR exists, so it discharged every criterion without measuring any of
# them (the OMN-18849 pattern). The test_passes item generated in the same run
# was left unbound, and its receipt's probe is the same PR read, not a test run.
#
# The ticket's own falsifiers name tests/scripts/test_c17_customer_surface_probe_omn19182.py.
# The tests landed in test_c17_customer_surface_probe_rate_limit_omn19997.py instead; run against
# the named file, each falsifier selects 0 of 68 tests and exits 5, so it can
# neither pass nor fail on the change. The items below run the file the tests
# are actually in, one selection per criterion, each shown to fail on the
# pre-change probe (a85caba4^) and pass on dev.
- id: "ac1-rate-limit-retry-and-token-fallback"
description: >-
AC1 -- fetch_source retries a 403/429 rate limit with growing backoff, then falls back to the workflow
token, and a read that succeeds on retry or on fallback grades normally. Three tests: the retry
recovers without the token, the exhausted retry falls back to the token, and the backoff grows between
attempts.
source: "generated"
checks:
- check_type: "test_passes"
check_value: "uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_rate_limit_retry_recovers_without_the_token
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_rate_limit_falls_back_to_the_workflow_token
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_backoff_grows_between_attempts
-q"
cwd: "${OMNI_HOME}/omninode_infra"
binds_ac:
- "AC1"
ac_bindings:
- label: "AC1"
criterion_hash: "c54778f67d7be9c3497669f7adffa66817e1a51b8f1cbf8df0d7e7aa39ed86e5"
accepted_by: "fc04d95f-0fcf-4a47-8ac2-26ee573a7cc3"
accepted_at: "2026-09-29T19:02:11Z"
supersedes_ac_binding:
- item: "dod-OmniNode-ai-omninode_infra-pr-1742"
label: "AC1"
reason: >-
Reads the PR's number and state. That answers whether a pull request exists, never whether fetch_source
retries a rate limit or falls back to the token.
- id: "ac2-public-visibility-check-fails-closed"
description: >-
AC2 -- with the anonymous read dropped, the guide-is-public proof is kept by a separate repository
visibility check, made before the token read, that fails closed when the repo is private.
source: "generated"
checks:
- check_type: "test_passes"
check_value: "uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py
-k public_visibility -q"
cwd: "${OMNI_HOME}/omninode_infra"
binds_ac:
- "AC2"
ac_bindings:
- label: "AC2"
criterion_hash: "dcbf9492d65800027ec9457c17423be3747aefb1c991efd3828ae3e95a85cab9"
accepted_by: "fc04d95f-0fcf-4a47-8ac2-26ee573a7cc3"
accepted_at: "2026-09-29T19:02:11Z"
supersedes_ac_binding:
- item: "dod-OmniNode-ai-omninode_infra-pr-1742"
label: "AC2"
reason: >-
Reads the PR's number and state, which says nothing about whether a private repo is refused.
- id: "ac3-unreadable-source-verdict-distinct-from-fail"
description: >-
AC3 -- an exhausted rate limit yields the UNREADABLE-SOURCE infrastructure verdict with its own
exit code and a summary naming the infrastructure cause, while a real clause failure stays FAIL
with exit 1.
source: "generated"
checks:
- check_type: "test_passes"
check_value: "uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_unreadable_source_verdict_when_the_limit_is_exhausted
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_unreadable_source_summary_names_the_infrastructure_cause
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_a_real_clause_failure_stays_fail_exit_one
-q"
cwd: "${OMNI_HOME}/omninode_infra"
binds_ac:
- "AC3"
ac_bindings:
- label: "AC3"
criterion_hash: "e3e8f962826bbf1a8815a1e595d12d67b5741c5c882abde9e2a6b10059521d2a"
accepted_by: "fc04d95f-0fcf-4a47-8ac2-26ee573a7cc3"
accepted_at: "2026-09-29T19:02:11Z"
supersedes_ac_binding:
- item: "dod-OmniNode-ai-omninode_infra-pr-1742"
label: "AC3"
reason: >-
Reads the PR's number and state, which cannot tell an UNREADABLE-SOURCE verdict from a FAIL.
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
schema_version: "1.0.0"
ticket_id: "OMN-19997"
evidence_item_id: "ac1-rate-limit-retry-and-token-fallback"
check_type: "test_passes"
check_value: "uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_rate_limit_retry_recovers_without_the_token
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_rate_limit_falls_back_to_the_workflow_token
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_backoff_grows_between_attempts
-q"
contract_sha256: "sha256:a4f35e55a93886bd930850f1428f34372ab2b534af2c3fb1da868c8c2d266e76"
contract_entry_sha256: "sha256:20f1814e59bba1ec4e84f881f692355a4c5f53f1b5aff7d812e010488512c20a"
status: PASS
run_timestamp: "2026-09-29T19:02:47Z"
commit_sha: "3a44a4956b98f293ea39c739b66086866e9473fe"
runner: "claude-opus-5-5-omn-19997"
verifier: "pytest-tests-scripts-omn19997"
probe_command: >-
uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_rate_limit_retry_recovers_without_the_token
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_rate_limit_falls_back_to_the_workflow_token
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_backoff_grows_between_attempts
-q # cwd OmniNode-ai/omninode_infra @ 3a44a4956b98f293ea39c739b66086866e9473fe (dev, post-merge of #1742)
# authority: repos/OmniNode-ai/omninode_infra/commits/3a44a4956b98f293ea39c739b66086866e9473fe
probe_stdout: |
... [100%]
3 passed in 0.12s
exit_code: 0
pr_number: 1742
actual_output: >-
PASS on merged dev. NEGATIVE CONTROL: the same command run with scripts/ci/c17_customer_surface_probe.py
restored to its pre-change revision (a85caba4^) gives 3 errors (exit 1), so the selection discriminates
the change rather than passing on anything.
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
schema_version: "1.0.0"
ticket_id: "OMN-19997"
evidence_item_id: "ac2-public-visibility-check-fails-closed"
check_type: "test_passes"
check_value: "uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py -k public_visibility
-q"
contract_sha256: "sha256:a4f35e55a93886bd930850f1428f34372ab2b534af2c3fb1da868c8c2d266e76"
contract_entry_sha256: "sha256:eb0c11c09f62bbad563234e95c0b7bb1daaa0005379262fd8154ab14de15e176"
status: PASS
run_timestamp: "2026-09-29T19:02:47Z"
commit_sha: "3a44a4956b98f293ea39c739b66086866e9473fe"
runner: "claude-opus-5-5-omn-19997"
verifier: "pytest-tests-scripts-omn19997"
probe_command: >-
uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py -k public_visibility
-q # cwd OmniNode-ai/omninode_infra @ 3a44a4956b98f293ea39c739b66086866e9473fe (dev, post-merge of #1742)
# authority: repos/OmniNode-ai/omninode_infra/commits/3a44a4956b98f293ea39c739b66086866e9473fe
probe_stdout: |
.. [100%]
2 passed, 8 deselected in 0.13s
exit_code: 0
pr_number: 1742
actual_output: >-
PASS on merged dev. NEGATIVE CONTROL: the same command run with scripts/ci/c17_customer_surface_probe.py
restored to its pre-change revision (a85caba4^) gives 2 errors (exit 1), so the selection discriminates
the change rather than passing on anything.
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
schema_version: "1.0.0"
ticket_id: "OMN-19997"
evidence_item_id: "ac3-unreadable-source-verdict-distinct-from-fail"
check_type: "test_passes"
check_value: "uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_unreadable_source_verdict_when_the_limit_is_exhausted
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_unreadable_source_summary_names_the_infrastructure_cause
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_a_real_clause_failure_stays_fail_exit_one
-q"
contract_sha256: "sha256:a4f35e55a93886bd930850f1428f34372ab2b534af2c3fb1da868c8c2d266e76"
contract_entry_sha256: "sha256:a2ddf1997079e792a78e3f4926a7f93f98325c0012a8157b3765b714d6bfb725"
status: PASS
run_timestamp: "2026-09-29T19:02:47Z"
commit_sha: "3a44a4956b98f293ea39c739b66086866e9473fe"
runner: "claude-opus-5-5-omn-19997"
verifier: "pytest-tests-scripts-omn19997"
probe_command: >-
uv run pytest tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_unreadable_source_verdict_when_the_limit_is_exhausted
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_unreadable_source_summary_names_the_infrastructure_cause
tests/scripts/test_c17_customer_surface_probe_rate_limit_omn19997.py::test_a_real_clause_failure_stays_fail_exit_one
-q # cwd OmniNode-ai/omninode_infra @ 3a44a4956b98f293ea39c739b66086866e9473fe (dev, post-merge of #1742)
# authority: repos/OmniNode-ai/omninode_infra/commits/3a44a4956b98f293ea39c739b66086866e9473fe
probe_stdout: |
... [100%]
3 passed in 0.14s
exit_code: 0
pr_number: 1742
actual_output: >-
PASS on merged dev. NEGATIVE CONTROL: the same command run with scripts/ci/c17_customer_surface_probe.py
restored to its pre-change revision (a85caba4^) gives 2 failed, 1 error (exit 1), so the selection discriminates
the change rather than passing on anything.
18 changes: 18 additions & 0 deletions drift/occ_bindings/OMN-19997/occ-self-bind-pr-11828/command.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
schema_version: "1.0.0"
ticket_id: "OMN-19997"
evidence_item_id: "occ-self-bind-pr-11828"
check_type: "command"
check_value: "gh pr view 11828 --repo OmniNode-ai/onex_change_control --json number,state,headRefName"
contract_sha256: "sha256:a4f35e55a93886bd930850f1428f34372ab2b534af2c3fb1da868c8c2d266e76"
status: PASS
run_timestamp: "2026-09-29T21:09:33Z"
commit_sha: "9237f01f0c8c668157a2533dc5c9ea6388e373fd"
runner: "occ-self-bind-mint"
verifier: "github-pr-readback"
probe_command: "gh pr view 11828 --repo OmniNode-ai/onex_change_control --json number,state,headRefName"
probe_stdout: |
{"headRefName":"jake/omn-19997-bind-ac-to-behaviour-proof","number":11828,"state":"OPEN"}
exit_code: 0
pr_number: 11828
branch: "jake/omn-19997-bind-ac-to-behaviour-proof"
Loading