Skip to content

docs(OMN-20818): repin the onboarding guide to the multi-model script - #2702

Merged
jake-b-omni merged 5 commits into
devfrom
omn-20818-onboarding-repin
Oct 12, 2026
Merged

jake-b-omni merged 5 commits into
devfrom
omn-20818-onboarding-repin

Conversation

@jake-b-omni

@jake-b-omni jake-b-omni commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

plugins/onex/ONBOARDING.md downloads lab-onboarding.sh from a pinned commit and checks its sha256. The pin was 296be3fa (2026-10-06), which is older than #2637, so anyone following the guide got the old single-model script.

Test plan

  • Ran the documented curl + shasum -a 256 -c - against the new pin: lab-onboarding.sh: OK

Evidence-Ticket: OMN-20818
Evidence-Source: OCC#13742

🤖 Generated with Claude Code

ONBOARDING.md pinned lab-onboarding.sh at 296be3f (sha256 8a059b09...), which predates omniclaude#2637, so the documented download fetched the single-model script. The pin is now c15d2e3 with sha256 faedcec95e3a9da58d54ee279d3f7c75af0402ac6a81bd5921fa54c650c0d2d4. Only the URL and checksum lines changed; the onex plugin is bumped to 2.4.61.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@onexbot-occ-writer

Copy link
Copy Markdown
Contributor

OCC autobind did not mint a companion for this PR: no changed-file candidate could be proven RED against the merge base, and emitting a PR-existence probe instead would be non-falsifiable evidence (OMN-15247). Hand-authored evidence is required.

Considered 3 changed file(s):

  • plugins/onex-dev-marketplace/.claude-plugin/marketplace.json: no candidate grammar reads this file type (only Python declarations, uv.lock lines, release-artefact lines and contract-pin lines are proposed)
  • plugins/onex/.claude-plugin/plugin.json: no candidate grammar reads this file type (only Python declarations, uv.lock lines, release-artefact lines and contract-pin lines are proposed)
  • plugins/onex/ONBOARDING.md: no candidate grammar reads this file type (only Python declarations, uv.lock lines, release-artefact lines and contract-pin lines are proposed)

This verdict is occ-autobind's, derived from this head and its merge base alone. A second producer, the occ-companion-effect backstop, consumes its own command on the lab bus and may still mint a companion under the generic binding. Whether it does depends on that consumer running, not on this diff (OMN-18876).

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Hostile Reviewer — PASSED

Blocking findings (critical/error): 0
Total findings: 9
Models succeeded: correctness,security-contracts

Findings

Severity Location Finding
warning tests/skills/test_onboarding_script_pin_omn20818.py:58-65:1 ai-reviewer:correctness:completeness AC12 test does not verify the pinned commit is the multi-model script's commit | The test test_pin_is_not_older_than_the_multi_model_script only checks that the pinned commit is a descendant of MULTI_MODEL_SCRIPT_COMMIT (c15d2e3). This means any commit after c15d2e3 would pass, even if the script at that later commit is different from the multi-model script. The AC12 statement says the pin should be 'at the multi-model script's commit (c15d2e3, omniclaude#2637) or a later one', but the test does not
warning tests/skills/test_onboarding_script_pin_omn20818.py:28:1 ai-reviewer:correctness:correctness Test regex for sha256 digest is fragile and may not match the actual format | The regex PIN_SHA256 = re.compile(r"echo \"([0-9a-f]{64}) lab-onboarding\.sh\"") expects exactly two spaces between the digest and the filename. However, the shasum -a 256 -c command in the ONBOARDING.md uses a specific format that may vary. If the actual echo statement in the guide uses different spacing or a different format, the test would fail to find the digest. The test should be more robust to whitespace variations or
info tests/skills/test_onboarding_script_pin_omn20818.py:34-45:1 ai-reviewer:correctness:testing Test does not handle the case where the guide has no pinned URL or digest | The helper functions _pinned_commit() and _pinned_digest() use assert len(urls) == 1 and assert len(digests) == 1, which will fail with an AssertionError if the guide has no pinned URL or digest. This is acceptable for a test that expects exactly one pin, but the error message could be more informative. More importantly, if the guide is modified to have zero or multiple pins, the test will fail with a generic assertion error
info contracts/OMN-20818.yaml:50-53:1 ai-reviewer:correctness:completeness AC12 does not specify what happens if the pinned commit is deleted or unreachable | The AC12 statement says the pin should be 'at the multi-model script's commit (c15d2e3, omniclaude#2637) or a later one', but does not specify behavior if the pinned commit is later deleted from the repository (e.g., via force-push or branch cleanup). The test _require_commit skips if the commit is not in the checkout's history, but this does not address the case where the commit is unreachable from the main branch. The A
hint tests/skills/test_onboarding_script_pin_omn20818.py:34-45:1 ai-reviewer:correctness:performance Test reads the guide file multiple times | The helper functions _pinned_commit() and _pinned_digest() each call GUIDE.read_text(encoding="utf-8"), which means the guide file is read twice for each test that uses both helpers. This is not a performance issue for a small file, but it is inefficient. The file could be read once and cached. | Evidence: def _pinned_commit() -> str: urls = PIN_URL.findall(GUIDE.read_text(encoding="utf-8")) ... def _pinned_digest() -> str: digests = PIN_SHA256.fi
error plugins/onex/ONBOARDING.md:1 ai-reviewer:security-contracts:security Remote script execution via unauthenticated curl and bash | The onboarding guide instructs users to curl a script from raw.githubusercontent.com and immediately execute it with bash. Although a SHA256 check is performed, the user is still executing arbitrary code from a remote source. The SHA256 check mitigates against MITM or supply-chain attacks on the raw content, but it does not mitigate against a compromised repository or a malicious commit. More critically, the test `test_pinned_digest_is_the_script_a
warning tests/skills/test_onboarding_script_pin_omn20818.py:1 ai-reviewer:security-contracts:correctness Test does not verify remote content matches local git object | The test test_pinned_digest_is_the_script_at_the_pinned_commit uses git show to retrieve the script content from the local git repository and compares its SHA256 to the digest in the onboarding guide. However, it does not verify that the content served by the raw.githubusercontent.com URL matches the local git object. If the remote repository is compromised or if the URL is incorrect, the test will pass even though the user would download an
warning tests/skills/test_onboarding_script_pin_omn20818.py:1 ai-reviewer:security-contracts:feasibility Test depends on local git history, which may not contain the pinned commit | The test uses git cat-file and git merge-base to verify that the pinned commit is in the local git history and that it is an ancestor of the multi-model script commit. However, if the local checkout does not contain the full history (e.g., a shallow clone), the test will skip. This means the test may not run in all environments, reducing its effectiveness. The test should either ensure that the required commits are present or p
info tests/skills/test_onboarding_script_pin_omn20818.py:1 ai-reviewer:security-contracts:completeness No test for the case where the pinned commit is the same as the multi-model script commit | The test test_pin_is_not_older_than_the_multi_model_script verifies that the pinned commit is an ancestor of the multi-model script commit. However, it does not explicitly test the case where the pinned commit is the same as the multi-model script commit. While this is technically covered by the merge-base --is-ancestor check (a commit is an ancestor of itself), an explicit test would improve clarity and maintain

Gate semantics (pilot phase)

Verdict Meaning Blocks merge?
passed No critical/error findings No
blocked CRITICAL or ERROR findings found Yes
degraded Reviewer ran but every model failed mid-run No (pilot)
infra_error cli_review failed to run at all (install/crash) after 2 attempts Yes
unavailable_fork_isolation Fork PR — the LAN-only model is unreachable and the fleet is off limits Yes

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524)

jake-b-omni and others added 4 commits October 12, 2026 10:11
Adds tests/skills/test_onboarding_script_pin_omn20818.py (3 tests) and adds AC12 plus a dod_evidence item binding it to contracts/OMN-20818.yaml.
AC12: the guide pins lab-onboarding.sh at commit c15d2e3 (omniclaude#2637) or a later one, with the sha256 of the script at the pinned commit.
Why: receipt-gate.yml (pinned in call-repo-evidence-gate.yml) refuses a bound test with no test-side change in the PR; omniclaude#2702 had none.
Observed: the 3 tests pass at the PR head; with the old pin (296be3f) restored, test_pin_is_not_older_than_the_multi_model_script fails and the other 2 pass.
The test skips when the pinned commit is not in the checkout's history.
repo-evidence / dod-verify and test_every_repo_contract_binds_every_criterion both refused the contract: AC1-AC11 had no binds_ac. A new dod_evidence item runs the #2637 onboarding tests (test_lab_onboarding_script_omn20150.py, test_lab_onboarding_proven_set_omn20381.py) and binds AC1-AC11. The merged #2637 item and the AC12 item are unchanged. Providers in these tests are stubs, so AC9's live per-model delegation is still owed by a real run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Repo omniclaude, PR #2702, ticket OMN-20818, head 0cc6b4a.
Check repo-evidence / dod-verify failed with: "OMN-20818 [dod-omn20818-ac1-11-onboarding-tests]: bound test also passes at the merge base: the control did not fail (always-pass)". The item binds AC1-AC11 to the two onboarding script test files, which test a script that merged earlier in omniclaude#2637, so they pass at the merge base.
Change: contracts/OMN-20818.yaml only. The check of dod-omn20818-ac1-11-onboarding-tests now also runs tests/skills/test_onboarding_script_pin_omn20818.py, and the item description says so: the evidence holds only while the onboarding guide pins the multi-model script and not the single-model one it pinned before.
Measured locally by running the item's command: at this head it exits 0 (95 passed). With plugins/onex/ONBOARDING.md restored from origin/dev (the guide's earlier pin) it exits 1 (1 failed, 94 passed). dod-omn20818-ac12-onboarding-pin exits 0 at the head (3 passed) and 1 with the earlier guide (1 failed, 2 passed).
No code, test or gate changed.
@jake-b-omni
jake-b-omni merged commit eb7cca2 into dev Oct 12, 2026
148 checks passed
@jake-b-omni
jake-b-omni deleted the omn-20818-onboarding-repin branch October 12, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants