Skip to content

fix: repair six stale documented commands and add a check so they stop rotting - #1635

Merged
justinchuby merged 1 commit into
mainfrom
fix-stale-invocations
Aug 21, 2026
Merged

justinchuby merged 1 commit into
mainfrom
fix-stale-invocations

Conversation

@justinchuby

Copy link
Copy Markdown
Owner

Answering "is everything broken fixed?" properly meant checking rather than
recalling, so I validated every cargo command written down in the repo against
cargo metadata. 262 commands; six named something that does not exist.

Two were my own fallout from the feature rename (#1629), which I missed because
I updated .github/skills/ and not .agents/skills/:

  • .agents/skills/profiling/SKILL.md still asked for bench-native,cuda.
  • crates/onnx-genai-capi/README.md still asked for --features cuda.

One is a shell script with exactly the two bugs I fixed in #1630, which I should
have found then by looking at the class of defect instead of the instance:

  • scripts/build_real_model.sh built -p onnx-genai --bin onnx-genai (that
    package ships only the diffusion bins) and called generate --model DIR (the
    model directory is positional).

Three predate all of this and are worse than stale -- they were never true:

  • docs/ep-plugin/EP_PLUGIN_EXPORT_TEST_PLAN.md built a plugin-export feature
    on onnx-runtime-ep-cpu and grepped for CreateEpApiFactories. What shipped
    is a separate onnx-runtime-ep-cpu-plugin cdylib exporting CreateEpFactories
    and ReleaseEpFactory. Verified by building it and reading nm -D.
  • docs/performance/CPU_MATMUL_ASSIGNMENT.md gave a repro recipe using a
    bench_prec binary with --native-threads / --ort-intra-threads. git log
    finds no commit that ever added or removed it. The recipe is removed rather
    than annotated, because a command that cannot run is not worth keeping; the
    section now says plainly that those numbers cannot be reproduced as written.
  • A fixture generator credited its canonical output to
    cargo run -p onnx-std --example convert_fixture. That example has never
    existed either.

The reason all six survived is that nothing ever checks a command that nothing
ever runs; the failure then looks like the reader's environment rather than the
line. scripts/check_documented_commands.py now validates the parts that can be
checked statically -- package, --bin, --example, --test, --bench,
--features -- and runs in the rust-quality lane. Dated docs and .squad
archives are skipped: they record what was run at the time.

Verification:

  • The check reports 262 commands, all resolving, on this branch.
  • It has teeth, by the same --self-test convention the dispatch-manifest lint
    uses: six cases, five that must be detected (bad feature, bad bin, unknown
    package, bad bench, bad dep/feature) and one valid command that must stay
    silent. Also confirmed by hand against the real tree, by breaking
    docs/build-features.md and the profiling skill and watching each get caught.
  • bash -n clean on the repaired script, and its new CLI form reaches model
    loading instead of being rejected by clap.
  • ci.yml parses as YAML.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com
Copilot-Session: 0190e2eb-abe4-451f-b36d-44a035a99b7e

…p rotting

Answering "is everything broken fixed?" properly meant checking rather than
recalling, so I validated every `cargo` command written down in the repo against
`cargo metadata`. 262 commands; six named something that does not exist.

Two were my own fallout from the feature rename (#1629), which I missed because
I updated `.github/skills/` and not `.agents/skills/`:

- `.agents/skills/profiling/SKILL.md` still asked for `bench-native,cuda`.
- `crates/onnx-genai-capi/README.md` still asked for `--features cuda`.

One is a shell script with exactly the two bugs I fixed in #1630, which I should
have found then by looking at the class of defect instead of the instance:

- `scripts/build_real_model.sh` built `-p onnx-genai --bin onnx-genai` (that
  package ships only the diffusion bins) and called `generate --model DIR` (the
  model directory is positional).

Three predate all of this and are worse than stale -- they were never true:

- `docs/ep-plugin/EP_PLUGIN_EXPORT_TEST_PLAN.md` built a `plugin-export` feature
  on `onnx-runtime-ep-cpu` and grepped for `CreateEpApiFactories`. What shipped
  is a separate `onnx-runtime-ep-cpu-plugin` cdylib exporting `CreateEpFactories`
  and `ReleaseEpFactory`. Verified by building it and reading `nm -D`.
- `docs/performance/CPU_MATMUL_ASSIGNMENT.md` gave a repro recipe using a
  `bench_prec` binary with `--native-threads` / `--ort-intra-threads`. `git log`
  finds no commit that ever added or removed it. The recipe is removed rather
  than annotated, because a command that cannot run is not worth keeping; the
  section now says plainly that those numbers cannot be reproduced as written.
- A fixture generator credited its canonical output to
  `cargo run -p onnx-std --example convert_fixture`. That example has never
  existed either.

The reason all six survived is that nothing ever checks a command that nothing
ever runs; the failure then looks like the reader's environment rather than the
line. `scripts/check_documented_commands.py` now validates the parts that can be
checked statically -- package, `--bin`, `--example`, `--test`, `--bench`,
`--features` -- and runs in the `rust-quality` lane. Dated docs and `.squad`
archives are skipped: they record what was run at the time.

Verification:
- The check reports 262 commands, all resolving, on this branch.
- It has teeth, by the same `--self-test` convention the dispatch-manifest lint
  uses: six cases, five that must be detected (bad feature, bad bin, unknown
  package, bad bench, bad `dep/feature`) and one valid command that must stay
  silent. Also confirmed by hand against the real tree, by breaking
  `docs/build-features.md` and the profiling skill and watching each get caught.
- `bash -n` clean on the repaired script, and its new CLI form reaches model
  loading instead of being rejected by clap.
- `ci.yml` parses as YAML.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0190e2eb-abe4-451f-b36d-44a035a99b7e
@justinchuby
justinchuby merged commit 44224bd into main Aug 21, 2026
3 checks passed
@justinchuby
justinchuby deleted the fix-stale-invocations branch August 21, 2026 06:15
@codecov

codecov Bot commented Aug 21, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.49%. Comparing base (f8eb8a3) to head (1b2c8a7).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1635      +/-   ##
==========================================
- Coverage   81.63%   81.49%   -0.15%     
==========================================
  Files         384      384              
  Lines      180249   180249              
  Branches   180249   180249              
==========================================
- Hits       147150   146885     -265     
- Misses      28155    28418     +263     
- Partials     4944     4946       +2     
Flag Coverage Δ
cli-ort-linux 82.60% <ø> (ø)
cli-ort-windows 82.20% <ø> (+0.09%) ⬆️
mlas 85.09% <ø> (ø)
offline 81.38% <ø> (-0.16%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

justinchuby added a commit that referenced this pull request Aug 21, 2026
Main's #1635 added a lint that resolves every documented `cargo` command
against the actual workspace, and it caught two real inconsistencies this
branch had left behind.

`crates/onnx-genai/src/bin/run_diffusion.rs` was deleted here in "Remove
legacy composite pipeline execution", because it drove the strategy/phase
composite runtime that `pipeline.workflow` replaces. Ten `scripts/*.py`
helpers still shelled out to `target/release/run_diffusion` and told the
reader to build it with `cargo build -p onnx-genai --bin run_diffusion`.
None of them could run: the binary they exec cannot be built. Delete them
with the runner they drive rather than leaving instructions that cannot be
followed -- exactly the rot the new lint exists to prevent. Nothing outside
the group references them; the only mentions are among themselves and in a
dated decisions archive.

Also point the workflow performance doc at `native-cuda`, since main's #1629
renamed the feature its example passed to `cargo test`.

`scripts/check_documented_commands.py` now exits clean, as do the other
quality-gate lints.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chuby <justinchuby@users.noreply.github.com>
justinchuby added a commit that referenced this pull request Aug 21, 2026
Main's #1635 added a lint that resolves every documented `cargo` command
against the actual workspace, and it caught two real inconsistencies this
branch had left behind.

`crates/onnx-genai/src/bin/run_diffusion.rs` was deleted here in "Remove
legacy composite pipeline execution", because it drove the strategy/phase
composite runtime that `pipeline.workflow` replaces. Ten `scripts/*.py`
helpers still shelled out to `target/release/run_diffusion` and told the
reader to build it with `cargo build -p onnx-genai --bin run_diffusion`.
None of them could run: the binary they exec cannot be built. Delete them
with the runner they drive rather than leaving instructions that cannot be
followed -- exactly the rot the new lint exists to prevent. Nothing outside
the group references them; the only mentions are among themselves and in a
dated decisions archive.

Also point the workflow performance doc at `native-cuda`, since main's #1629
renamed the feature its example passed to `cargo test`.

`scripts/check_documented_commands.py` now exits clean, as do the other
quality-gate lints.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chuby <justinchuby@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant