fix(verify): -rs replaced pytest's -rfE and stopped listing failures - #4
Conversation
Found by running the new instrument on a real red, which is the only way this
was ever going to show up.
The previous commit added "print EVERY pytest failure, not the last 12 lines",
and in the same breath passed `-rs` to get skip reasons. pytest's default is
`-rfE`; passing `-r` REPLACES that set rather than adding to it. So FAILED and
ERROR lines left the short summary, and `run_pytest` — which harvests failures
from exactly those lines — found none.
Observed, on a tree with one genuine failure:
- 1 pytest failure(s)/error(s)
pytest failures (0):
An instrument reporting less, introduced by the change that existed to make it
report more. Same defect class as the one this repo is named for, shipped inside
its own countermeasure.
`-rfEs` restores failures and errors alongside skip reasons. The selftest now
asserts on the flag itself, because the symptom appears only on a red run: a
green suite has no FAILED lines to lose, so nothing local would ever catch it.
Deliberate break -> revert: putting `-rs` back fails the selftest with "-rs omits
'f': failures would not be listed"; reverted byte-identical (`cmp`).
Verified: 330 passed / 0 failed / 0 of 24 skipped / 81 of 81 selftests / 5 of 5
gates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 24 minutes Limit details: You’ve used the included review currently available. Your 64 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Comment |
…wanted, plus .json (#110) Maint Coverage Guard failed the moment it was unheld, on both runs (#3 scheduled, #4 manual), with: error: Coverage verification could not find required coverage artifacts on a successful Gate run. warn: Unable to locate a recent successful Gate workflow run with required coverage artifacts: gate-coverage-trend, gate-coverage-trend-history, gate-coverage. maint-coverage-guard.yml downloads exactly three artifact NAMES. The Gate produced the first two under those names and the third as "gate-coverage.json", so the guard probed ten successful runs, found two of three every time, and reported no usable run at all. Measured rather than inferred: Gate run #241 (main, success) uploaded gate-coverage-summary.md, gate-coverage.json, gate-summary.md, gate-coverage-3.13-1, gate-coverage-trend-history, gate-coverage-trend, gate-coverage-summary, gate-coverage-3.12-1 -- trend and trend-history present, "gate-coverage" absent. An artifact NAME and the FILE inside it are different things, and this step conflated them. Renaming the artifact is safe and was checked, not assumed: "gate-coverage.json" as an artifact name appears nowhere else in this repo, and the only download-artifact steps in any workflow here are the guard's three. The file inside is still gate-coverage.json, so anything reading contents is unaffected. A paired literal in two files that disagreed -- this repo's signature defect -- and it stayed invisible for as long as the guard was held. That is the second thing the holds were concealing, after PR #90's six F821s. pr-00-gate.yml is create_only in the Workflows sync manifest with Orchestrator absent from overwrite_repos, so this local fix persists. The same mismatch likely affects every consumer whose Gate emits coverage, which is worth raising upstream separately. 448 collected, ruff and black clean, test_ci_gate_config 12 passed. Co-authored-by: Tim Stranske <tim@stranskemo.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Follow-up to #3, found by pointing the new instrument at a real red.
#3 added "print EVERY pytest failure, not the last 12 lines" — and in the same breath passed
-rsto get skip reasons. pytest's default is-rfE, and passing-rreplaces that set rather than adding to it. SoFAILED/ERRORlines left the short summary, andrun_pytest— which harvests failures from exactly those lines — found none.Observed on a tree with one genuine failure:
An instrument reporting less, introduced by the change that existed to make it report more. Same defect class this repo is named for, shipped inside its own countermeasure.
-rfEsrestores failures and errors alongside skip reasons.The selftest now asserts on the flag itself rather than on behaviour, because the symptom only appears on a red run: a green suite has no
FAILEDlines to lose, so no amount of local verification would have caught it. Deliberate break → revert: restoring-rsfails with-rs omits 'f': failures would not be listed; reverted byte-identical.Verified: 330 passed / 0 failed / 0 of 24 skipped / 81 of 81 selftests / 5 of 5 gates.
🤖 Generated with Claude Code