From bdda499650d515065d6b95e0af763e60a5ec93d5 Mon Sep 17 00:00:00 2001 From: Brian Searls Date: Tue, 12 May 2026 21:25:01 -0400 Subject: [PATCH 1/3] WIP: T-WAD Slice 6: slow_test_exemptions_dissolved (test cost-dim cuts) --- .../slow-test-residual-sweep-2026-05-13.md | 24 +++++++++++++++++++ scripts/test-node-wall-clock-ratchet.jsonl | 1 - 2 files changed, 24 insertions(+), 1 deletion(-) create mode 100644 docs/debt/slow-test-residual-sweep-2026-05-13.md diff --git a/docs/debt/slow-test-residual-sweep-2026-05-13.md b/docs/debt/slow-test-residual-sweep-2026-05-13.md new file mode 100644 index 00000000000..cf029c22256 --- /dev/null +++ b/docs/debt/slow-test-residual-sweep-2026-05-13.md @@ -0,0 +1,24 @@ +# Slow-Test Residual Sweep 2026-05-13 + +**Scope:** D1 residual paydown after the `TestNodeCostDimension` + JSONL +manifest bridge landed. This is not a new #102 closure claim: the ratchet still +reads `scripts/test-node-wall-clock-ratchet.jsonl` directly until policy is +projected from modeled timing facts. + +**Name audit:** `RUSTC_BOOTSTRAP=1 cargo test -p v3-compiler -- --list` +confirmed all 83 manifest names existed in the current v3 test list before this +cut. No stale-name rows were found. + +**Timing evidence:** `ctrl-build -- env RUSTC_BOOTSTRAP=1 cargo test -p +v3-compiler -- -Z unstable-options --report-time` captured 391 lib-test timing +rows before stopping on a pre-existing remote-environment helper lookup failure +for `gunbc_execute_command_bootstrap`. The timeout ratchet parser accepted that +partial log and reported no unexpected over-budget tests. + +**Retired warn row:** `bootstrap::tests::kernel_bool_path_a_attaches_diagnostic_when_boolean_algebra_unresolvable` +measured **16 ms**, below the 2000 ms Phase-0 budget. Its warn-policy row was +removed from `scripts/test-node-wall-clock-ratchet.jsonl`. + +**Rows intentionally left in place:** integration-test manifest rows were not +re-timed by this partial run. Seven lib-test manifest rows still exceeded 2000 +ms in the captured log and remain warn-policy backlog. diff --git a/scripts/test-node-wall-clock-ratchet.jsonl b/scripts/test-node-wall-clock-ratchet.jsonl index a1b526f0d61..8773f175ec5 100644 --- a/scripts/test-node-wall-clock-ratchet.jsonl +++ b/scripts/test-node-wall-clock-ratchet.jsonl @@ -46,7 +46,6 @@ {"test":"cost_lens_symbolic_consumer_test::recursive_countdown_pins_symbolic_cost_linear_and_sizevar_on_fixture","policy":"warn"} {"test":"cost_lens_symbolic_consumer_test::e_p_sub_value_relation_per_call_landed_cost_lens_routes_through_per_call_pattern_query","policy":"warn"} {"test":"e6_g1a_option3_static_lens_test::e6_g1a_option3_static_lens_mini_report_executes_without_reflection_imports","policy":"warn"} -{"test":"bootstrap::tests::kernel_bool_path_a_attaches_diagnostic_when_boolean_algebra_unresolvable","policy":"warn"} {"test":"lens_behavioral_parity_demonstration_test::r3_gate_73_demonstrates_complexity_certainty_is_proven","policy":"warn"} {"test":"sg2_parse_authority_test::parse_surface_dag_compiles_cleanly_for_regen_parse","policy":"warn"} {"test":"sg2c1_parse_tables_authority_test::parse_tables_generated_module_matches_checked_in_snapshot","policy":"warn"} From ac7d0f06924e4d209ccda0c30b63f0c20f2615a2 Mon Sep 17 00:00:00 2001 From: Brian Searls Date: Wed, 13 May 2026 01:25:46 +0000 Subject: [PATCH 2/3] docs: align slow-test residual sweep ledger --- ROADMAP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 911dfdd01dd..b4115e72de4 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -435,7 +435,7 @@ Tracked debt not dispatched with the 2026-04-21 lane wave (Lane 1e P3.0, SG-2c-4 - **Lane E-I — port `SubValueRelation` + inductive fields + cost algebra (M)**: port Tier 1 (structural) + Tier 2 (lattice) + Tier 3 (`CostBound` + master theorem). Requires E-T, E-C. Pre-flight: verify v3 lowers `CostBound`'s self-referential-through-`List` shape. Acceptance: carrier types ported with round-trip tests; no lens row moves on E-I alone because consumer-visible facts still wait for E-P producer/consumer wiring. - **Lane E-P — per-call descent-evidence provenance (M)**: producer coverage gate #76 receipt is landed as `v3_compiler::dag::per_call_descent_evidence` for the current ratcheted lowered callable surface (live `TransformTarget::Callable` edges; per-argument evidence vector; arithmetic, structural sub-value, preserved-value, and fail-closed unknown/absent cases; `TransformNode` stays unwidened). v2 stores this class of fact on `ExprCall.descent_evidence` (`src/v2/00_core.dag:199`); v3 derives it off-substrate for now. Requires E-T, E-C, E-I. Remaining acceptance for the broader lane: cost lens consumption and same-source v2/v3 cementing beyond the already-wired complexity recurrence consumer. - **Lane E-M — `MethodSemantics` port-or-subsume (S–M)**: **closed via M-b structural subsumption.** Transitive facts (`CollectionSizeEffect` / `CostShape` / `AlgebraFieldTemplate`) remain algebra-declaration facts; v3 method-like calls dispatch through `TransformTarget::{Callable, FieldProject, Operator}` and typed declaration/effect metadata. No `src/v3/std/method_semantics.dag`, no Rust mirror, and no `ExprMethodCall` attachment point. -- **CI ratchet architecture — PARTIAL (T-Receipts W1 second bundle)**: surfaced during the 2026-04-21 reflective analysis (`53b3110..ae8825a` range). Commits `37cd6128`, `4898983e`, `f84ed355` hardened the ratchet against CI-log instability, but `2d8396df` widened the exemption list. `scripts/slow-test-exemptions.txt` currently has **41** active non-comment exemptions (per `scripts/check-test-timeout.sh`’s sed-based count: non-empty lines with inline `#` comments stripped); `scripts/check-test-timeout.sh` meta-ratchet default `TEST_TIMEOUT_MAX_EXEMPTIONS` matches that count. **PR #1856 receipt:** the 41↔41 alignment was re-established by **removing a duplicate** `t_demo_fixture_test::t_demo_canonical_suites_are_runner_visible` exemption row (it had pushed `exempt_count` to **42** vs the default-**41** ratchet floor). **Landed partial:** classification audit [docs/debt/ci-ratchet-exemption-audit-2026-04-24.md](docs/debt/ci-ratchet-exemption-audit-2026-04-24.md) (2026-04-24 baseline) plus follow-up routing audit [docs/debt/ci-ratchet-exemption-audit-2026-05-02.md](docs/debt/ci-ratchet-exemption-audit-2026-05-02.md) (bounded scope — not a completed Phase 1 per-test timing pass). **Still open:** fresh CI-shaped per-test timing audit, stale exemption deletion if any entries measure under 2s, and per-exempt duration budgets once fresh timings exist. Dispatch brief: `docs/briefs/ci-ratchet-architecture-audit.md`. +- **CI ratchet architecture — PARTIAL (T-Receipts W1 second bundle)**: surfaced during the 2026-04-21 reflective analysis (`53b3110..ae8825a` range). Commits `37cd6128`, `4898983e`, `f84ed355` hardened the ratchet against CI-log instability, but `2d8396df` widened the original exemption list. **Landed partial:** classification audit [docs/debt/ci-ratchet-exemption-audit-2026-04-24.md](docs/debt/ci-ratchet-exemption-audit-2026-04-24.md) (2026-04-24 baseline) plus follow-up routing audit [docs/debt/ci-ratchet-exemption-audit-2026-05-02.md](docs/debt/ci-ratchet-exemption-audit-2026-05-02.md) (bounded scope — not a completed Phase 1 per-test timing pass). **Manifest bridge landed:** `scripts/slow-test-exemptions.txt` and `TEST_TIMEOUT_MAX_EXEMPTIONS` are retired; `scripts/check-test-timeout.sh` now reads structured warn policy from `scripts/test-node-wall-clock-ratchet.jsonl` pending #102 projection from modeled `TestNodeCostDimension` timing facts. **D1 residual sweep 2026-05-13:** [docs/debt/slow-test-residual-sweep-2026-05-13.md](docs/debt/slow-test-residual-sweep-2026-05-13.md) verified manifest names against current libtest list and retired one now-under-budget warn row. **Still open:** fresh CI-shaped per-test timing audits, stale warn-row deletion when entries measure under 2s, and per-node duration budgets once fresh timings exist. Dispatch brief: `docs/briefs/ci-ratchet-architecture-audit.md`. - **`IntegrationRsScan` / `integration_rs_active_line_contains` in `src/v3/compiler/tests/integration/common/mod.rs` — PARTIAL (T-Receipts; char/byte-char literal widening **PR #1856**).** Historical context (#686 / #705): the scanner was narrow enough that authors on its scan path avoided `b'\\'` / `b'"'` in favor of numeric byte constants. **Landed in #1856:** explicit `ByteCharLiteral` / `CharLiteral` states with escape-aware skipping plus lifetime vs `'…'` disambiguation and focused unit tests — byte-char and character literals no longer false-green Band-C `#[path` / `mod` needles. (Earlier ROADMAP text cited **PR #1791** for partial CI / row hygiene only; **#1856** is the live implementation receipt for the widening in `common/mod.rs`.) **Residual:** raw strings (`r#"…"#`) and byte strings (`b"…"`, `br#"…"#`) are still intentionally loud via the existing opener probe (panic in `Code`). Dissolution for that remainder: widen `IntegrationRsScan` for those forms **or** replace this scan with a structural reader. Owner: unassigned; P2 Boundary Discipline / T-Receipts bundle. - **Stale-receipt sweep in `docs/briefs/` — PARTIAL (T-Receipts W1).** The cited `DeclarationLookup` cleanup brief now carries a historical-receipt banner stating the lane is closed by SG-4b-1-fix and that `DeclarationLookup` / `find_declaration` are gone from `src/v3/lenses/variant_payload.dag`; its stale INVARIANTS refs were also updated to stable anchors. Remaining broad `docs/briefs/` sweep stays open for later post-wave stale claims. **Include:** `docs/design-pure-bootstrap.md`'s hand-maintained count ("78 hand-maintained .rs files") was stale vs. the SG-0 census — **CLOSED in this R1 PR**: all five occurrences (lines 7, 49, 82, 105, 242 of the design doc) now point at the live census, reframed as deltas against the illustrative 78-file baseline, or scoped to the non-test + TESTING residual framing. No frozen absolute count remains. - **Compiler–`std/` consolidation program — specific migrations**: end-state defined in [docs/thesis/compiler-std-consolidation.md](docs/thesis/compiler-std-consolidation.md). Each of the migrations below is a standalone lane; together they collapse the compiler-specific type surface toward the positive definition (pipeline + regen + lens-specific return-type carriers + accessor). **Ratchet:** count of `type` declarations in `src/v3/compiler/*.dag` AND `src/v3/lenses/*.dag` that are NOT in the positive-def set AND NOT exempted → 0. Positive-def: pipeline/regen types + lens-API return carriers (`Origin`, `UnusedParameter`, `ComplexitySummary`, `VariantPayloadShapeLookup` 3-variant, `TemplateArgumentBinding` semantic carrier, etc.). Exempted: `parse_tables.dag` (7 types, pending SG-2c-proper per-row classification). In-ratchet: **workaround-shaped infer-helper coproducts with named dissolution triggers (`TemplateArgumentsMatch`, `TemplateArgumentCursor`, `NormalizedInstantiationArgs`)** — Lookup-pattern carriers on lens surfaces import `v3.std.lookup::Lookup` (`complexity`, `cost`, `infer_helpers`). Primary count: 3. See thesis doc table for per-file disposition. From 1d973c0fe720ffecfc8f26d285fb5d37cd67d8d9 Mon Sep 17 00:00:00 2001 From: Brian Searls Date: Wed, 13 May 2026 00:17:25 -0400 Subject: [PATCH 3/3] WIP: T-WAD Slice 6: slow_test_exemptions_dissolved (test cost-dim cuts) --- src/v3/compiler/tests/HANDOFF_PHASE0.md | 61 ++++++++++--------------- 1 file changed, 24 insertions(+), 37 deletions(-) diff --git a/src/v3/compiler/tests/HANDOFF_PHASE0.md b/src/v3/compiler/tests/HANDOFF_PHASE0.md index 0b4020ada62..882c2f34b9d 100644 --- a/src/v3/compiler/tests/HANDOFF_PHASE0.md +++ b/src/v3/compiler/tests/HANDOFF_PHASE0.md @@ -6,17 +6,18 @@ migrations (m0_acceptance, m1_substrate, lens tests, thesis tests) are explicitly out of scope and move under their own workers. **Scope clarification — this PR is the report-only baseline.** The -acceptance-criterion phrasing "CI fails on any test >2s" is split +acceptance-criterion phrasing "CI fails on any test >2s" was split across two PRs by design: - **This PR (quick-crab-901)** lands the ratchet script, the exemption - file, the CI step, and the directory reorg. The CI step runs the + manifest, the CI step, and the directory reorg. The CI step runs the ratchet and publishes violations in the log, but is `continue-on-error: true`, so main still merges with arbitrarily slow per-test timings. This is the baseline-publication PR. -- **Follow-up PR** reads the first full CI run's report, populates - `scripts/slow-test-exemptions.txt` with one line per currently-slow - test + a reason, and removes `continue-on-error: true` from the CI - step. That is the PR that makes the ratchet binding. +- **Follow-up PRs** made the ratchet binding and later replaced the + retired free-form `scripts/slow-test-exemptions.txt` table with the + structured JSONL bridge at `scripts/test-node-wall-clock-ratchet.jsonl`. + The bridge is still an interim side manifest until #102 projects warn + policy from modeled `TestNodeCostDimension` timing facts. Splitting this way keeps the Phase 0 diff narrow (measurement + org only) and keeps the paydown decisions in a second diff with an @@ -40,21 +41,16 @@ single contradiction. quality gates inside the 25-minute v3 job budget. A local-fallback mode (invoke the script with no args) does run `cargo test` itself for users without a captured log. -- New: `scripts/slow-test-exemptions.txt` — paydown list. Tests named - here are tolerated with a `::warning::` line; every other slow test - fails the ratchet. Empty on landing. -- New CI step in `.github/workflows/ci.yml` (`v3` job, above the - clippy gate): `v3 tests (per-test 2s ratchet, report-only baseline)`. - Marked `continue-on-error: true` **and** `if: always()` — the first - keeps the job green while the exemption file is empty (baseline - publication); the second keeps the baseline visible on red runs too - (full-suite failure, 1200s wall-clock overflow), which is precisely - when the timing data is most diagnostic. `tee` captures the timing - log before the upstream budget check runs, so partial data survives. - A small guard in the step handles the degenerate case where the - upstream step errored before `cargo test` even ran, emitting a - `::notice::` instead of trying to parse a missing log. The follow-up - PR flips `continue-on-error` off to make the ratchet binding. +- Current warn-policy bridge: `scripts/test-node-wall-clock-ratchet.jsonl` + — one JSON object per libtest token, `{"test":"","policy":"warn"}`. + Tests named there are tolerated with a `::warning::` line; every other + slow test fails the ratchet. This replaced the retired + `scripts/slow-test-exemptions.txt` table and removed the old row-count + floor. +- CI step in `.github/workflows/ci.yml` (`v3` job): `v3 tests (per-test + 2s ratchet)`. It remains `if: always()` so partial timing data is still + reported on red runs, but the ratchet is now blocking: unknown tests + above the threshold fail the step. **Why `RUSTC_BOOTSTRAP=1`.** libtest's `--report-time` is still unstable on the 1.93 toolchain (tracking issue rust-lang/rust#64888); @@ -167,22 +163,13 @@ file, not in a narrow reorg diff): ## Next steps (unclaimed) — the binding half -1. **Populate `scripts/slow-test-exemptions.txt`.** Read the first - merged-main CI run's report-only baseline (or run - `scripts/check-test-timeout.sh` locally — no args invokes the - fallback that runs `cargo test` itself), and add each violating - test to the exemption file with a one-line reason that cites a - ROADMAP item or project memory. - **Entry format:** the token libtest emits as the second - whitespace-delimited field, e.g. `m1_5_testgen_test::heavy_case` - — no binary prefix, no `<…s>` timing. The exemption file header - documents the exact shape and an example line. -2. **Flip the CI step to blocking.** Remove - `continue-on-error: true` from the - `v3 tests (per-test 2s ratchet, report-only baseline)` step in - `.github/workflows/ci.yml` and rename to - `v3 tests (per-test 2s ratchet)` once the exemption file reflects - the real baseline. This is the PR that satisfies the Phase 0 +1. **Shrink `scripts/test-node-wall-clock-ratchet.jsonl`.** Reuse a + CI-shaped `--report-time` log, remove any `{"policy":"warn"}` row + whose test now measures at or below 2000 ms, and record the timing + evidence in the PR. +2. **Move warn policy into modeled facts.** Gate #102 remains open until + the ratchet consumes modeled `TestNodeCostDimension` timing facts + instead of the JSONL bridge. This is the dissolution path for the acceptance criterion "CI fails on any test >2s." 3. **Paydown (owned by sibling workers).** Each exempt test is a migration target — either speed it up (share bootstrap via