Skip to content

Design sketch: affected-set precompute pruning — guard precompute_whole_tree_published_mock_keys on scoped diff (docs only) - #5994

Merged
briansrls merged 17 commits into
mainfrom
docs/affected-set-precompute-pruning-sketch
Jun 30, 2026
Merged

briansrls merged 17 commits into
mainfrom
docs/affected-set-precompute-pruning-sketch

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

DESIGN SKETCH ONLY — no implementation. Returns to operator for sign-off before any code lands. Files docs/plans/affected-set-precompute-pruning.md only. Specifies: consumer fn + plug point (precompute_whole_tree_published_mock_keys at cli_run.rs:3509), diff_touches_published_mock_closure fn (module-level, no type-check/eval), input carrier (NodeArtifactProvenance), non-flaky structural None/Some witness + measured delta logged, soundness lemma (verified: output bounded to transitive closure of PublishedMockCase declarers, no whole-tree scan in resolve path), acceptance witness (real gunbc floor, 3 parts: structural+measured+control), fail-safe (intersection error → full precompute), composition with tidy-hawk-120 (#5959). End-user story: gunbc test on a small change re-precomputes and re-runs only the diff-affected slice instead of the whole ~537s / 8.7 GiB corpus.

…mplementation)

Spec for guarding precompute_whole_tree_published_mock_keys on a scoped diff.
Covers: consumer fn + plug point, diff_touches_published_mock_closure fn,
input carrier (NodeArtifactProvenance), non-flaky structural None/Some witness
+ measured delta logged, soundness lemma (verified against fn body: output
bounded to transitive closure of PublishedMockCase declarers, no whole-tree
scan in resolve path), acceptance witness (real gunbc floor, 3 parts),
fail-safe, and composition with tidy-hawk-120 (#5959).
Returns to stern-moth-225 → loyal-bee/operator for sign-off before any code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nd tri-state PrecomputeOutcome

Finding 1: §5 named NodeArtifactProvenance as the carrier — not in the v1 Rust
path. Actual carrier is line_ranges_by_file produced by floor_git_diff_range()
→ parse_unified_diff_line_ranges (cli_run.rs:3533-3536); collect_frontier_seeds
consumes it for execution-skip, not yields it. Corrected.

Finding 2: whole_tree_published_keys=None has two existing meanings (ran+empty at
line 3510; proposed skip). Indistinguishable without a carrier change. Fix: spec
now requires tri-state PrecomputeOutcome {Skipped, EmptyKeys, Keys(_)} so the
structural witness asserts outcome==Skipped unambiguously.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed both findings in commit aca7272:

Finding 1 (§5 carrier): Replaced with the correct v1 carrier: produced by (cli_run.rs:3533–3536). Clarified that consumes this map to produce for execution-skip; would be a second consumer of the same map.

Finding 2 (None ambiguity): Introduced a required tri-state into the spec. The structural witness now asserts (unambiguous — cannot arise from the ran-empty path). Existing / at line 3510 maps to /.

— sent from merry-fox-653

@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed both findings in commit aca7272:

Finding 1 (section 5 carrier): Replaced NodeArtifactProvenance with the correct v1 carrier: line_ranges_by_file (HashMap<String, Vec>) produced by floor_git_diff_range() -> parse_unified_diff_line_ranges (cli_run.rs:3533-3536). Clarified that collect_frontier_seeds_from_diff_line_ranges CONSUMES this map to produce NodeFrontierSeeds for execution-skip; diff_touches_published_mock_closure would be a second consumer of the same map. NodeArtifactProvenance is a .dag type not present in this v1 Rust path.

Finding 2 (None ambiguity): cli_run.rs:3510 confirms None already means "ran + empty keys". The proposed skip adds a third None path — indistinguishable. Fix: spec now requires a tri-state PrecomputeOutcome enum (Skipped / EmptyKeys / Keys(HashSet)). The structural witness asserts outcome == Skipped (unambiguous — cannot arise from the ran-empty path). Existing None/Some at line 3510 maps to EmptyKeys/Keys(_).

— sent from merry-fox-653

- Line 103 (§6 summary): 'structural None/Some' → 'PrecomputeOutcome::Skipped/Keys(_)'
- §9 acceptance criteria (a)(c): replace whole_tree_published_keys==None/Some(_)
  with outcome==PrecomputeOutcome::Skipped/Keys(_), matching §6 tri-state
- §7: replace 'Passing None' with 'PrecomputeOutcome::Skipped path'
- §10: 'None early-exit path' → 'PrecomputeOutcome::Skipped early-exit path'
- §3 code block: update guard sketch to show PrecomputeOutcome tri-state
  (Skipped / EmptyKeys / Keys) replacing the Option<HashSet> pattern

All Option/None/Some references that survived in doc now either explain the
OLD ambiguity (problem statement in §6 prerequisite, fine) or are in the
existing cli_run.rs code context (authoritative, not spec language).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed both findings in commit ccd4583:

Finding 1 (line 103 stale summary): "The structural None/Some is the executable green-by-execution proof" -> "The structural PrecomputeOutcome::Skipped / Keys(_) check is the executable green-by-execution proof."

Finding 2 (§9 acceptance criteria): Replaced whole_tree_published_keys==None/Some() with outcome==PrecomputeOutcome::Skipped / Keys() throughout §9(a) and §9(c), explicitly noting the None form is ambiguous (see §6 implementation prerequisite). Also updated §7 and §10 to use the tri-state language, and updated the §3 code sketch to show the full PrecomputeOutcome guard pattern (Skipped / EmptyKeys / Keys) replacing the Option placeholder.

Remaining None references in the doc are either (a) explaining the OLD ambiguity as the problem statement in §6's prerequisite, or (b) in quoted cli_run.rs code which is authoritative not spec language.

— sent from merry-fox-653

Brian Searls and others added 4 commits June 30, 2026 01:00
…ertion

Finding 1: mock-closure check alone is insufficient — a diff can miss the
PublishedMockCase closure yet still leave witnesses that use mock keys at eval
time (v1_interpreter.rs:1114-1121 fallback under-populates governed_services).
Fix: §3 guard requires BOTH !diff_touches_published_mock_closure AND
all_witnesses_will_skip; §7 names the fail-open path and the two safe
alternatives (conjunction guard, or M1 cache from tidy-hawk-120 §10).

Finding 2: in-closure control at §6 item 4 asserted Keys(_) specifically,
but declarers can legitimately produce EmptyKeys. Fix: assert
outcome != PrecomputeOutcome::Skipped (i.e. EmptyKeys | Keys(_)) instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a ROADMAP entry linking docs/plans/affected-set-precompute-pruning.md
so doc_graph_has_no_orphan_docs passes (CI was failing with Bool(false)).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Finding 1: §6 item 1 fixture was under-specified — "outside the mock
closure" alone is insufficient when the file is also in a witness
frontier. Tightened to require outside BOTH the mock closure AND every
witness's node frontier; explains why a test-only .dag edit can prevent
Skipped from being assertable.

Finding 2: §9(c) required Keys(_) but §6 item 4 allows EmptyKeys|Keys(_)
(!= Skipped). Aligned §9(c) to != Skipped (consistent with §6's
structural guarantee that precompute ran, not that it yielded keys).

Finding 3: M1 in tidy-hawk-120 is a resolve_entry_graph memo (~105s
Axis B), not a precompute-keys cache. Fixed §2 table row, §7 guard
description (removed CachedKeys/M1 alternative paragraph), §10 scope
boundary, and ROADMAP entry — all mis-scoped M1 as caching the precompute
result. The two lanes are orthogonal; M1 cuts resolve cost when the
corpus runs, this guard cuts precompute cost when all witnesses skip.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed all three findings in commit 49af5b8:

Finding 1 (§6 fixture under-specified): Changed §6 item 1 from 'outside the PublishedMockCase transitive closure' to 'outside BOTH the mock closure AND every witness's node frontier.' Added explicit note that a test-only .dag edit outside the closure is insufficient if it is imported by a witness — such a diff fires the node frontier, witnesses run, and Skipped becomes unassertable (the conjunction guard requires all_witnesses_will_skip).

Finding 2 (§6 vs §9(c) mismatch): Changed §9(c) from outcome == PrecomputeOutcome::Keys(_) to outcome != PrecomputeOutcome::Skipped (i.e. EmptyKeys | Keys(_)), consistent with §6 item 4. The structural guarantee is that the precompute was not skipped, not that it yielded non-empty keys. Added note that Keys(_) is not required at the structural bar.

Finding 3 (M1 mis-scope): Corrected all four sites: §2 table row (M1 is a within-walk resolve_entry_graph memo ~105s Axis B, not a precompute cache; marked as orthogonal); §7 (removed 'from M1 cache via tidy-hawk-120 #5959 §10, when available' from the guard description and deleted the 'Alternatively / CachedKeys from M1' paragraph); §10 (M1 memos resolve, not precompute keys; coordination point is about M1's assumptions, not caching); ROADMAP.md entry (changed 'Composes with tidy-hawk-120 M1 cache' to 'Orthogonal to tidy-hawk-120 M1 resolve memo').

— sent from merry-fox-653

… spec

Finding 1: §3 comment still said 'all_witnesses_skip OR M1 can serve keys'
— contradicts §7/§10/§2 which confirm M1 is a resolve memo (cannot serve
precompute keys). Removed the OR branch; condition is now AND-only.

Finding 2: all_witnesses_will_skip was referenced as a required conjunct
in §3/:44 with no spec. Added §4b with signature, algorithm (bulk
pre-flight of the existing per-row predicate at cli_run.rs:3721-3731),
tie to !current_entry_touches && !function_edited, cost analysis, and
fail-closed invariant (must replicate or over-approximate the inner loop
predicate, never under-approximate).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed both findings in commit e67e135:

Finding 1 (§3 M1 OR branch): Removed the stale 'OR when the M1 cache (tidy-hawk-120 §10) can serve keys' from the §3 soundness comment (line 38-39). The comment now reads: 'skip is only safe when zero witnesses will execute (all_witnesses_will_skip) AND the diff does not touch the mock closure.' The OR branch was a carry-forward from before §7/§10/§2 were corrected to say M1 is a resolve memo — it contradicted the fail-closed story.

Finding 2 (all_witnesses_will_skip unspecified): Added §4b with full spec:

  • Signature: fn all_witnesses_will_skip(rows: &[DiscoveryRow], frontier_seeds: &NodeFrontierSeeds) -> bool
  • Algorithm: bulk pre-flight of the existing per-row predicate at cli_run.rs:3721-3731 (!current_entry_touches && !function_edited) — replicates both conditions (entry_touches_frontier_seeds + edited_test_fns) across all rows
  • Explicit tie to the inner loop: must match or over-approximate cli_run.rs:3721-3731; if the inner loop gains a new conjunct, this must match or fall back to false
  • Cost: O(|rows| × |edited_test_fns|), no resolve or eval
  • Invariant: over-inclusive is safe; false-positive is unsound (would skip precompute while a witness observes missing keys via v1_interpreter.rs:1114-1121 fallback)

— sent from merry-fox-653

Brian Searls and others added 2 commits June 30, 2026 02:05
Finding 1: §4b cost model cited frontier_seeds.frontier_files which does
not exist. NodeFrontierSeeds (cli_run.rs:3275-3278) has only
overlapping_data_items, edited_test_fns, force_run_all. Rewrote
all_witnesses_will_skip algorithm to use the actual fields: O(1)
conservative check (!force_run_all && overlapping_data_items.is_empty()
&& edited_test_fns.is_empty()). This avoids resolve entirely: when
overlapping_data_items is empty, entry_frontier_nodes_from_seeds returns
empty for every entry and entry_touches_frontier_seeds returns false —
all witnesses skip. Explained why the per-entry touch check requires a
full InterpContext and cannot be used at the precompute plug point.

Finding 2: §9(a) only required "disjoint from the PublishedMockCase
closure" but the §3 guard also requires all_witnesses_will_skip. A diff
outside the closure that fires a witness frontier would fail Skipped.
Added "disjoint from BOTH closure AND every witness's node frontier
(produces empty NodeFrontierSeeds)" and explicit note that a diff
satisfying only one conjunct correctly runs precompute and fails (a).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed both findings in commit 149453d:

Finding 1 (frontier_files field does not exist): Rewrote §4b algorithm. NodeFrontierSeeds (cli_run.rs:3275-3278) has: overlapping_data_items: HashSet<(String, String)>, edited_test_fns: HashSet<(String, String)>, force_run_all: bool — no frontier_files. The entry-level touch check (entry_touches_frontier_seeds, cli_run.rs:3714) requires a full resolved InterpContext and cannot be called at the precompute plug point. The correct pre-flight is O(1) and purely structural:

fn all_witnesses_will_skip(frontier_seeds: &NodeFrontierSeeds) -> bool {
    !frontier_seeds.force_run_all
        && frontier_seeds.overlapping_data_items.is_empty()
        && frontier_seeds.edited_test_fns.is_empty()
}

When overlapping_data_items is empty, entry_frontier_nodes_from_seeds returns empty for every entry → entry_touches_frontier_seeds returns false → all witnesses skip. Conservative (over-approximate towards running precompute); no resolve required.

Finding 2 (§9(a) missing node-frontier condition): §9(a) now requires the diff to be disjoint from BOTH the PublishedMockCase closure AND every witness's node frontier (produces empty NodeFrontierSeeds: overlapping_data_items empty, edited_test_fns empty, force_run_all false). Added explicit note: a diff that satisfies only the closure conjunct but fires a witness frontier correctly runs precompute (outcome != Skipped) and would fail criterion (a).

— sent from merry-fox-653

@gunbai-bot

gunbai-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

The two cursor findings (#33543) are about content in the prior SHA (149453d) that no longer exists. The plan was completely rewritten at HEAD 1392bdd as a 6-step affected-set de-fork audit — there is no §4b, no all_witnesses_will_skip function, no LineRange type in the current document. The rewrite was triggered by the operator re-scope (full de-fork, not Rust-guard). New reviews on HEAD will see the current document. — sent from merry-fox-653

Brian Searls and others added 6 commits June 30, 2026 02:48
Three substantive fixes from cursor RC (#33547):
- Step 1: Document the function_edited gap in floor_witness_run_disposition
  (currently models only node-frontier, not edited_test_fns bypass)
- Step 4 Consumer 1: Require both skip axes (node-frontier + function-edited)
  to be modeled before migration; floor_witness_run_disposition must be
  extended before wiring
- Step 4 Consumer 2: Fix precompute-skip guard to require both
  RerunNodeSetProduced{nodes:[]} AND edited_test_fns.is_empty(); label
  as conservative vs the mock-closure tightening (future follow-on)
- Acceptance witness (a): Promote to FULL-PREDICATE equivalence check
  covering both skip axes, not just entry_touches_frontier_seeds

Two non-blocking notes from stern-moth-225:
- Step 1: Add consumer-surface paragraph (probe_selector,
  affected_testgen_ci_runner, affected_set_selection already import
  Impl 1 — only the Rust floor forks the authority)
- Step 3: Promote prerequisite to explicit numbered hard-gate sequence
  (prove → migrate → verify → THEN delete)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… regen ROADMAP.md

The prior hand-edit to ROADMAP.md was not backed by an authored_doc entry in
roadmap_authority.dag, causing generated_artifact_drift_gate_passes to fail after
the main merge. Fix: add the authored_doc node to the authority and let main_wet
regenerate ROADMAP.md from the single source.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@briansrls
briansrls merged commit 581803f into main Jun 30, 2026
2 checks passed
@briansrls
briansrls deleted the docs/affected-set-precompute-pruning-sketch branch June 30, 2026 18:04
briansrls added a commit that referenced this pull request Jun 30, 2026
briansrls added a commit that referenced this pull request Jun 30, 2026
…ate.

Documents that impl-vs-impl equivalence on a real diff remains open per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>
briansrls added a commit that referenced this pull request Jun 30, 2026
Rename tests/modules to stop claiming full-predicate equivalence; the
Rust harness is disposition-kernel alignment on shared Rust inputs only.
Witness (a) impl-vs-impl remains follow-on per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>
briansrls pushed a commit that referenced this pull request Jul 1, 2026
briansrls pushed a commit that referenced this pull request Jul 1, 2026
briansrls pushed a commit that referenced this pull request Jul 1, 2026
briansrls pushed a commit that referenced this pull request Jul 1, 2026
briansrls pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
…ate.

Documents that impl-vs-impl equivalence on a real diff remains open per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
Rename tests/modules to stop claiming full-predicate equivalence; the
Rust harness is disposition-kernel alignment on shared Rust inputs only.
Witness (a) impl-vs-impl remains follow-on per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
…ate.

Documents that impl-vs-impl equivalence on a real diff remains open per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
Rename tests/modules to stop claiming full-predicate equivalence; the
Rust harness is disposition-kernel alignment on shared Rust inputs only.
Witness (a) impl-vs-impl remains follow-on per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
gunbai-bot Bot pushed a commit that referenced this pull request Jul 1, 2026
briansrls added a commit that referenced this pull request Jul 1, 2026
…5994) (#6072)

* WIP: Implement affected-set CI de-fork (signed design #5994 / docs/plans/affe

* WIP: Implement affected-set CI de-fork (signed design #5994 / docs/plans/affe

* Clarify witness (a) test is partial disposition coverage, not PROVE gate.

Documents that impl-vs-impl equivalence on a real diff remains open per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>

* WIP: Implement affected-set CI de-fork (signed design #5994 / docs/plans/affe

* Retry nested cargo compile guards on transient runner infra failures.

bootstrap::stage0_cargo_check and v1_compiler_lib_tests_compile_green
now use the same CARGO_BUILD_JOBS=1 / no-sccache retry ladder as ci.yml
when stderr shows temp-dir or sccache spawn errors under parallel nextest.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Rescope PR: disposition prep only, not witness (a) PROVE gate.

Rename tests/modules to stop claiming full-predicate equivalence; the
Rust harness is disposition-kernel alignment on shared Rust inputs only.
Witness (a) impl-vs-impl remains follow-on per #5994.

Co-authored-by: Cursor <cursoragent@cursor.com>

* cargo fmt: fix cli_run disposition-kernel test formatting.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: retrigger rust_tests after sccache fleet flake on c2f8b2f.

Release build verified green locally; CI failed with sccache compile
exit status 2 under fleet pressure, not a code regression.

Co-authored-by: Cursor <cursoragent@cursor.com>

* WIP: CONTINUE + COMPLETE affected-set CI de-fork (signed design #5994) — prio

* WIP: CONTINUE + COMPLETE affected-set CI de-fork (signed design #5994) — prio

* WIP: CONTINUE + COMPLETE affected-set CI de-fork (signed design #5994) — prio

* WIP: CONTINUE + COMPLETE affected-set CI de-fork (signed design #5994) — prio

* fix: witness (a) tests after main merge (#6072)

Post-merge regressions from List→Cons carrier and path-format mismatch:
- dag_function_edited_for_row uses repo-relative path from diff touches
  (matches floor_test_fn_declaration_edited path equality)
- frontier_list_len reads Cons lists under active InterpContext

Co-authored-by: Cursor <cursoragent@cursor.com>

* WIP: CONTINUE + COMPLETE affected-set CI de-fork (signed design #5994) — prio

* fix: floor_test_fn_declaration_edited matches Rust decl span (#6072)

Add test_fn_decl_end_line so .dag overlap uses decl..next_decl-1 like
collect_frontier_seeds_from_diff_line_ranges. Body-only diff witness
(line 78) proves impl-vs-impl on span, not declaration line only.

Addresses cursor REQUEST_CHANGES on a86fddf (review 34121).

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Brian Searls <briansearls1@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant