Census sample: the sampling rule as one .dag fold, checked against the pin at the pin's revision

[gunbai-bot]

Census sample: the sampling rule as one .dag fold, checked against the pin at the pin's revision

Follow-up to #12334, agreed with the XL-2 manager (option A of the escalation). reference_conservation_stratified_sample_paths was a hand-copied list with the selection rule stated only in prose. DESIGN §6 says "name the instrument"; this gives the list an instrument.

What is added (v2.compiler.reference_conservation_census)

• sample_paths_by_rule(files), the rule's single authority.
  • It keeps .dag files under dag/ and src/v2/ and sorts them by path. One pass then groups the strata: every stratum's paths share a prefix ending at a segment boundary, so sorting makes each stratum a contiguous run.
  • Strata are the first 2 path segments under dag/ and the first 3 under src/v2/.
  • Quota: k = max(6, round_half_even(300 * stratum count / total)), capped at the stratum's small-file count.
  • Selection: indices floor(i*n/k) over the stratum's files of at most 8,000 bytes, sorted by path.
• sample_file_within_cap(content). UTF-8 never takes fewer bytes than code points, so text over 8,000 code points is over the cap without being encoded. Only shorter text is encoded to count its bytes.
• reference_conservation_stratified_sample_revision records the pin's revision next to the list (8fcd8e77b8…), so the pairing is explicit.
• reference_conservation_sample_by_rule_at_pin_exit, the instrument. It reads the tree at the pinned revision from the git object store (gunbc.namespace_step0_subject_collector collect_step0_subject_vector_at), applies the rule, and exits non-zero, naming both differences, unless the result is exactly the pinned list.

Why at the pin's revision and not the checked-out tree. The sample is a snapshot. Comparing it to a tree that has since grown would red on every PR that adds or resizes a .dag file: a change detector, not a check (DESIGN §5). Checked at its own revision, a difference can only mean the pin or the rule is wrong.

Receipt (the instrument, run once)

gunbc run --source-root dag --source-root src/v2 \
  --entry src/v2/compiler/reference_conservation_census.dag \
  --function reference_conservation_sample_by_rule_at_pin_exit
→ rc=0 (ExitSuccess) in 536 s; read all 6,880 .dag blobs at 8fcd8e77b8 (== git ls-tree count)

The rule, in .dag, reproduces the pinned 316-path list exactly.

Recorded, not hidden:

• The original pin at fec339d561 was produced by a Python one-off that took indices with a float step. At that tree, the float step and exact floor(i*n/k) differ in one stratum.
• The current pin agrees with the exact form. The exact form is now the rule, and the list's comment says so.

Controls (floor, supplied file sets; each red by mutation)

In v2.test.claim.namespace_xl0.reference_conservation_census:

claim	pins	mutation that turns it red
the_rule_takes_its_quota_at_the_floor_indices_holds	10 small among 500 → quota 6 → exactly f000, f001, f003, f005, f006, f008	indices rounded up (ceil(i*n/k))
a_share_under_the_floor_takes_six_holds	round(4.92) = 5 is raised to 6	no floor of 6
a_half_share_rounds_to_even_holds	300·13/312 = 12.5 → 12, not 13	half-up rounding
the_cap_counts_bytes_not_code_points_holds	7,999 ASCII + é (8,000 code points, 8,001 bytes) is over; 7,998 + é is within	code points instead of bytes
the_cap_admits_exactly_eight_thousand_bytes_holds	8,000 ASCII within, 8,001 over	—
a_small_stratum_takes_every_small_file_holds, a_file_outside_the_rule_roots_is_never_selected_holds, strata_are_two_segments_under_dag_and_three_under_src_v2_holds	quota capped at n; src/v1 excluded; stratum depth	—

All claims are green locally. The three large fixture file sets are nullary and enrolled warm in v2.workflow.floor_pure_producer_share, so each claim pays only for the rule it runs.

🤖 Generated with Claude Code

[gunbai-bot]

Re review 71995's observation about "checked rather than asserted": agreed, the wording claimed more than any gate enforces. Fixed in 9035df4.

The comment beside the pin now says:

• the list-to-revision check executes only when the instrument is run (gunbc run --function reference_conservation_sample_by_rule_at_pin_exit);
• no gate executes it, because it reads every .dag blob at the revision from the object store, which a hermetic floor claim cannot;
• the floor claims pin only the rule's arithmetic on supplied inputs;
• a repin must run the instrument again and record its receipt, as this PR does (rc=0 over 6,880 blobs at 8fcd8e7).

— sent from tidy-otter-111