diff --git a/DESIGN.md b/DESIGN.md index a4ff1df07af..32931953d36 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -116,4 +116,4 @@ hollow alias (minimality ≠ grounding) · state-space conflation (an `Option`/` - `cargo test --workspace` · `cargo clippy --all-targets -- -D warnings` · `cargo fmt --all --check` - one-time: `git config core.hooksPath .githooks` (pre-push runs `cargo fmt`) -- CI (`gunbc ci` generates `.github/workflows/ci.yml`; all `v1-compiler` seed bins, no shell gate): **one** composed floor pass — `claim_executor --source-root src/v2 --source-root dag --plan-entry src/v2/workflow/ci_floor_plan.dag --plan-function gunbc_ci_floor_batches`. The v2 scheduler decides the batches from the single-authority spec (`gunbc.ci_spec`); the only structural fact the plan adds is *compile-clean gates the rest* (batch-1 `dag_compile_clean_gate` → batch-2 everything else, a dependency edge). Witness enrollment: **discovery shrunk by the affected set** (operator acceptance 2026-07-09, firing the 2026-07-04 opt-in inversion's dissolve-on — see `gunbc.ci_spec` `ci_spec_discovery_flip_note`): the corpus batch scans `CiSpec.discovery_scan_dirs` (plus the source-root `*_test.dag` walk) with `SelectionApplied`, so a PR runs the tree-wide witness corpus (~1,721 rows at flip time) shrunk to the diff's affected set; selection is fail-closed (a provenance gap refuses, never widens; host-scaffold/live-tree rows never predict-skip) and the scheduled `affected-set-falsifier` (every 4 hours) runs the corpus cold with predictions recorded, so a missing selection edge surfaces as a counted divergence within one cadence window. `CiSpec.witness_entries` (`CommitWitnessClaim` rows on the `GithubActionsCiJob` surface, projected by `project_ci_floor_witness_entries`) remains the explicit-entry roster — execution-kind rows with their own resource profile, and any row that must run as declared. Empty entries AND empty discovery dirs = zero witness-corpus nodes (the regen spec's shape). Naming hygiene stays fail-closed (a `test fn` outside `*_test.dag` is still a violation — the executor runs the zero-enrollment naming walk when no discovery batch is scheduled), and tree-wide unselected discovery remains the **local** path (`claim_batch --roster-from-discovery --source-root dag --source-root src/v2 --scan-dir dag/test/claim --scan-dir src/v2/test/claim/manual`). Batch-2 also carries the effectful gates: the rust fmt gate (when a `.rs` changes; nextest was removed from CI 2026-07-11 — operator ruling recorded in `gunbc.commit_workflow` `rust_tests_removed_disposition`, the suite runs locally only; clippy removed from CI 2026-07-08 — crate-wide `#![allow(clippy::all)]` made it zero-signal over ~44m/run, still available as a local dev check), emit-host MVP smokes, source-root-ingest, and the `ci.yml` drift+parse gate (ci.yml == `gunbc ci` output). The compile-clean gate is `--target dag`, and its per-PR scope is the same import-closure authority (2026-07-16, channel 2 of the 2026-07-10 grain fork): an all-`.dag`/docs diff compiles only the affected shard-entry closures (`tools.dag_compile_clean_scope`, host fast path `entry_file_touched_via_import_closure`); any non-selectable touched path (`.rs`, workflow yml, manifests), any non-docs departed path, or any selection refusal keeps the whole-tree baseline, loudly; the falsifier cadence carries the deterministic whole-tree cold control (`GUNBC_CI_COMPILE_CLEAN_COLD_CONTROL=1`, widen-only). The regen step (self-host fixed-point) is scoped the same way at its own closure (#6732 + this change's departed-path guard): `regen_floor_skip_witness` skips the pull_request step only when a non-empty merge-base diff is provably disjoint from the regen input set (`cli_run::regen_input_sources` — the SAME closure authority `regen_stage0` compiles — plus the src/v1/** prefix and Cargo/toolchain config); empty diffs, departed non-docs paths, and every failure arm run regen, and the skip is shell-gated to pull_request events so main pushes stay the unconditional cold control. Parse is grammar-owned: `ingest_yaml_source` (`dag/extdeps/languages/yaml/ingest.dag`), no shell/host parser. +- CI (`gunbc ci` generates `.github/workflows/ci.yml`; all `v1-compiler` seed bins, no shell gate): **one** composed floor pass — `claim_executor --source-root src/v2 --source-root dag --plan-entry src/v2/workflow/ci_floor_plan.dag --plan-function gunbc_ci_floor_batches`. The v2 scheduler decides the batches from the single-authority spec (`gunbc.ci_spec`); the only structural fact the plan adds is *compile-clean gates the rest* (batch-1 `dag_compile_clean_gate` → batch-2 everything else, a dependency edge). Witness enrollment: **discovery shrunk by the affected set** (operator acceptance 2026-07-09, firing the 2026-07-04 opt-in inversion's dissolve-on — see `gunbc.ci_spec` `ci_spec_discovery_flip_note`): the corpus batch scans `CiSpec.discovery_scan_dirs` (plus the source-root `*_test.dag` walk) with `SelectionApplied`, so a PR runs the tree-wide witness corpus (~1,721 rows at flip time) shrunk to the diff's affected set; selection is fail-closed (a provenance gap refuses, never widens; host-scaffold/live-tree rows never predict-skip) and the scheduled `affected-set-falsifier` (every 4 hours) runs the corpus cold with predictions recorded, so a missing selection edge surfaces as a counted divergence within one cadence window. `CiSpec.witness_entries` (`CommitWitnessClaim` rows on the `GithubActionsCiJob` surface, projected by `project_ci_floor_witness_entries`) remains the explicit-entry roster — execution-kind rows with their own resource profile, and any row that must run as declared. Empty entries AND empty discovery dirs = zero witness-corpus nodes (the regen spec's shape). Naming hygiene stays fail-closed (a `test fn` outside `*_test.dag` is still a violation — the executor runs the zero-enrollment naming walk when no discovery batch is scheduled), and tree-wide unselected discovery remains the **local** path (`claim_batch --roster-from-discovery --source-root dag --source-root src/v2 --scan-dir dag/test/claim --scan-dir src/v2/test/claim/manual`). Batch-2 also carries the effectful gates: the rust fmt gate (when a `.rs` changes; nextest was removed from CI 2026-07-11 — operator ruling recorded in `gunbc.commit_workflow` `commit_gate_rust_suite_removed_disposition`, the suite runs locally only; clippy removed from CI 2026-07-08 — crate-wide `#![allow(clippy::all)]` made it zero-signal over ~44m/run, still available as a local dev check), emit-host MVP smokes, source-root-ingest, and the `ci.yml` drift+parse gate (ci.yml == `gunbc ci` output). The compile-clean gate is `--target dag`, and its per-PR scope is the same import-closure authority (2026-07-16, channel 2 of the 2026-07-10 grain fork): an all-`.dag`/docs diff compiles only the affected shard-entry closures (`tools.dag_compile_clean_scope`, host fast path `entry_file_touched_via_import_closure`); any non-selectable touched path (`.rs`, workflow yml, manifests), any non-docs departed path, or any selection refusal keeps the whole-tree baseline, loudly; the falsifier cadence carries the deterministic whole-tree cold control (`GUNBC_CI_COMPILE_CLEAN_COLD_CONTROL=1`, widen-only). The regen step (self-host fixed-point) is scoped the same way at its own closure (#6732 + this change's departed-path guard): `regen_floor_skip_witness` skips the pull_request step only when a non-empty merge-base diff is provably disjoint from the regen input set (`cli_run::regen_input_sources` — the SAME closure authority `regen_stage0` compiles — plus the src/v1/** prefix and Cargo/toolchain config); empty diffs, departed non-docs paths, and every failure arm run regen, and the skip is shell-gated to pull_request events so main pushes stay the unconditional cold control. Parse is grammar-owned: `ingest_yaml_source` (`dag/extdeps/languages/yaml/ingest.dag`), no shell/host parser. diff --git a/dag/extdeps/provisioning/ubuntu_install_media_fetch.dag b/dag/extdeps/provisioning/ubuntu_install_media_fetch.dag index 550471d0342..d0fbf4fcdc0 100644 --- a/dag/extdeps/provisioning/ubuntu_install_media_fetch.dag +++ b/dag/extdeps/provisioning/ubuntu_install_media_fetch.dag @@ -5,6 +5,7 @@ import std.disposition { Disposition, Terminal } import std.types { NonEmptyStr, List, String, Bool, FilePath } import extdeps.external_authority { ExternalAuthority } import extdeps.uri { Uri, Https } +import gunbc.output_policy { OutcomeIsData } import extdeps.shell import extdeps.http.client import extdeps.tools.sha256sum { sha256sum_verify_via_shell, Sha256Matches, Sha256Mismatch, Sha256VerifyInfraFailure } @@ -51,8 +52,10 @@ type MirrorFoldAcc = MirrorFoldAcc { terminal: InstallMediaFetchReconcileOutcome data mirror_fold_refusal_doc: String = "DESIGN §5 fix (cursor review, PR #6596): install_media_fetch_try_mirror previously returned `none` for BOTH an unreachable mirror (transport miss, keep trying) AND a Sha256Mismatch (a genuine digest failure), so a run where every mirror downloaded but none verified fell through the fold to 'all mirrors unreachable' - a state-space conflation between reachability and digest correctness. MirrorAttempt now names the three outcomes distinctly, and the fold tracks saw_mismatch so the final refusal reads 'digest verification failed' when that's what actually happened." +data observe_install_media_fetch_outcome_is_data_note: String = "The presence probe is OutcomeIsData: a non-zero `test -f` here means the media is ABSENT, which is one of this fn's four declared answers (InstallMediaFetchAbsent), not a fault. The type already draws the distinction the arm formalises — InstallMediaFetchVerifyInfraFailure is the separate 'the probe itself broke' arm — so subject-state and probe-breakage were never conflated here; only the effect layer's glyph conflated them. Admissible under the §5 guard: this fn returns InstallMediaFetchObservation, a typed observation whose arms ARE the adjudication, and the exit selects among them." + fn observe_install_media_fetch(path: NonEmptyStr, expected_hash: NonEmptyStr) -> InstallMediaFetchObservation { - let present = shell.Test.IsFile(path: path as FilePath) + let present = shell.Test.IsFile(path: path as FilePath, expect: OutcomeIsData) if present.is_file == false { InstallMediaFetchAbsent } else { diff --git a/dag/extdeps/provisioning/ubuntu_seeded_install_media_remaster.dag b/dag/extdeps/provisioning/ubuntu_seeded_install_media_remaster.dag index adf44951bf4..c2ee262d97b 100644 --- a/dag/extdeps/provisioning/ubuntu_seeded_install_media_remaster.dag +++ b/dag/extdeps/provisioning/ubuntu_seeded_install_media_remaster.dag @@ -1,5 +1,6 @@ module extdeps.provisioning.ubuntu_seeded_install_media_remaster +import gunbc.output_policy { OutcomeIsData } import extdeps.external_authority { ExternalAuthority } import extdeps.uri { Uri, Https } import extdeps.shell @@ -140,8 +141,10 @@ fn observe_install_media_remaster(artifact: UbuntuSeededInstallMediaArtifactRow) } } +data ensure_grub_cmdline_probe_note: String = "grep exits non-zero when the pattern is ABSENT, and absence is precisely the question this fn asks — it is the idempotence probe that decides whether to substitute or leave the file alone. Both answers are ordinary: on a fresh image the cmdline is missing and gets inserted, on a re-run it is present and nothing happens. The §5 guard holds on the belt_program_available shape: the Bool is consumed as the branch discriminator, never discarded. A grep that could not RUN is a different event and does not travel this arm." + fn install_media_remaster_ensure_grub_cmdline(grub_cfg: NonEmptyStr, cmdline: NonEmptyStr) -> Bool { - let already_present = grep.Grep.MatchesFixedString(pattern: cmdline, path: grub_cfg).matches + let already_present = grep.Grep.MatchesFixedString(pattern: cmdline, path: grub_cfg, expect: OutcomeIsData).matches let substituted = if already_present { true } else { diff --git a/dag/extdeps/render/glyphs.dag b/dag/extdeps/render/glyphs.dag index 37662b4dd8d..c2b6c20ab68 100644 --- a/dag/extdeps/render/glyphs.dag +++ b/dag/extdeps/render/glyphs.dag @@ -25,7 +25,7 @@ data standard_symbols: List = [ { id: NodeRunning, glyphs: glyphs_for_tiers(emoji: glyph(text: "🔄"), unicode: glyph(text: "◐"), ascii: glyph(text: "[~]")), color: Active }, { id: NodeCompleted, glyphs: glyphs_for_tiers(emoji: glyph(text: "✅"), unicode: glyph(text: "✓"), ascii: glyph(text: "[x]")), color: Success }, { id: NodeFailed, glyphs: glyphs_for_tiers(emoji: glyph(text: "❌"), unicode: glyph(text: "✗"), ascii: glyph(text: "[!]")), color: Error }, - { id: NodeSkipped, glyphs: glyphs_for_tiers(emoji: glyph(text: "⏩"), unicode: glyph(text: "⊘"), ascii: glyph(text: "[-]")), color: Dim }, + { id: NodeSkipped, glyphs: glyphs_for_tiers(emoji: glyph(text: "⏭️"), unicode: glyph(text: "⊘"), ascii: glyph(text: "[-]")), color: Dim }, { id: NodeIntercepted, glyphs: glyphs_for_tiers(emoji: glyph(text: "🔍"), unicode: glyph(text: "◆"), ascii: glyph(text: "[m]")), color: Info }, { id: EdgeIdle, glyphs: glyphs_for_tiers(emoji: glyph(text: "┄"), unicode: glyph(text: "┄"), ascii: glyph(text: "-")), color: Dim }, @@ -56,6 +56,17 @@ data standard_symbols: List = [ { id: StatusWarning, glyphs: glyphs_for_tiers(emoji: glyph(text: "⚠"), unicode: glyph(text: "⚠"), ascii: glyph(text: "WARN")), color: Warning }, { id: StatusInfo, glyphs: glyphs_for_tiers(emoji: glyph(text: "💡"), unicode: glyph(text: "ℹ"), ascii: glyph(text: "INFO")), color: Info }, + { id: StatusRefused, glyphs: glyphs_for_tiers(emoji: glyph(text: "🚫"), unicode: glyph(text: "⛔"), ascii: glyph(text: "REFUSED")), color: Error }, + { id: StatusBlocked, glyphs: glyphs_for_tiers(emoji: glyph(text: "⏳"), unicode: glyph(text: "◷"), ascii: glyph(text: "BLOCKED")), color: Warning }, + { id: StatusTimedOut, glyphs: glyphs_for_tiers(emoji: glyph(text: "⌛"), unicode: glyph(text: "⏱"), ascii: glyph(text: "TIMEOUT")), color: Error }, + { id: StatusDwell, glyphs: glyphs_for_tiers(emoji: glyph(text: "⏱"), unicode: glyph(text: "◴"), ascii: glyph(text: "DWELL")), color: Warning }, + { id: StatusFinal, glyphs: glyphs_for_tiers(emoji: glyph(text: "✦"), unicode: glyph(text: "✦"), ascii: glyph(text: "DONE")), color: Success }, + { id: StatusPulse, glyphs: glyphs_for_tiers(emoji: glyph(text: "🕐"), unicode: glyph(text: "◷"), ascii: glyph(text: "STATUS")), color: Dim }, + + { id: ChangeAdded, glyphs: glyphs_for_tiers(emoji: glyph(text: "+"), unicode: glyph(text: "+"), ascii: glyph(text: "+")), color: Success }, + { id: ChangeModified, glyphs: glyphs_for_tiers(emoji: glyph(text: "~"), unicode: glyph(text: "~"), ascii: glyph(text: "~")), color: Warning }, + { id: ChangeRemoved, glyphs: glyphs_for_tiers(emoji: glyph(text: "-"), unicode: glyph(text: "-"), ascii: glyph(text: "-")), color: Error }, + { id: DataList, glyphs: glyphs_for_tiers(emoji: glyph(text: "☰"), unicode: glyph(text: "☰"), ascii: glyph(text: "[L]")), color: Default }, { id: DataMap, glyphs: glyphs_for_tiers(emoji: glyph(text: "⊞"), unicode: glyph(text: "⊞"), ascii: glyph(text: "[M]")), color: Default }, { id: DataSecret, glyphs: glyphs_for_tiers(emoji: glyph(text: "🔒"), unicode: glyph(text: "✱"), ascii: glyph(text: "[*]")), color: Warning }, @@ -105,3 +116,41 @@ fn symbol_color(id: SymbolId) -> SemanticColor? { Absent => none } } + +data reward_animal_note: String = "The reward animal shown on a terminal success lands as ROWS in the one glyph authority, never as a hardcoded function in a renderer — the reference implementation studied for this lane keeps its emoji table mirrored by hand between Go and shell, which is the maintained dual representation we are here to not have. Selection is DETERMINISTIC: the index is derived from the run's own identity, so replaying a captured run reproduces its animal exactly and the event stream stays replayable by construction. A random draw would make the stream unreplayable and would be a non-determinism root of precisely the kind the determinism gate walls." + +data reward_animals: List = [ + glyphs_for_tiers(emoji: glyph(text: "🦦"), unicode: glyph(text: "✦"), ascii: glyph(text: "otter")), + glyphs_for_tiers(emoji: glyph(text: "🦊"), unicode: glyph(text: "✦"), ascii: glyph(text: "fox")), + glyphs_for_tiers(emoji: glyph(text: "🦉"), unicode: glyph(text: "✦"), ascii: glyph(text: "owl")), + glyphs_for_tiers(emoji: glyph(text: "🐢"), unicode: glyph(text: "✦"), ascii: glyph(text: "turtle")), + glyphs_for_tiers(emoji: glyph(text: "🦫"), unicode: glyph(text: "✦"), ascii: glyph(text: "beaver")), + glyphs_for_tiers(emoji: glyph(text: "🐋"), unicode: glyph(text: "✦"), ascii: glyph(text: "whale")) +] + +type RewardPick { + idx: Int + found: TierGlyphs? +} + +fn reward_animal_at(index: Int, tier: Tier) -> String? { + let n = count(reward_animals) + if n == 0 { + none + } else { + let target = index % n + let picked = fold(reward_animals, + init: RewardPick { idx: 0, found: none }, + f: (acc, g) => + if acc.idx == target { + RewardPick { idx: acc.idx + 1, found: Present { value: g } } + } else { + RewardPick { idx: acc.idx + 1, found: acc.found } + } + ) + match picked.found { + Present { value: g } => Present { value: glyph_at(glyphs: g, tier: tier) } + Absent => none + } + } +} diff --git a/dag/extdeps/tools/sha256sum.dag b/dag/extdeps/tools/sha256sum.dag index 4d905cdede7..6874b3a12a2 100644 --- a/dag/extdeps/tools/sha256sum.dag +++ b/dag/extdeps/tools/sha256sum.dag @@ -4,6 +4,7 @@ import std.types { NonEmptyStr, String, Bool } import extdeps.external_authority { ExternalAuthority } import extdeps.uri { Uri, Https } import extdeps.tools { CliTool, SourceCoreutils } +import gunbc.output_policy { OutcomeIsData } import extdeps.shell import extdeps.filesystem.filesystem_io import std.roster_frontier { @@ -65,6 +66,8 @@ data sha256_verify_native_transport_frontier_rows: List = [ ) ] +data sha256_check_outcome_is_data_note: String = "sha256sum --check exits non-zero on DIGEST MISMATCH, which is this fn's Sha256Mismatch answer, not a fault — so the check call is OutcomeIsData. The guard holds on the same evidence as the caller: Sha256VerifyVerdict already separates the two content answers (Matches / Mismatch) from Sha256VerifyInfraFailure, the arm for the verification machinery itself breaking, so annotating the content comparison cannot silence an infra failure — that arm is reached by different conditions entirely." + fn sha256sum_verify_via_shell(path: NonEmptyStr, expected_digest: NonEmptyStr) -> Sha256VerifyVerdict { let checksum_dir = shell.Mktemp.Dir() if checksum_dir.success == false { @@ -77,7 +80,7 @@ fn sha256sum_verify_via_shell(path: NonEmptyStr, expected_digest: NonEmptyStr) - let cleaned = shell.Remove.RecursiveForce(path: checksum_dir.path) Sha256VerifyInfraFailure { reason: "checksum file write failed" as NonEmptyStr } } else { - let checked = sha256sum.Sha256.CheckFile(checksum_file: checksum_path) + let checked = sha256sum.Sha256.CheckFile(checksum_file: checksum_path, expect: OutcomeIsData) let cleaned = shell.Remove.RecursiveForce(path: checksum_dir.path) if checked.matches { Sha256Matches diff --git a/dag/gunbc/ci_layer_roots.dag b/dag/gunbc/ci_layer_roots.dag index d089188e867..136a8d5e693 100644 --- a/dag/gunbc/ci_layer_roots.dag +++ b/dag/gunbc/ci_layer_roots.dag @@ -478,6 +478,12 @@ data witness_exclusion_frontier: List = [ reason: "manual-home witness whose two round-trip fns are additionally quarantined known-red via known_red_probe_entries (known_red_frontier_note)", dissolve_on: "the S2 behavioral-closure lane greens it locally; the probe entry and this row delete" }, + WitnessExclusionRow { + pattern: "no_fake_anomalies_witness_test.dag", + classification: BinWitnessWet, + reason: excl_no_fake_anomalies_reason, + dissolve_on: excl_no_fake_anomalies_dissolve + }, WitnessExclusionRow { pattern: "srv3_install_media_fetch_real_execution_witness_test.dag", classification: BinWitnessWet, @@ -731,6 +737,10 @@ fn bin_wet(entry: String, f: String) -> ScheduleWitnessEntry { ScheduleWitnessEntry { entry: entry, function: f, kind: ExecutionWitnessKind } } +data excl_no_fake_anomalies_reason: String = "The expectation-backfill WALL spawns claim_batch as a child process to read a passing corpus's own stderr, so it is Wet BY CONSTRUCTION — its whole subject is what the observation layer prints, which only exists when the effects really run. The Hermetic discovery corpus refuses it exactly as install_media_real_execution_wet_note describes ('no mock_response for operation Run — refusing to fabricate Unit'), and that refusal is correct: a mocked child emits no real observation stream, so a hermetic version of this witness could only ever pass vacuously. Landed into hermetic discovery by filename convention on first authoring (run 30178544618 @ b2411f0, batch 3) and re-homed here." + +data excl_no_fake_anomalies_dissolve: String = "no dissolution is expected: reading a real run's rendered stream is the irreducible content of this check, so it stays Wet as long as the anomaly-glyph wall exists. If the wall is ever superseded by a structural lens over the effect graph (task #15's call-node admissibility), this row deletes with the witness." + data install_media_real_execution_wet_note: String = "The four srv3_install_media_fetch_real_execution_witness_test.dag / srv3_seeded_install_media_real_execution_witness_test.dag entries at the tail of bin_witness_wet_entries below share this lane with the bin-execution witnesses (same RunnableDiscoveryBatch Wet mechanism, not a forked one - DESIGN §3) though they are not bin-execution: they drive real shell.Mktemp.Dir/Filesystem.Write/shell.Remove.RecursiveForce effects that dag/extdeps/shell/shell.dag has no mock_response for (only shell.Exec.Run/Check do), so the Hermetic discovery corpus refuses them by construction (typed 'no mock_response for operation Dir' error). Split out of their parent files (shell-to-dag B1, 2026-07-14) so the parent files' pure classification witnesses keep hermetic per-PR discovery undiminished. Dissolve-on: shell.dag's file-effect operations gain mock_response coverage AND these fns' assertions are re-checked for correctness under a fixed mock (some assert behavior-dependent outcomes, e.g. matched == false on a wrong hash, which a naive fixed mock would falsify) - then they re-enroll as ordinary hermetic discovery rows and these rows delete." data bin_witness_wet_entries: List = [ @@ -786,12 +796,14 @@ data bin_witness_wet_entries: List = [ bin_wet(entry: "dag/test/claim/http_client_get_real_execution_witness_test.dag", f: "http_client_get_roundtrip_by_real_execution"), bin_wet(entry: "dag/test/claim/http_client_get_real_execution_witness_test.dag", f: "http_client_get_missing_target_red_control_by_real_execution"), bin_wet(entry: "dag/test/claim/host_build_cache_provision_real_execution_witness_test.dag", f: "provision_read_back_failure_not_converged_by_real_execution"), - bin_wet(entry: "dag/test/claim/host_effect_plan_real_execution_witness_test.dag", f: "host_effect_plan_shell_witness_holds_by_real_execution") + bin_wet(entry: "dag/test/claim/host_effect_plan_real_execution_witness_test.dag", f: "host_effect_plan_shell_witness_holds_by_real_execution"), + bin_wet(entry: "dag/test/claim/no_fake_anomalies_witness_test.dag", f: "passing_run_shows_no_fake_anomaly_holds"), + bin_wet(entry: "dag/test/claim/no_fake_anomalies_witness_test.dag", f: "red_control_entry_with_real_anomaly_still_shows_glyph_holds") ] data bin_witness_wet_per_row_wall_budget_seconds: Int = 60 -data bin_witness_wet_per_row_budget_note: String = "Short/quick-witness discipline for the per-PR wet batch (CI floor endgame D6, operator ruling 2026-07-23): per-PR CI is short/quick witnesses, and a bin-execution row's subprocess wall is invisible to the fast-lane 5-second EVAL deadline by construction (the deadline counts interpreter CPU; child-process wall is host-effect time), so the bin lane carries its own per-row wall budget — this datum. A row whose MEASURED wall exceeds it re-homes to the falsifier wet cadence as a typed frontier row (falsifier_rehomed_bin_wet_rows below, reason + dissolve_on per row), keeping the remaining rows as the per-PR smoke subset so plumbing-diff PRs retain a discriminating signal. Classification basis: run 30009199696's per-row cost table (the 54-row pool's 561s eval was 94 percent two rows: floor_skip_discovery 289s, cross_shard_seam live-tree 183s — both re-homed). Enforcement layering: this datum is the classification rule applied at review; the EXECUTABLE wall against silent regrowth is the batch-wall budget (gunbc.ci_spec gunbc_ci_floor_batch_wall_budget_seconds — a new heavy row blows the wet batch's budget and reds with a typed refusal), so an over-budget enrollment cannot ride silently even if review misses it." +data bin_witness_wet_per_row_budget_note: String = "Short/quick-witness discipline for the per-PR wet batch (CI floor endgame D6, operator ruling 2026-07-23): per-PR CI is short/quick witnesses, and a bin-execution row's subprocess wall is invisible to the fast-lane 5-second EVAL deadline by construction (the deadline counts interpreter CPU; child-process wall is host-effect time), so the bin lane carries its own per-row wall budget — this datum. A row whose MEASURED wall exceeds it re-homes to the falsifier wet cadence as a typed frontier row (falsifier_rehomed_bin_wet_rows below, reason + dissolve_on per row), keeping the remaining rows as the per-PR smoke subset so plumbing-diff PRs retain a discriminating signal. Classification basis: run 30009199696's per-row cost table (the 54-row pool's 561s eval was 94 percent three rows: floor_skip_discovery 289s, cross_shard_seam live-tree 183s, dag_compile_clean_perturb 53s — the third is under budget and stays). Enforcement layering: this datum is the classification rule applied at review; the EXECUTABLE wall against silent regrowth is the batch clamp (gunbc.ci_spec gunbc_ci_floor_batch_clamp_params — a new heavy row grows the wet batch's unit count and can blow its derived clamp, reding with a typed refusal), so an over-budget enrollment cannot ride silently even if review misses it." type RehomedBinWetRow { entry: String diff --git a/dag/gunbc/ci_spec.dag b/dag/gunbc/ci_spec.dag index dd2aa338d1c..50ed09e313e 100644 --- a/dag/gunbc/ci_spec.dag +++ b/dag/gunbc/ci_spec.dag @@ -174,9 +174,24 @@ fn ci_spec_with_discovery_scan_dirs(spec: CiSpec, dirs: List) -> CiSpec } } -data gunbc_ci_floor_batch_wall_budget_note: String = "THE COST WALL (CI floor endgame D5, operator brief 2026-07-23 — the resolve-regression journey's section-3 finding made mechanism: nothing redded a merge that added floor minutes, so every recovered minute was re-spent by the next lane's enrollment, and the step timeout's own bounce history 30 -> 60 -> 90 -> 120 -> 180 -> 270 -> 55 is the record of the budget being raised to fit). Per-BATCH wall budgets for gunbc_ci_floor_batches, enforced by claim_executor at walk time: each batch's measured wall is recorded as a typed receipt row (target/floor-batch-wall-receipt.txt), and a batch over its budget is a typed, located refusal — FLOOR-BATCH-OVER-BUDGET naming the batch index, measured wall, budget row, and this carrier — that reds the walk. A failure arm refuses, never widens: over-budget never triggers a rerun, a wider scope, or a silent cap raise. Denominated PER-BATCH, never per-run, because plumbing PRs have a structurally different cost profile (attribution doc section 9.2: a PR touching host_prelude/cli_run legitimately runs 46 of 55 wet rows — a per-run wall would red every such PR spuriously; per-batch budgets absorb the profile where it lands). RULING RECONCILIATION (the operator dispatching the 2026-07-23 endgame brief is the sign-off on this reading): the standing rule 'no wall-clock term in any verdict' (ci_floor_materialization_receipt_note, operator ruling 2026-07-10) governs WITNESS VERDICTS — a witness's pass/fail must never depend on how long it took. Batch budgets do not touch witness verdicts: they are admission/scheduling facts at the walk grain, the same split the 5-second fast-lane eval law already uses (gunbc_ci_fast_lane_rule_note: the deadline is lane admission, not a verdict term), so a batch refusal names the BATCH, and every witness verdict inside it stands as evaluated. RAISE DISCIPLINE: raising any budget requires appending a dated receipt note here naming the run id and the enrollment that grew the batch — the same discipline as the timeout-note bounce history — never a silent number edit. Budget basis (run 30009199696, post-#7122, with the endgame fixes priced in): batch 1 cheap gates 185s measured -> 125s post-pool, budget 240; batch 2 receipt consume ~0s, budget 60; batch 3 discovery 1024s measured (corpus-denominated resolve wall, lever-1 re-diagnosis pending), budget 1320; batch 4 wet corpora 649s plumbing-profile measured -> ~175s post-re-home, budget 420; batch 5 emit-host 7s, budget 120; batch 6 ingest 414s measured (4 composed overlay children — the genuine mktemp constraint), budget 600; batch 7 reads-real-bytes 206s, budget 420. Sum 3180s = 53min under the 55min step cap. OPERATOR SIGNATURE (briansrls, 2026-07-23): the admission/verdict reading above is affirmed — per-batch wall budgets are scheduling-admission facts, not wall-clock terms in witness verdicts; this line is the declared human-signed exemption row the 2026-07-10 ruling requires. Raise discipline amended: RAISING any budget row requires a new dated operator-signed line here naming the run id and the enrollment that grew the batch; TIGHTENING may land by ordinary receipt note. The on-call remedy for FLOOR-BATCH-OVER-BUDGET is diagnose-or-signed-raise, never rerun. FOLLOW-UP ROWS (operator rework push, 2026-07-23): (a) PRELUDE COVERAGE HOLE — the ~5min before batch-1 arms (naming-hygiene walk, policy install, plan resolve/eval, governor arm, eager compile-clean install) sits OUTSIDE every batch budget and can only red at the 55min step cap; a prelude budget row lands when the phase_mark walls get their own receipt keys. (b) IDENTITY-KEYED BUDGETS — rows are keyed by batch INDEX today (the coverage witness pins length, so a schedule change reds loudly rather than misassigning); keying by batch content-identity dissolves the index coupling and rides the ComputationIdentity lane. (c) K-CAP UNION-RSS RECEIPT — cheap_gate_pool_max_claims_per_child=16 is provisional until the pooled child's union RSS is receipted (claim_batch already prints per-shard-peak-rss; the first CI runs' logs back or re-size the cap; local proxy receipt: the 4-gate outer child peaked 1.82GB with the 12-claim pool nested). OPERATOR-SIGNED RAISE (briansrls, 2026-07-24): batch 3 (row index [2]) 1320 -> 1440 seconds, +9.1 percent (operator: 'lets raise the budget slightly more to give it some room over main' / 'like 10%'; 24min keeps the row's whole-minute grain). RUN IDS: 30063268739 @ e7a9006f wall_ms=1629287, 30058048605 wall_ms=1344115, 30055079462 wall_ms=1380561 — all batch=3 against budget_ms=1320000 on PR #7137. THE ENROLLMENT THAT GREW THE BATCH: none, and that mismatch with this discipline's template IS the receipt. Batch 3 is the affected-set-selected discovery corpus, so its wall is denominated in the DIFF's affected entry count, not in a roster edit: #7137 touches gunbc.ci_layer_roots (broadly imported) and src/v1/stage0/src/cli_run.rs, selecting 839-841 entries where a typical main push selects 600-601. Per-entry cost is at PARITY with green controls, so nothing regressed — #7137 1.602/1.609/1.646 s-per-entry vs main 1.574, clever-ram-110 1.756, clever-pike-49 1.504. The retired 1320 divided by 841 entries = 1.570 s-per-entry, BELOW main's own 1.574 measured rate: the row had zero headroom for any broad-touch diff and would red a no-op PR that merely touched a widely-imported carrier. 1440/841 = 1.712 s-per-entry, ~8.8 percent over main's rate — that is the 'room over main'. HONEST RESIDUAL: 1440 covers three of the four observed runs; the 1629287 outlier (run 30063268739, host peaked 9GB vs 16GB on the other three, 1.937 s-per-entry) still reds, and that is the wall working as designed — the remedy stays diagnose-or-signed-raise, never rerun. SUM ACCOUNTING (the one invariant this raise spends): the ceiling sum moves 3180 -> 3300 seconds = exactly 55min, so this note's original 'Sum 3180s = 53min under the 55min step cap' margin is now CONSUMED. Ceilings are per-batch maxima rather than a predicted run wall, but the sum was this note's stated safety argument and it no longer holds with margin, so it is recorded here rather than dropped. FOLLOW-UP (not landed here): restore margin by TIGHTENING the rows with the largest declared-vs-measured gap once post-merge receipts exist — batch 4 (budget 420, ~175s measured post-re-home) and batch 6 (budget 600, 414s measured) — which lands by ordinary receipt note under this same discipline. OPERATOR-SIGNED RAISE (briansrls, 2026-07-24, second): batch 3 (row index [2]) 1440 -> 1680 seconds, +16.7 percent (28min, whole-minute grain). Operator's word this raise: 'you may bump the budget to 1680 - we're redoing it completely anyway' — so 1680 is a STOPGAP to green this PR while the batch-budget mechanism itself is reworked (the FOLLOW-UP ROWS above: prelude coverage, identity-keyed budgets, k-cap union-RSS), not a claim that 1680 is the settled ceiling. WHY 1440 WAS UNDER-SIZED: the first raise sized off three observations (1344/1381/1629) where one landed on a fast host; five batch-3 walls are now observed on #7137 — 1344115, 1380561, 1558226 (run 30081341899 @ 06a1f2c, host srv4-03), 1581666 (run 30067149030 @ 93642bc, host srv4-03), 1629287 (run 30063268739) — and THREE of five exceed 1440, so 1440 reds the majority, not a lone outlier. This is host variance on the ~840-entry affected set, not a per-entry regression (per-entry stays at parity with green controls, ~1.6-1.9 s-per-entry across hosts). THE ENROLLMENT THAT GREW THE BATCH: still none — batch 3 is affected-set-denominated and #7137 touches broadly-imported carriers (gunbc.ci_layer_roots, cli_run.rs), the same profile the FLOOR-BATCH-OVER-BUDGET template's 'enrollment' field does not capture. SIZING: 1680 clears all five observed walls (max 1629287) with ~3 percent margin. SUM ACCOUNTING CORRECTED: the prior line's 'sum = 55min, margin consumed' was over-conservative — per-batch maxima never co-occur, and run 30081341899's ACTUAL floor step wall was ~44min (b1 185s + b2 ~0s + b3 1558s + b4 175s + b5 7s + b6 414s + b7 206s), well under the 55min step timeout that is the real constraint; the ceiling-sum (now 3540s) is a paper figure nothing hits. The remedy stays diagnose-or-signed-raise, never rerun. OPERATOR-SIGNED RAISE (briansrls, 2026-07-25, third): batch 3 (row index [2]) 1680 -> 2100 seconds, +25 percent (35min, whole-minute grain). Operator's word this raise: '2100, go ... sized to the fleet host envelope over two observations; a run exceeding 2100 extends the basis by dated receipt, not a fresh diagnosis'. COST DRIVER: the namespace-only flip #7178 (merged 2026-07-25T02:02) — bare references now resolve by containment-walk (#6848 per-entry bare-reference fixpoint), adding ~200s to the affected-set discovery-corpus resolve wall vs the import-scoped basis the prior 1680 was sized on (#7137, all pre-flip). NOT a roster enrollment and NOT a per-PR regression: THREE independent post-flip observations exceed 1680 on disjoint denominators — #7198 (layering deletion, run 30140810666) wall_ms=1820001, #7188 (extdeps dedup, run 30142221249) wall_ms=1859345, and the falsifier lane's post-flip warm run (~36min corpus resolve, same signature). SIZING: 1980 clears the two PR observations at ~6.5 percent, but both are single host placements and this row's own history (1440 sized off three fast-biased observations, redded the majority, re-raised the same day) is the cautionary tale; 2100 is sized to the ±20-percent fleet host envelope so a slow-host broad-touch post-flip run (~2100-2200s) does not round-trip this row a third time — the drift is attributed, not suspicious. STOPGAP until #6848 cuts per-entry resolve (dispatched as a lane now, not deferred) and the row dissolves into #7162's derived clamp (units x rate absorbs the width axis; the flip's cost becomes a re-signed rate, not a wall); re-tighten by ordinary receipt note then. SUM ACCOUNTING: ceiling sum 3540 -> 3960s, still a paper figure (per-batch maxima never co-occur; the actual floor step wall stays well under the 55min step cap). The remedy stays diagnose-or-signed-raise, never rerun." +data gunbc_ci_floor_batch_wall_budget_note: String = "SUPERSEDED by the derived clamp gunbc_ci_floor_batch_clamp_note (Piece 3, 2026-07-24): the hand-set gunbc_ci_floor_batch_wall_budget_seconds list this note governed is deleted, replaced by overhead + units*rate computed at run time; this note is kept for the operator-signed static-era history (including the two 1320->1440->1680 raises below) and the raise discipline, which carries forward to the clamp constants. THE COST WALL (CI floor endgame D5, operator brief 2026-07-23 — the resolve-regression journey's section-3 finding made mechanism: nothing redded a merge that added floor minutes, so every recovered minute was re-spent by the next lane's enrollment, and the step timeout's own bounce history 30 -> 60 -> 90 -> 120 -> 180 -> 270 -> 55 is the record of the budget being raised to fit). Per-BATCH wall budgets for gunbc_ci_floor_batches, enforced by claim_executor at walk time: each batch's measured wall is recorded as a typed receipt row (target/floor-batch-wall-receipt.txt), and a batch over its budget is a typed, located refusal — FLOOR-BATCH-OVER-BUDGET naming the batch index, measured wall, budget row, and this carrier — that reds the walk. A failure arm refuses, never widens: over-budget never triggers a rerun, a wider scope, or a silent cap raise. Denominated PER-BATCH, never per-run, because plumbing PRs have a structurally different cost profile (attribution doc section 9.2: a PR touching host_prelude/cli_run legitimately runs 46 of 55 wet rows — a per-run wall would red every such PR spuriously; per-batch budgets absorb the profile where it lands). RULING RECONCILIATION (the operator dispatching the 2026-07-23 endgame brief is the sign-off on this reading): the standing rule 'no wall-clock term in any verdict' (ci_floor_materialization_receipt_note, operator ruling 2026-07-10) governs WITNESS VERDICTS — a witness's pass/fail must never depend on how long it took. Batch budgets do not touch witness verdicts: they are admission/scheduling facts at the walk grain, the same split the 5-second fast-lane eval law already uses (gunbc_ci_fast_lane_rule_note: the deadline is lane admission, not a verdict term), so a batch refusal names the BATCH, and every witness verdict inside it stands as evaluated. RAISE DISCIPLINE: raising any budget requires appending a dated receipt note here naming the run id and the enrollment that grew the batch — the same discipline as the timeout-note bounce history — never a silent number edit. Budget basis (run 30009199696, post-#7122, with the endgame fixes priced in): batch 1 cheap gates 185s measured -> 125s post-pool, budget 240; batch 2 receipt consume ~0s, budget 60; batch 3 discovery 1024s measured (corpus-denominated resolve wall, lever-1 re-diagnosis pending), budget 1320; batch 4 wet corpora 649s plumbing-profile measured -> ~175s post-re-home, budget 420; batch 5 emit-host 7s, budget 120; batch 6 ingest 414s measured (4 composed overlay children — the genuine mktemp constraint), budget 600; batch 7 reads-real-bytes 206s, budget 420. Sum 3180s = 53min under the 55min step cap. OPERATOR SIGNATURE (briansrls, 2026-07-23): the admission/verdict reading above is affirmed — per-batch wall budgets are scheduling-admission facts, not wall-clock terms in witness verdicts; this line is the declared human-signed exemption row the 2026-07-10 ruling requires. Raise discipline amended: RAISING any budget row requires a new dated operator-signed line here naming the run id and the enrollment that grew the batch; TIGHTENING may land by ordinary receipt note. The on-call remedy for FLOOR-BATCH-OVER-BUDGET is diagnose-or-signed-raise, never rerun. FOLLOW-UP ROWS (operator rework push, 2026-07-23): (a) PRELUDE COVERAGE HOLE — the ~5min before batch-1 arms (naming-hygiene walk, policy install, plan resolve/eval, governor arm, eager compile-clean install) sits OUTSIDE every batch budget and can only red at the 55min step cap; a prelude budget row lands when the phase_mark walls get their own receipt keys. (b) IDENTITY-KEYED BUDGETS — rows are keyed by batch INDEX today (the coverage witness pins length, so a schedule change reds loudly rather than misassigning); keying by batch content-identity dissolves the index coupling and rides the ComputationIdentity lane. (c) K-CAP UNION-RSS RECEIPT — cheap_gate_pool_max_claims_per_child=16 is provisional until the pooled child's union RSS is receipted (claim_batch already prints per-shard-peak-rss; the first CI runs' logs back or re-size the cap; local proxy receipt: the 4-gate outer child peaked 1.82GB with the 12-claim pool nested). OPERATOR-SIGNED RAISE (briansrls, 2026-07-24): batch 3 (row index [2]) 1320 -> 1440 seconds, +9.1 percent (operator: 'lets raise the budget slightly more to give it some room over main' / 'like 10%'; 24min keeps the row's whole-minute grain). RUN IDS: 30063268739 @ e7a9006f wall_ms=1629287, 30058048605 wall_ms=1344115, 30055079462 wall_ms=1380561 — all batch=3 against budget_ms=1320000 on PR #7137. THE ENROLLMENT THAT GREW THE BATCH: none, and that mismatch with this discipline's template IS the receipt. Batch 3 is the affected-set-selected discovery corpus, so its wall is denominated in the DIFF's affected entry count, not in a roster edit: #7137 touches gunbc.ci_layer_roots (broadly imported) and src/v1/stage0/src/cli_run.rs, selecting 839-841 entries where a typical main push selects 600-601. Per-entry cost is at PARITY with green controls, so nothing regressed — #7137 1.602/1.609/1.646 s-per-entry vs main 1.574, clever-ram-110 1.756, clever-pike-49 1.504. The retired 1320 divided by 841 entries = 1.570 s-per-entry, BELOW main's own 1.574 measured rate: the row had zero headroom for any broad-touch diff and would red a no-op PR that merely touched a widely-imported carrier. 1440/841 = 1.712 s-per-entry, ~8.8 percent over main's rate — that is the 'room over main'. HONEST RESIDUAL: 1440 covers three of the four observed runs; the 1629287 outlier (run 30063268739, host peaked 9GB vs 16GB on the other three, 1.937 s-per-entry) still reds, and that is the wall working as designed — the remedy stays diagnose-or-signed-raise, never rerun. SUM ACCOUNTING (the one invariant this raise spends): the ceiling sum moves 3180 -> 3300 seconds = exactly 55min, so this note's original 'Sum 3180s = 53min under the 55min step cap' margin is now CONSUMED. Ceilings are per-batch maxima rather than a predicted run wall, but the sum was this note's stated safety argument and it no longer holds with margin, so it is recorded here rather than dropped. FOLLOW-UP (not landed here): restore margin by TIGHTENING the rows with the largest declared-vs-measured gap once post-merge receipts exist — batch 4 (budget 420, ~175s measured post-re-home) and batch 6 (budget 600, 414s measured) — which lands by ordinary receipt note under this same discipline. OPERATOR-SIGNED RAISE (briansrls, 2026-07-24, second): batch 3 (row index [2]) 1440 -> 1680 seconds, +16.7 percent (28min, whole-minute grain). Operator's word this raise: 'you may bump the budget to 1680 - we're redoing it completely anyway' — so 1680 is a STOPGAP to green this PR while the batch-budget mechanism itself is reworked (the FOLLOW-UP ROWS above: prelude coverage, identity-keyed budgets, k-cap union-RSS), not a claim that 1680 is the settled ceiling. WHY 1440 WAS UNDER-SIZED: the first raise sized off three observations (1344/1381/1629) where one landed on a fast host; five batch-3 walls are now observed on #7137 — 1344115, 1380561, 1558226 (run 30081341899 @ 06a1f2c, host srv4-03), 1581666 (run 30067149030 @ 93642bc, host srv4-03), 1629287 (run 30063268739) — and THREE of five exceed 1440, so 1440 reds the majority, not a lone outlier. This is host variance on the ~840-entry affected set, not a per-entry regression (per-entry stays at parity with green controls, ~1.6-1.9 s-per-entry across hosts). THE ENROLLMENT THAT GREW THE BATCH: still none — batch 3 is affected-set-denominated and #7137 touches broadly-imported carriers (gunbc.ci_layer_roots, cli_run.rs), the same profile the FLOOR-BATCH-OVER-BUDGET template's 'enrollment' field does not capture. SIZING: 1680 clears all five observed walls (max 1629287) with ~3 percent margin. SUM ACCOUNTING CORRECTED: the prior line's 'sum = 55min, margin consumed' was over-conservative — per-batch maxima never co-occur, and run 30081341899's ACTUAL floor step wall was ~44min (b1 185s + b2 ~0s + b3 1558s + b4 175s + b5 7s + b6 414s + b7 206s), well under the 55min step timeout that is the real constraint; the ceiling-sum (now 3540s) is a paper figure nothing hits. The remedy stays diagnose-or-signed-raise, never rerun." -data gunbc_ci_floor_batch_wall_budget_seconds: List = [240, 60, 2100, 420, 120, 600, 420] +type FloorBatchClamp { + overhead_seconds: Int + per_unit_ms: Int +} + +data gunbc_ci_floor_batch_clamp_note: String = "DERIVED per-batch wall clamp (Piece 3, ci-two-tier-placement-redesign.md §9.8, operator 2026-07-24). Supersedes the hand-set gunbc_ci_floor_batch_wall_budget_seconds list (deleted; its two operator-signed raises 1320->1440->1680 are the static-era history kept in gunbc_ci_floor_batch_wall_budget_note): a scalar wall budget conflated workload size (affected-set selection is diff-proportional BY DESIGN, ~5x swing), host speed (±20% fleet envelope), and the quantity actually worth bounding (per-unit cost creep). The clamp re-denominates — per batch, clamp_ms = overhead_seconds*1000 + runtime_unit_count * per_unit_ms — computed by claim_executor from THIS authority plus the affected-set-selected unit count it alone knows (the schedule holds one opaque discovery runnable; the witness count is runtime, not schedule data). Rows are index-aligned to gunbc_ci_floor_batches (the length-match witness pins the alignment, mirroring the deleted list's discipline). The load-bearing row is the discovery witness batch (index 2): overhead 300s + 1000ms/witness, so a full corpus of ~2316 witnesses clamps at ~44min (under the 55-min step cap, ~1.6x the ~25-min healthy wall) while every legitimate observed full-corpus wall (1344-1629s) passes, and a runaway reds proportionally instead of at the fixed 1680s that the two hand-raises had to keep chasing. Fixed-count gate batches carry per_unit_ms 0 (their count does not vary, so overhead IS the clamp) at their measured basis. Index 3 (wet corpora) stays a fixed overhead pending a wet-per-witness rate from the D2 probe — a declared calibration gap, not a hidden default. SIGNED CONSTANTS (operator, 2026-07-24): the witness aggregate coefficient 1000ms and the discovery overhead 300s; the per-WITNESS hard max is NOT redefined here — it stays the single fast-lane authority gunbc_ci_fast_lane_eval_budget_ms (5s). BASIS OF THE 1000ms COEFFICIENT (operator, 2026-07-24): the aggregate coefficient is denominated against the observed 0.58-0.70 s/witness (the 1344-1629s full-corpus fleet envelope over ~2316 witnesses) on the srv fleet host class (arm64 self-hosted, capped runners) at the adaptive governor's realized worker width. Naming host-class-x-worker-width as the basis makes a future width or fleet change a DELIBERATE re-sign of this constant (s/witness re-denominates when the fleet or width moves), never a rediscovered fleet-wide red; the ~1.4-1.7x headroom the 1000ms average carries over the observed top rate IS the >=~1.6x runaway the clamp is sized to catch, and sub-threshold creep below that ratio is owned by the gauntlet's per-cadence s/unit receipt (not this clamp). RAISE DISCIPLINE (carried from gunbc_ci_floor_batch_wall_budget_note): raising any overhead or rate requires an appended dated operator-signed line naming the run id and the enrollment that grew the batch; tightening may land by ordinary receipt note; unit counts need no signature (the schedule computes them). The clamp is interim mechanics — the structural wall is the complexity lens (§8, cost <= a + b*n asserted at compile time), and the clamp demotes to host-pathology backstop when that lens goes Blocking. The 55-min step cap stays the absolute backstop for the total floor wall; GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS lowers the COMPUTED clamp (min), never raises — the RED-control hook, never an escape hatch.OPERATOR-SIGNED RAISE (briansrls, 2026-07-25): batch 4 (row index [3]) 420 -> 540 seconds, +28.6 percent. THE MARGIN RULING THIS INSTANTIATES, signed once for the budget FAMILY rather than re-litigated per incident: a clamp was doing double duty as merge-refusal threshold ('this must not merge') and growth detector ('something got slower - look'). Tight margins serve the second job and demonstrably worked - the perturb row's 53 -> 141s growth was caught precisely because the clamp was tight - but they make the FIRST job fire on host roulette, and a breach that means 'you landed on srv2-02' trains the on-call to rerun, which is the crying-wolf failure mode wearing budget clothes. POLICY: clamps are sized to cover MEASURED fleet spread, so a breach means content grew, never which host answered. The growth-detector job moves to per-row trend receipts on the falsifier cadence (row grew >2x against its dated basis = a counted drift receipt), which is what makes a larger clamp margin safe - sensitivity is preserved at row grain while merge-refusal stops firing on variance. BASIS OF 540: post-re-home projection 377s (the worst of the two observed walls, 613864ms on srv2-02, minus the ~237s of eval the six re-homed rows carried) x 1.2 worst observed fleet spread = ~452s, + ~20 percent policy margin = 540 (whole-minute grain, 9min). RUN IDS: 30148859947 @ 619bba5 wall_ms=613864 units=74 host srv2-02; 30163496549 @ 9f87967 wall_ms=571556 units=74 host srv3-01 - identical content, 7.5 percent apart, which IS the spread this raise funds. THE ENROLLMENT THAT GREW THE BATCH: none - and as with the batch-3 raises, that mismatch with this discipline's template IS the receipt. Batch 4 SHRANK this cycle (six rows re-homed to FalsifierCadenceJob, units 74 -> 68); the raise buys spread coverage on a batch that got smaller, not headroom for growth. WHY THIS IS NOT THE FORBIDDEN WIDEN: batch 4's per_unit_ms is 0, which this note ALREADY declares 'a fixed overhead pending a wet-per-witness rate from the D2 probe - a declared calibration gap, not a hidden default'; 540 funds that gap with an honest, dated, dissolving interim instead of leaving the flat rate to fire on host variance. It refuses exactly as before - only the threshold moved, and it moved on a stated measurement, not to make a red go away. HOST SPREAD IS NOT WEATHER and has a named owner: srv1/srv2 still run the pre-#7213 sccache units and are typed expected-latent-defective until re-provisioned (the A1/reach gap), so part of the 7.5-20 percent collapses when the fleet lane re-converges them; this margin covers the genuine hardware-heterogeneity residue, eventually the machine-shape lane's. DISSOLVE-ON: the dag_compile_clean_perturb_receipts_holds diagnosis lands (141s, 38 percent of the remaining batch; if the growth is the cold-spawn / per-entry-reconcile class the wasted_ms=166525 reading supports, returning it toward its 53s basis puts worst case near 350s - under even the retired 420) AND the per-unit rate replaces the flat clamp from the D2 probe; then this row recalibrates by ordinary receipt note under the tightening rule. SAME DISEASE, NOT YET DOSED: gunbc_falsifier_self_host_wet_receipt_wall_budget (600s, the 707s falsifier red) rests on a ten-day-stale basis with zero spread allowance. The margin POLICY above governs it too, but the NUMBER is deliberately not moved here - it is gated on its own sccache-vs-growth attribution, so that dosing lands with that measurement rather than by analogy. WORKER COUNTER-SIGNATURE (claude, 2026-07-25, per this discipline's dated-line requirement): I ran the dissolve-on diagnosis and it REFUTES the mechanism this note hypothesised. The growth is NOT the cold-spawn / per-entry-reconcile class the wasted_ms=166525 reading suggested. Measured by execution on the perturb spawn shape (trivial 1-module compile, 4 source roots) there are TWO independent corpus-denominated costs, in different places. (A) THE CENSUS, inside reconcile: main.rs:491-514 reads every indexed module outside the closure off disk into census_only_sources, and v1_compiler_compile.rs:2580 parse_census_fill_sources parses all ~2543 of them BETWEEN compile.normalize.done and compile.reconcile.done - so it books to the reconcile bin and reads as 'reconcile is slow' when it is a whole-tree parse wearing reconcile's label. ~23s local, ~12-13s on fleet. (B) THE POLICY TAIL, after the compile: cli_run.rs:2105 compile_clean_unlisted_import_use_blocks_from_policy calls default_source_roots() (WHOLE TREE) + resolve_entry_graph_shared to evaluate ONE nullary Bool fn. ~34-42s, once per process, and INDEPENDENT of what was compiled - proven by running the same module with only its own source root: zero pool, all phases 0ms, still 42.4s wall. THE ASYMMETRY that closes the arithmetic: main.rs:566/598 exit(1) on hard diagnostics, so a RED compile never reaches the gate (measured 27.4s total, ~2s tail) while a GREEN compile pays it in full (57.8s total, ~34s tail). The perturb row makes 4 spawns, 2 red and 2 green: ~2x14.5s + ~2x60s = ~149s against the measured 148495.7ms. Pre-#7179 that same row is 4 spawns x census-only = ~52s, which IS the 53s basis. So cost B is the ENTIRE 53->148s regression (~94s of the 148s) and cost A was always present. CONSEQUENCE FOR THIS ROW: the note's projected 'near 350s' still holds directionally but by a different mechanism - fixing B alone returns the row to ~55s, so worst case lands well under even the retired 420 and the 540 becomes pure spread coverage, which is what this note already says it is for. NO BISECT WAS RUN AND NONE IS NEEDED: the three-run measurement attributes the split directly, so the bisect across #7178/#7179 named in the probe plan is superseded work (DESIGN 2 - redundant work is not free just because it would confirm). DISSOLVE-ON, unchanged in shape but now half-discharged: the diagnosis half is DONE (this line); recalibration still waits on the cost-B fix landing (scope the policy resolve to the policy module's own import closure) plus the D2 wet-per-witness rate replacing per_unit_ms 0 on this row." + +data gunbc_ci_floor_batch_clamp_params: List = [ + FloorBatchClamp { overhead_seconds: 240, per_unit_ms: 0 }, + FloorBatchClamp { overhead_seconds: 60, per_unit_ms: 0 }, + FloorBatchClamp { overhead_seconds: 300, per_unit_ms: 1000 }, + FloorBatchClamp { overhead_seconds: 540, per_unit_ms: 0 }, + FloorBatchClamp { overhead_seconds: 120, per_unit_ms: 0 }, + FloorBatchClamp { overhead_seconds: 600, per_unit_ms: 0 }, + FloorBatchClamp { overhead_seconds: 420, per_unit_ms: 0 } +] data ci_release_features: String = "text_lookup_work_counter" @@ -194,7 +209,7 @@ fn ci_release_build_line() -> String { ) } -data ci_fmt_gate_note: String = "cargo fmt --all --check, RE-ENROLLED in CI 2026-07-15 (operator ruling, this PR) as a standalone build-job step — NOT a RustMonolithGate re-enrollment, which gunbc.commit_workflow.rust_tests_removed_disposition rejects and this row does not touch. That ruling (2026-07-11) removed the fmt+nextest bundle for reasons that are all facts about NEXTEST — ~37 GiB, red on main, non-required, a permanent tolerated red (DESIGN 5) — and fmt was collateral damage of a DESIGN 3 fusion: tools.rust_gates_ci.run_gates welds two separable facts (fmt: 4.3s measured, parse-only, no build, green; nextest: 37 GiB compile+run, red) into one ProcessExit, so killing the bundle killed the cheap green half too. The ruling's declared replacement coverage — the pre-push hook (CommitCargoFmtCheck on GitPrePushHook) — is an escape hatch (DESIGN 5: opt-in per clone via a manual core.hooksPath config, bypassable with --no-verify, absent in container worktrees) and is PROVEN ineffective: #6658 landed an unformatted .rs on main 2026-07-15 and nothing caught it; the falsifier lane found it only by running fmt by hand. Placement: FIRST step of the build job, before the ~33min release build, so a 4-second violation fails in 4 seconds rather than after the build; build is not protection-required itself but ci needs:[build], so a red here blocks the required job by construction. Cost: 4.3s measured whole-workspace, on a toolchain that already installs the rustfmt component in ci_prelude_steps. 🟡 dissolve-on: the fmt spelling here is a shell-transport realization of extdeps.cargo_build cargo.Build.Fmt (the same operation tools.rust_gates_ci calls through the effect model); it collapses onto that single authority when the ci.yml emit seam can bind a typed cargo operation instead of a hand-spelled argv line — the same de-fork ci_release_build_line awaits. Retires WITH the v1 seed at DESIGN 7 terminal collapse, when no .rs remains to format." +data ci_fmt_gate_note: String = "cargo fmt --all --check, RE-ENROLLED in CI 2026-07-15 (operator ruling, this PR) as a standalone build-job step — NOT a RustMonolithGate re-enrollment, which gunbc.commit_workflow.commit_gate_rust_suite_removed_disposition rejects and this row does not touch. That ruling (2026-07-11) removed the fmt+nextest bundle for reasons that are all facts about NEXTEST — ~37 GiB, red on main, non-required, a permanent tolerated red (DESIGN 5) — and fmt was collateral damage of a DESIGN 3 fusion: tools.rust_gates_ci.run_gates welds two separable facts (fmt: 4.3s measured, parse-only, no build, green; nextest: 37 GiB compile+run, red) into one ProcessExit, so killing the bundle killed the cheap green half too. The ruling's declared replacement coverage — the pre-push hook (CommitCargoFmtCheck on GitPrePushHook) — is an escape hatch (DESIGN 5: opt-in per clone via a manual core.hooksPath config, bypassable with --no-verify, absent in container worktrees) and is PROVEN ineffective: #6658 landed an unformatted .rs on main 2026-07-15 and nothing caught it; the falsifier lane found it only by running fmt by hand. Placement: FIRST step of the build job, before the ~33min release build, so a 4-second violation fails in 4 seconds rather than after the build; build is not protection-required itself but ci needs:[build], so a red here blocks the required job by construction. Cost: 4.3s measured whole-workspace, on a toolchain that already installs the rustfmt component in ci_prelude_steps. 🟡 dissolve-on: the fmt spelling here is a shell-transport realization of extdeps.cargo_build cargo.Build.Fmt (the same operation tools.rust_gates_ci calls through the effect model); it collapses onto that single authority when the ci.yml emit seam can bind a typed cargo operation instead of a hand-spelled argv line — the same de-fork ci_release_build_line awaits. Retires WITH the v1 seed at DESIGN 7 terminal collapse, when no .rs remains to format." fn ci_fmt_gate_line() -> String { "\"$CARGO_BIN\" fmt --all --check" diff --git a/dag/gunbc/ci_workflow.dag b/dag/gunbc/ci_workflow.dag index dbd675c0768..2e15c0299d6 100644 --- a/dag/gunbc/ci_workflow.dag +++ b/dag/gunbc/ci_workflow.dag @@ -427,7 +427,7 @@ data gunbc_ci_regen_step_timeout_minutes: Duration = 15 data gunbc_ci_regen_step_timeout_note: String = "The regen step gets its OWN budget (operator main-timeout ruling 2026-07-23): it measured ~5min on every recent green run, and it previously borrowed the floor step's cap — which double-counted the floor budget in the job backstop sum and let a wedged regen sit for hours. 15m = 3x the measured envelope; a regen that exceeds it is a defect to diagnose, never headroom to grant." -data gunbc_ci_floor_step_timeout_discovery_flip_note: String = "COST WALL landed 2026-07-23 (CI floor endgame, appended per the raise-discipline this history IS the receipt for): the bounce pattern below — every raise fitting the cap to the cost — now has its construction answer: per-BATCH wall budgets as data (gunbc.ci_spec gunbc_ci_floor_batch_wall_budget_seconds, sum 53m under this 55m cap) enforced by claim_executor as typed FLOOR-BATCH-OVER-BUDGET refusals with per-batch receipt rows, so a lane that adds minutes reds ITS batch at merge time instead of bouncing this cap upward later. This cap is unchanged by that PR; 55 -> 40 is proposed as a data change AFTER the post-merge receipts exist (operator signs). PRIOR HISTORY: -> 55 operator ruling 2026-07-23 (main wall ~75m: 'even 1 hour is absolutely ridiculous'): the 270 cap legalized a 4.5-hour crawl — receipt run 29976854620 @ 76fa6548e killed by hand at t=227m with current=16.3G pinned at memory.high, swap=34.4G, high_events=37,096,823, psi_some_avg10=33.82, oom_kill=0, governor 'hard back-off 1->1' every few seconds: the memory.high throttle-crawl class (retention finally exceeded the slot cap; the prior 40-50m greens were already pinned at 15.2-16.1G with zero headroom), and 8 subsequent main pushes wedged identically behind it fleet-wide. Recent green floors run 39-45m, so 55 is a real ceiling not a target; the regression ledger to hold onto: ~8m pre-#6848 -> ~18m (#6848) -> 40-50m (retention accreting to the cap) -> 4h (cap lost) — one disease (retention, not footprint), never an accepted baseline. Named follow-ups so this cap becomes the backstop rather than the diagnostic: the governor gains a terminal crawl-refusal arm (sustained high_events storm at width=1 -> typed FloorRefusedMemoryBudget naming batch/peak/swap, minutes not hours), and the #7106-attributed levers (one-tree-one-resolve, #6848 once-per-entry fixpoint, M2 eviction) bring the floor back under the cap with margin. Superseded main runs are NOT cancelled by policy (operator 2026-07-23): the per-commit verdict history is bisection evidence — the timeout IS the bound. PRIOR HISTORY: -> 270 restored 2026-07-12 on #6512 after 60m fail-fast regressed merge CI (receipt run 29197126623 @ 35f212fa5d: 60m step kill with 0 witness FAILs — batch-1 compile-clean PASS @ 15:01, batch-2 skip walk ~26m to recompute_trace_probe_test, then ~34m silent eval until kill; same serial-floor-wall class as 29183446733). Prior -> 60 operator ruling 2026-07-12: lower floor step cap from 270m so CI fails fast while the serial floor wall is debugged separately (receipt run 29183446733 @ 2e856a5617: 270m kill with 0 witness FAILs, batch-2 still in discovery SKIP walk). Prior -> 270 at PR #6464 receipt run 29151777611 (2026-07-11): 180m step cap still timed out mid batch 2 — 1535 SKIP rows finished @ 12:35:43, then ~147m silent witness-execution phase until the 180m kill @ 15:03:12 with batch 2 never completing (~177m in-batch from 12:06:38; ~443 non-skipped rows in roster of 1978). Prior -> 180 at run 29148344466 (~116m in-batch, ~87m post-skip). Prior -> 120 at run 29145270700. Prior -> 90 at run 29141663541. Prior -> 90 at body_lowering normalize hook (#6459 receipt run 29148735992 @ 3a372b7: floor step timed out at 60m MID discovery corpus after ~320 SKIP lines — batch-1 compile-clean normalize reconcile ~263s with body_lowering_fold in the normalize import closure; discovery still resolves every unique entry file before skip, so resolve->normalize pulled the ~1.3k-line scaffold into most closure walks). Structural fix on the same lane: NormalizedTree moved to v2.compiler.normalized_tree so v2.compiler.resolve no longer imports normalize (dissolve-on: skip-before-resolve so skipped rows never pay entry resolve). Prior step budget -> 60 at the discovery flip (gunbc.ci_spec ci_spec_discovery_flip_note; authored as 30 -> 60 on #6403, main had meanwhile bumped 30 -> 45 with the #6422 enrollments): the corpus discovery batch adds a whole-tree resolve plus the always-run live-tree rows to the floor step. Discovery corpus spawn_width_cap stays pinned to 1 (ci_corpus_discovery_spawn_width_cap in v2.workflow.ci_floor_plan): at width W the executor holds the parent's process-shared index PLUS W private shard indexes ((1+W) x whole-tree residency; width=2 OOM receipt run 28999086030), while width=1 runs rows on the main thread against the ONE shared index (union-resolve S1). CORRECTED RECEIPT (2026-07-10): run 29000557166's floor did NOT complete - the executor was host-OOM-killed ~8min in, MID discovery corpus (the log's later ExitSuccess belongs to the merge-admission STAMP tool, which stamped CI_FLOOR_EXIT=137); no flipped corpus has completed in CI yet - the only completion receipt is local (33.5GiB container, ~40min at width 5). The kill vector is host-level oversubscription (see gunbc_falsifier_plan_spawn_width_note), not this width model. #6475 receipt run 29143617420 @ 0c0f73: batch-1 compile-clean ~4m green; batch-2 discovery killed at 90m step cap mid-manual (last skip rust_wire_serde @ 07:47:49; ~15GiB peak). #6475 receipt run 29146814967 @ 6a61fce: same stall at 120m (last skip rust_wire_serde @ 09:34:49; ~85m silent eval). #6475 receipt run 29150477894 @ 6cd5326: same stall at 180m (last skip rust_wire_serde @ 11:43:46; ~88m silent eval) — local gunbc run on s1_closure_parses_holds reads 3/40 closure paths in 10m before timeout, matching the silent-eval class (live filesystem_read per path, not a skip-logged row). #6475 receipt run 29161709373 @ 9d9852c: batch-1 compile-clean refused — gunbc_ci_floor_step_timeout_discovery_flip_note string literal terminated early at col 2043 (body_lowering suffix spliced after closing quote during rebase merge); restored single-line literal on rebase to main #6464 note. Revisit down when floor memoization / resolver graph-major shrink the resolve wall; the affected-set selection receipts (skip counts per PR) are the cost dial to watch." +data gunbc_ci_floor_step_timeout_discovery_flip_note: String = "COST WALL landed 2026-07-23 (CI floor endgame, appended per the raise-discipline this history IS the receipt for): the bounce pattern below — every raise fitting the cap to the cost — now has its construction answer: per-BATCH clamps derived from data (gunbc.ci_spec gunbc_ci_floor_batch_clamp_params, superseding the 53m static gunbc_ci_floor_batch_wall_budget_seconds list) enforced by claim_executor as typed FLOOR-BATCH-OVER-BUDGET refusals with per-batch receipt rows, so a lane that adds minutes reds ITS batch at merge time instead of bouncing this cap upward later. This cap is unchanged by that PR; 55 -> 40 is proposed as a data change AFTER the post-merge receipts exist (operator signs). PRIOR HISTORY: -> 55 operator ruling 2026-07-23 (main wall ~75m: 'even 1 hour is absolutely ridiculous'): the 270 cap legalized a 4.5-hour crawl — receipt run 29976854620 @ 76fa6548e killed by hand at t=227m with current=16.3G pinned at memory.high, swap=34.4G, high_events=37,096,823, psi_some_avg10=33.82, oom_kill=0, governor 'hard back-off 1->1' every few seconds: the memory.high throttle-crawl class (retention finally exceeded the slot cap; the prior 40-50m greens were already pinned at 15.2-16.1G with zero headroom), and 8 subsequent main pushes wedged identically behind it fleet-wide. Recent green floors run 39-45m, so 55 is a real ceiling not a target; the regression ledger to hold onto: ~8m pre-#6848 -> ~18m (#6848) -> 40-50m (retention accreting to the cap) -> 4h (cap lost) — one disease (retention, not footprint), never an accepted baseline. Named follow-ups so this cap becomes the backstop rather than the diagnostic: the governor gains a terminal crawl-refusal arm (sustained high_events storm at width=1 -> typed FloorRefusedMemoryBudget naming batch/peak/swap, minutes not hours), and the #7106-attributed levers (one-tree-one-resolve, #6848 once-per-entry fixpoint, M2 eviction) bring the floor back under the cap with margin. Superseded main runs are NOT cancelled by policy (operator 2026-07-23): the per-commit verdict history is bisection evidence — the timeout IS the bound. PRIOR HISTORY: -> 270 restored 2026-07-12 on #6512 after 60m fail-fast regressed merge CI (receipt run 29197126623 @ 35f212fa5d: 60m step kill with 0 witness FAILs — batch-1 compile-clean PASS @ 15:01, batch-2 skip walk ~26m to recompute_trace_probe_test, then ~34m silent eval until kill; same serial-floor-wall class as 29183446733). Prior -> 60 operator ruling 2026-07-12: lower floor step cap from 270m so CI fails fast while the serial floor wall is debugged separately (receipt run 29183446733 @ 2e856a5617: 270m kill with 0 witness FAILs, batch-2 still in discovery SKIP walk). Prior -> 270 at PR #6464 receipt run 29151777611 (2026-07-11): 180m step cap still timed out mid batch 2 — 1535 SKIP rows finished @ 12:35:43, then ~147m silent witness-execution phase until the 180m kill @ 15:03:12 with batch 2 never completing (~177m in-batch from 12:06:38; ~443 non-skipped rows in roster of 1978). Prior -> 180 at run 29148344466 (~116m in-batch, ~87m post-skip). Prior -> 120 at run 29145270700. Prior -> 90 at run 29141663541. Prior -> 90 at body_lowering normalize hook (#6459 receipt run 29148735992 @ 3a372b7: floor step timed out at 60m MID discovery corpus after ~320 SKIP lines — batch-1 compile-clean normalize reconcile ~263s with body_lowering_fold in the normalize import closure; discovery still resolves every unique entry file before skip, so resolve->normalize pulled the ~1.3k-line scaffold into most closure walks). Structural fix on the same lane: NormalizedTree moved to v2.compiler.normalized_tree so v2.compiler.resolve no longer imports normalize (dissolve-on: skip-before-resolve so skipped rows never pay entry resolve). Prior step budget -> 60 at the discovery flip (gunbc.ci_spec ci_spec_discovery_flip_note; authored as 30 -> 60 on #6403, main had meanwhile bumped 30 -> 45 with the #6422 enrollments): the corpus discovery batch adds a whole-tree resolve plus the always-run live-tree rows to the floor step. Discovery corpus spawn_width_cap stays pinned to 1 (ci_corpus_discovery_spawn_width_cap in v2.workflow.ci_floor_plan): at width W the executor holds the parent's process-shared index PLUS W private shard indexes ((1+W) x whole-tree residency; width=2 OOM receipt run 28999086030), while width=1 runs rows on the main thread against the ONE shared index (union-resolve S1). CORRECTED RECEIPT (2026-07-10): run 29000557166's floor did NOT complete - the executor was host-OOM-killed ~8min in, MID discovery corpus (the log's later ExitSuccess belongs to the merge-admission STAMP tool, which stamped CI_FLOOR_EXIT=137); no flipped corpus has completed in CI yet - the only completion receipt is local (33.5GiB container, ~40min at width 5). The kill vector is host-level oversubscription (see gunbc_falsifier_plan_spawn_width_note), not this width model. #6475 receipt run 29143617420 @ 0c0f73: batch-1 compile-clean ~4m green; batch-2 discovery killed at 90m step cap mid-manual (last skip rust_wire_serde @ 07:47:49; ~15GiB peak). #6475 receipt run 29146814967 @ 6a61fce: same stall at 120m (last skip rust_wire_serde @ 09:34:49; ~85m silent eval). #6475 receipt run 29150477894 @ 6cd5326: same stall at 180m (last skip rust_wire_serde @ 11:43:46; ~88m silent eval) — local gunbc run on s1_closure_parses_holds reads 3/40 closure paths in 10m before timeout, matching the silent-eval class (live filesystem_read per path, not a skip-logged row). #6475 receipt run 29161709373 @ 9d9852c: batch-1 compile-clean refused — gunbc_ci_floor_step_timeout_discovery_flip_note string literal terminated early at col 2043 (body_lowering suffix spliced after closing quote during rebase merge); restored single-line literal on rebase to main #6464 note. Revisit down when floor memoization / resolver graph-major shrink the resolve wall; the affected-set selection receipts (skip counts per PR) are the cost dial to watch." data gunbc_ci_step_timeout_measure_grounding_disposition: Disposition = Scaffold { dissolves_to: SingleAuthority, diff --git a/dag/gunbc/commit_workflow.dag b/dag/gunbc/commit_workflow.dag index e54a0d05599..90e3fb8910a 100644 --- a/dag/gunbc/commit_workflow.dag +++ b/dag/gunbc/commit_workflow.dag @@ -8,7 +8,7 @@ import gunbc.ci_gate { SelfHostReadsRealBytesGate, SelfHostStalenessGate } import gunbc.generated_artifact { - GeneratedArtifact, CiYamlArtifact, GithooksPrePushArtifact, artifact_path + GeneratedArtifact, CiYamlArtifact, FalsifierYamlArtifact, GithooksPrePushArtifact, artifact_path } import std.disposition { Disposition, Scaffold, SingleAuthority, Terminal } import std.decl_ref { DeclarationRef, WholeDeclaration } @@ -28,6 +28,7 @@ type CommitWorkflowSurface = GithubActionsCiJob | GithubActionsCiRegenJob | GitPrePushHook + | FalsifierCadenceJob type CommitCheckKind = CommitSpecGate { gate: Gate } @@ -509,14 +510,60 @@ data commit_gate_roster: List = [ kind: ExecutionWitnessKind, span: SpanUndeclared }, + surfaces: [FalsifierCadenceJob] + }, + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/long/parse_binding_fidelity_witness_test.dag", + check_fns: ["witness_resolve_distinct_param_bindings_holds"], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, surfaces: [GithubActionsCiJob] }, CommitCheckEnrollment { check: CommitWitnessClaim { entry: "src/v2/test/claim/long/parse_binding_fidelity_witness_test.dag", + check_fns: ["witness_parse_binding_fidelity_distinct_identifiers_holds"], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, + surfaces: [FalsifierCadenceJob] + }, + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/long/affected_set_provenance_producer_witness_test.dag", check_fns: [ - "witness_parse_binding_fidelity_distinct_identifiers_holds", - "witness_resolve_distinct_param_bindings_holds" + "lens_affected_set_provenance_producer_single_read_emits_row_holds", + "lens_affected_set_provenance_producer_parse_coverage_known_diff_fail_closed_holds", + "lens_affected_set_provenance_producer_git_diff_wire_holds", + "lens_affected_set_provenance_producer_unknown_diff_fail_closed_holds" + ], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, + surfaces: [GithubActionsCiJob] + }, + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/long/affected_set_provenance_producer_witness_test.dag", + check_fns: [ + "lens_affected_set_provenance_producer_rejects_malformed_ingest_holds", + "lens_affected_set_provenance_producer_rejects_beyond_surface_ingest_holds", + "lens_affected_set_provenance_producer_coverage_receipt_holds" + ], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, + surfaces: [FalsifierCadenceJob] + }, + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/long/gauntlet_lane_enrollment_witness_test.dag", + check_fns: [ + "gauntlet_lane_rehomed_rows_are_cadence_enrolled_holds", + "gauntlet_lane_rehome_is_never_bare_deenrollment_holds", + "gauntlet_lane_rehome_bare_deenrollment_RED_control_holds" ], kind: ExecutionWitnessKind, span: SpanUndeclared @@ -550,6 +597,8 @@ fn commit_workflow_surface_eq(a: CommitWorkflowSurface, b: CommitWorkflowSurface match b { GithubActionsCiRegenJob => true _ => false } GitPrePushHook => match b { GitPrePushHook => true _ => false } + FalsifierCadenceJob => + match b { FalsifierCadenceJob => true _ => false } } } @@ -622,6 +671,49 @@ fn project_ci_floor_witness_entries(roster: List) -> List ) } +data falsifier_cadence_surface_note: String = "FalsifierCadenceJob is the SCHEDULED home for declared witness rows whose measured wall prices them out of the per-PR floor (batch-4 clamp FLOOR-BATCH-OVER-BUDGET, run 30148859947: wall_ms=613864 vs clamp_ms=420000 at units=74; authority gunbc.ci_spec gunbc_ci_floor_batch_clamp_params[3]). It extends the axis that ALREADY answers 'where does this check run' — the enrollment surface — rather than minting a second re-home authority beside gunbc.ci_layer_roots falsifier_rehomed_bin_wet_rows, which is specifically the bin_witness_wet_entries per-row-budget overflow (its constructor is bin_wet, its budget datum is bin_witness_wet_per_row_wall_budget_seconds) and does not model CommitWitnessClaim rows. Re-homing here is NEVER a bare de-enrollment: a row moved off GithubActionsCiJob must land on this surface, so it stays enrolled somewhere and its execution is receipted on the 4-hour cadence. The failure mode this guards is measured, not hypothetical — 63 *_test.dag files live under the two test/claim/long/ dirs, 9 are enrolled here, and the rest are enrolled NOWHERE (long/ is excluded from discovery at dir grain and the falsifier does not walk it), so 'deferred' has historically meant 'unscheduled'; this surface is the lane those rows enroll into. GRAIN: enrollment splits at the check_fn, not the file — an entry whose expensive fns re-home keeps its cheap fns on GithubActionsCiJob as the per-PR smoke grain (the mocks-layer-never-replace rule), so a re-home costs the corpus its wet grain per-PR, never the concern's per-PR coverage entirely. 🟡 dissolve-on: v2.workflow.ci_placement Placement (PrTier | Gauntlet) supersedes this surface tag as the witness-scheduling authority. It cannot today: PrTier is unwritable without a WarmCostReceipt { measured_ms, run_ref } AND measured_ms <= gunbc_ci_fast_lane_eval_budget_ms (5s), so wiring placement onto the 36 CommitWitnessClaim rows requires a per-row measured receipt for each — the D2 warm-cost probe on fleet srv hardware is the named filler (ci_placement_law_note), and it has not run. Fabricating those receipts would be the fail-open the placement law exists to forbid; defaulting the unmeasured rows to Gauntlet would de-enroll all 36 at once. Encouraging convergence receipt: the 5s law selects EXACTLY the twelve rows the run-30148859947 slow-witness tables measured over 5s (group A 141s/57s/15s/7s, group B 141s/29s/27s/14s/13s/13s/13s/13s) and admits every other batch-4 row at <=1s, so placement and this surface agree on today's roster — the wiring is blocked on receipts, not on a disputed boundary." + +data provenance_wall_cadence_rehome_risk: String = "ACCEPTED RISK, recorded where it was decided (operator ruling 2026-07-25). Three affected-set provenance walls — lens_affected_set_provenance_producer_rejects_malformed_ingest_holds, _rejects_beyond_surface_ingest_holds, _coverage_receipt_holds (14s + 13s + 13s measured, run 30148859947 group-B slow-witness table) — move from per-PR to the 4-hour falsifier cadence. THE COST: a selection-provenance regression now has up to a 24-hour detection window instead of one PR. THE CONTEXT THAT MAKES THIS UNCOMFORTABLE, stated rather than elided: this is accepted in the same week that exact class let #7196's stale slice control ride main green — the infer_semantics witness's transport (dag/tools/infer_semantics_witness_transport.dag) declares no dependency on v1_compiler_infer_access.rs, so the affected set could not see the edge and main's run at bb4347d skipped 1761 of 2373 witnesses, that one among them (fixed as a control by #7222; the SELECTION escape is untouched). WHAT KEEPS THIS FROM BEING A BARE NARROW: the four cheap fns of the same file stay per-PR (single_read_emits_row, parse_coverage_known_diff_fail_closed, git_diff_wire, unknown_diff_fail_closed — all <=161ms), so the producer keeps per-PR coverage at the fixture grain and loses only the wet grain; and gauntlet_lane_enrollment_witness_test proves by execution that these rows are cadence-enrolled rather than merely dropped. 🟡 dissolve-on — THE COMPENSATING CONTROL, which converts this from cadence-detection to construction: v2.lens.affected_set.declared_source_ref_selection today answers NoDeclaredSourceRefs => false in declared_source_refs_blocks_skip, i.e. a witness that declares NO source refs does not block its own skip — the precise fail-open #7196 rode. When the SourceRef child lands declared source-ref edges on witness transports AND that arm flips to refuse (NoDeclaredSourceRefs => true, an undeclared-provenance witness is never skippable), the wall is CONSTRUCTION rather than a witness watching for the breach, and these three rows can stay cadence-only permanently. Until that flip, this row is the standing record that per-PR detection was traded away deliberately, priced at 40s of a 194s clamp breach." + +fn project_falsifier_witness_entries(roster: List) -> List { + witness_entries_from_enrollments( + rows: filter(roster, e => enrollment_has_surface(e: e, surface: FalsifierCadenceJob)) + ) +} + +fn enrollment_is_witness_claim(e: CommitCheckEnrollment) -> Bool { + match e.check { + CommitWitnessClaim { entry: _, check_fns: _, kind: _, span: _ } => true + CommitSpecGate { gate: _ } => false + CommitCargoFmtCheck => false + } +} + +fn witness_enrollment_is_scheduled(e: CommitCheckEnrollment) -> Bool { + enrollment_has_surface(e: e, surface: GithubActionsCiJob) + || enrollment_has_surface(e: e, surface: FalsifierCadenceJob) +} + +data doc_reachability_cadence_enrollment_note: String = "FOUND BY THE WALL, on its first execution (2026-07-25). gauntlet_lane_rehome_is_never_bare_deenrollment_holds redded against the LIVE roster, not a synthetic fixture: the doc-reachability witness (doc_graph_has_no_orphan_docs, doc_graph_has_no_dangling_links) rode surfaces: [GitPrePushHook] alone — enforced only by a mechanism this repo has already ruled is NOT an enforcement authority (commit_gate_rust_suite_removed_disposition: opt-in per clone via core.hooksPath, bypassable with --no-verify, absent in container worktrees, and PROVEN ineffective — #6658 landed an unformatted .rs on main 2026-07-15 and nothing caught it). So orphaned docs and dangling doc links have had no wall on any CI path; they were one --no-verify from landing. FIXED by adding FalsifierCadenceJob rather than by widening witness_enrollment_is_scheduled to accept GitPrePushHook — widening the predicate until the failure disappears is the absorbing fallback §5 forbids, and it would have re-buried the finding in the same motion that surfaced it. The hook enrollment STAYS for fast local feedback; the cadence row is the authority. Kind is CorpusWitnessKind, not ExecutionWitnessKind: it reads the live doc graph, so per-PR admission would need a warm-cost receipt this change does not have — the same D2-probe gap falsifier_cadence_surface_note records. Promoting it to per-PR when that receipt exists is a strict improvement, not a requirement of this change." + +data bare_deenrollment_wall_note: String = "The construction wall behind falsifier_cadence_surface_note's 'a re-home is NEVER a bare de-enrollment'. A CommitWitnessClaim row must ride at least one EXECUTING witness surface — GithubActionsCiJob (per-PR floor) or FalsifierCadenceJob (4-hour cadence). GithubActionsCiRegenJob and GitPrePushHook do not count: the regen job projects GATES only (project_ci_regen_job_gates), and the pre-push hook is an escape hatch by the standing ruling (opt-in per clone, bypassable with --no-verify, absent in container worktrees — commit_gate_rust_suite_removed_disposition), so neither is an enforcement authority for a witness. A row satisfying neither is enrolled NOWHERE while still LOOKING enrolled, which is precisely the ~54-dark-file state test/claim/long/ is in and the shape a lazy re-home would add to. This is detection, not construction (the surface list is still writable as []) — the honest classification per DESIGN §5: it reds in the per-PR floor via gauntlet_lane_enrollment_witness_test with a synthetic RED control, and the construction form waits on the same D2-probe-gated placement wiring as falsifier_cadence_surface_note's dissolve-on, where Gauntlet-vs-PrTier is total over the coproduct and 'neither' becomes unwritable rather than merely counted." + +fn bare_deenrolled_witness_rows(roster: List) -> List { + filter(roster, e => enrollment_is_witness_claim(e: e) && !witness_enrollment_is_scheduled(e: e)) +} + +fn roster_has_no_bare_deenrolled_witness(roster: List) -> Bool { + bare_deenrolled_witness_rows(roster: roster).count() == 0 +} + +fn live_roster_has_no_bare_deenrolled_witness() -> Bool { + roster_has_no_bare_deenrolled_witness(roster: commit_gate_roster) +} + +fn live_roster_falsifier_witness_entries() -> List { + project_falsifier_witness_entries(roster: commit_gate_roster) +} + fn witness_seam_eq(a: WitnessSeam, b: WitnessSeam) -> Bool { (a.producer == b.producer) && (a.consumer == b.consumer) } @@ -740,6 +832,7 @@ fn realization_artifact_path(surface: CommitWorkflowSurface) -> String { GithubActionsCiJob => artifact_path(a: CiYamlArtifact) GithubActionsCiRegenJob => artifact_path(a: CiYamlArtifact) GitPrePushHook => artifact_path(a: GithooksPrePushArtifact) + FalsifierCadenceJob => artifact_path(a: FalsifierYamlArtifact) } } @@ -748,6 +841,7 @@ fn realization_handler_for_surface(surface: CommitWorkflowSurface) -> CommitWork GithubActionsCiJob => CiYamlHandler GithubActionsCiRegenJob => CiYamlHandler GitPrePushHook => GitPrePushScriptHandler + FalsifierCadenceJob => CiYamlHandler } } @@ -756,6 +850,7 @@ fn realization_artifact_for_surface(surface: CommitWorkflowSurface) -> Generated GithubActionsCiJob => CiYamlArtifact GithubActionsCiRegenJob => CiYamlArtifact GitPrePushHook => GithooksPrePushArtifact + FalsifierCadenceJob => FalsifierYamlArtifact } } diff --git a/dag/gunbc/design_document.dag b/dag/gunbc/design_document.dag index 30dbc8996d2..c4a2cc1e325 100644 --- a/dag/gunbc/design_document.dag +++ b/dag/gunbc/design_document.dag @@ -172,7 +172,7 @@ fn building_checks_blocks() -> List { ul(items: [ li(text: "`cargo test --workspace` · `cargo clippy --all-targets -- -D warnings` · `cargo fmt --all --check`"), li(text: "one-time: `git config core.hooksPath .githooks` (pre-push runs `cargo fmt`)"), - li(text: "CI (`gunbc ci` generates `.github/workflows/ci.yml`; all `v1-compiler` seed bins, no shell gate): **one** composed floor pass — `claim_executor --source-root src/v2 --source-root dag --plan-entry src/v2/workflow/ci_floor_plan.dag --plan-function gunbc_ci_floor_batches`. The v2 scheduler decides the batches from the single-authority spec (`gunbc.ci_spec`); the only structural fact the plan adds is *compile-clean gates the rest* (batch-1 `dag_compile_clean_gate` → batch-2 everything else, a dependency edge). Witness enrollment: **discovery shrunk by the affected set** (operator acceptance 2026-07-09, firing the 2026-07-04 opt-in inversion's dissolve-on — see `gunbc.ci_spec` `ci_spec_discovery_flip_note`): the corpus batch scans `CiSpec.discovery_scan_dirs` (plus the source-root `*_test.dag` walk) with `SelectionApplied`, so a PR runs the tree-wide witness corpus (~1,721 rows at flip time) shrunk to the diff's affected set; selection is fail-closed (a provenance gap refuses, never widens; host-scaffold/live-tree rows never predict-skip) and the scheduled `affected-set-falsifier` (every 4 hours) runs the corpus cold with predictions recorded, so a missing selection edge surfaces as a counted divergence within one cadence window. `CiSpec.witness_entries` (`CommitWitnessClaim` rows on the `GithubActionsCiJob` surface, projected by `project_ci_floor_witness_entries`) remains the explicit-entry roster — execution-kind rows with their own resource profile, and any row that must run as declared. Empty entries AND empty discovery dirs = zero witness-corpus nodes (the regen spec's shape). Naming hygiene stays fail-closed (a `test fn` outside `*_test.dag` is still a violation — the executor runs the zero-enrollment naming walk when no discovery batch is scheduled), and tree-wide unselected discovery remains the **local** path (`claim_batch --roster-from-discovery --source-root dag --source-root src/v2 --scan-dir dag/test/claim --scan-dir src/v2/test/claim/manual`). Batch-2 also carries the effectful gates: the rust fmt gate (when a `.rs` changes; nextest was removed from CI 2026-07-11 — operator ruling recorded in `gunbc.commit_workflow` `rust_tests_removed_disposition`, the suite runs locally only; clippy removed from CI 2026-07-08 — crate-wide `#![allow(clippy::all)]` made it zero-signal over ~44m/run, still available as a local dev check), emit-host MVP smokes, source-root-ingest, and the `ci.yml` drift+parse gate (ci.yml == `gunbc ci` output). The compile-clean gate is `--target dag`, and its per-PR scope is the same import-closure authority (2026-07-16, channel 2 of the 2026-07-10 grain fork): an all-`.dag`/docs diff compiles only the affected shard-entry closures (`tools.dag_compile_clean_scope`, host fast path `entry_file_touched_via_import_closure`); any non-selectable touched path (`.rs`, workflow yml, manifests), any non-docs departed path, or any selection refusal keeps the whole-tree baseline, loudly; the falsifier cadence carries the deterministic whole-tree cold control (`GUNBC_CI_COMPILE_CLEAN_COLD_CONTROL=1`, widen-only). The regen step (self-host fixed-point) is scoped the same way at its own closure (#6732 + this change's departed-path guard): `regen_floor_skip_witness` skips the pull_request step only when a non-empty merge-base diff is provably disjoint from the regen input set (`cli_run::regen_input_sources` — the SAME closure authority `regen_stage0` compiles — plus the src/v1/** prefix and Cargo/toolchain config); empty diffs, departed non-docs paths, and every failure arm run regen, and the skip is shell-gated to pull_request events so main pushes stay the unconditional cold control. Parse is grammar-owned: `ingest_yaml_source` (`dag/extdeps/languages/yaml/ingest.dag`), no shell/host parser."), + li(text: "CI (`gunbc ci` generates `.github/workflows/ci.yml`; all `v1-compiler` seed bins, no shell gate): **one** composed floor pass — `claim_executor --source-root src/v2 --source-root dag --plan-entry src/v2/workflow/ci_floor_plan.dag --plan-function gunbc_ci_floor_batches`. The v2 scheduler decides the batches from the single-authority spec (`gunbc.ci_spec`); the only structural fact the plan adds is *compile-clean gates the rest* (batch-1 `dag_compile_clean_gate` → batch-2 everything else, a dependency edge). Witness enrollment: **discovery shrunk by the affected set** (operator acceptance 2026-07-09, firing the 2026-07-04 opt-in inversion's dissolve-on — see `gunbc.ci_spec` `ci_spec_discovery_flip_note`): the corpus batch scans `CiSpec.discovery_scan_dirs` (plus the source-root `*_test.dag` walk) with `SelectionApplied`, so a PR runs the tree-wide witness corpus (~1,721 rows at flip time) shrunk to the diff's affected set; selection is fail-closed (a provenance gap refuses, never widens; host-scaffold/live-tree rows never predict-skip) and the scheduled `affected-set-falsifier` (every 4 hours) runs the corpus cold with predictions recorded, so a missing selection edge surfaces as a counted divergence within one cadence window. `CiSpec.witness_entries` (`CommitWitnessClaim` rows on the `GithubActionsCiJob` surface, projected by `project_ci_floor_witness_entries`) remains the explicit-entry roster — execution-kind rows with their own resource profile, and any row that must run as declared. Empty entries AND empty discovery dirs = zero witness-corpus nodes (the regen spec's shape). Naming hygiene stays fail-closed (a `test fn` outside `*_test.dag` is still a violation — the executor runs the zero-enrollment naming walk when no discovery batch is scheduled), and tree-wide unselected discovery remains the **local** path (`claim_batch --roster-from-discovery --source-root dag --source-root src/v2 --scan-dir dag/test/claim --scan-dir src/v2/test/claim/manual`). Batch-2 also carries the effectful gates: the rust fmt gate (when a `.rs` changes; nextest was removed from CI 2026-07-11 — operator ruling recorded in `gunbc.commit_workflow` `commit_gate_rust_suite_removed_disposition`, the suite runs locally only; clippy removed from CI 2026-07-08 — crate-wide `#![allow(clippy::all)]` made it zero-signal over ~44m/run, still available as a local dev check), emit-host MVP smokes, source-root-ingest, and the `ci.yml` drift+parse gate (ci.yml == `gunbc ci` output). The compile-clean gate is `--target dag`, and its per-PR scope is the same import-closure authority (2026-07-16, channel 2 of the 2026-07-10 grain fork): an all-`.dag`/docs diff compiles only the affected shard-entry closures (`tools.dag_compile_clean_scope`, host fast path `entry_file_touched_via_import_closure`); any non-selectable touched path (`.rs`, workflow yml, manifests), any non-docs departed path, or any selection refusal keeps the whole-tree baseline, loudly; the falsifier cadence carries the deterministic whole-tree cold control (`GUNBC_CI_COMPILE_CLEAN_COLD_CONTROL=1`, widen-only). The regen step (self-host fixed-point) is scoped the same way at its own closure (#6732 + this change's departed-path guard): `regen_floor_skip_witness` skips the pull_request step only when a non-empty merge-base diff is provably disjoint from the regen input set (`cli_run::regen_input_sources` — the SAME closure authority `regen_stage0` compiles — plus the src/v1/** prefix and Cargo/toolchain config); empty diffs, departed non-docs paths, and every failure arm run regen, and the skip is shell-gated to pull_request events so main pushes stay the unconditional cold control. Parse is grammar-owned: `ingest_yaml_source` (`dag/extdeps/languages/yaml/ingest.dag`), no shell/host parser."), ]), ] } diff --git a/dag/gunbc/non_fold_residue.dag b/dag/gunbc/non_fold_residue.dag index 3f6beccda84..830e6fa7b9e 100644 --- a/dag/gunbc/non_fold_residue.dag +++ b/dag/gunbc/non_fold_residue.dag @@ -54,6 +54,8 @@ data nfr_reason_native_agreement_octet: String = "RuntimeValue primitive-discrim data nfr_dissolve_native_agreement_octet: String = "RuntimeValue model↔realization grounds each primitive variant and the wildcard arm migrates to a total match — or the receipt formatter derives from inhabitance and the row deletes" +data nfr_reason_observation_hold_cause: String = "the CI-log renderer names the two memory-pressure holds (PsiPressure, CurrentHighWater — the crawl-window narrative) with a specific cause and gives the other five SchedulerHold variants a generic 'blocked on scheduler admission'; landed with the progress-and-observation stack (session/sleek-ibex-634, #7168) and surfaced by the whole-corpus nfr receipt after per-PR affected-set selection predict-skipped the corpus-read nfr witness at merge time (the masking class); declared so the ratchet re-arms" + data non_fold_residue_frontier: List = [ FrontierRow { subject: PathSubject { path: "dag/gunbc/local_tidy_spec.dag::local_tidy_path_matches_trigger" }, @@ -650,6 +652,11 @@ data non_fold_residue_frontier: List = [ reason: nfr_reason_ci_floor_gate_membership, trigger: TriggerProse { text: nfr_dissolve_gate_membership }, }, + FrontierRow { + subject: PathSubject { path: "dag/gunbc/observation_ci_render.dag::ci_hold_cause_text" }, + reason: nfr_reason_observation_hold_cause, + trigger: TriggerProse { text: nfr_dissolve_wildcard_total }, + }, ] fn non_fold_residue_frontier_units() -> List { diff --git a/dag/gunbc/observation_ci_render.dag b/dag/gunbc/observation_ci_render.dag new file mode 100644 index 00000000000..45a29646c0b --- /dev/null +++ b/dag/gunbc/observation_ci_render.dag @@ -0,0 +1,465 @@ +module gunbc.observation_ci_render + +import std.types { String, NonEmptyStr, Bool, List } +import std.nat { Nat } +import std.measure { + Millisecond, + ByteSize, + BasisPoint, + millisecond, + millisecond_count, + byte_size, + byte_size_count, + basis_point, + basis_point_count, + gibibyte_scale_factor_bytes, + milliseconds_per_second, + seconds_per_minute, +} +import std.symbols { SymbolId, Tier, Emoji, Unicode, Ascii, StatusPulse, StatusDwell, StatusBlocked, StatusRefused, StatusFinal, NodeCompleted, NodeFailed, NodeRunning, NodeSkipped } +import std.disposition { Disposition, Terminal } +import extdeps.render.glyphs { resolve_symbol } +import extdeps.github.log_annotations { annotation_group_prefix, annotation_endgroup, neutralize_workflow_commands } +import std.observation { + ObservationSubject, + ObservationSegment, + RunSegment, + BatchSegment, + EntrySegment, + ModuleSegment, + PhaseSegment, + observation_subject_render, + observation_subject_contains, + Measured, + MeasuredValue, + MeasuredUnavailable, + ObservationEvent, + ObservationOutcome, + Done, + Refused, + Failed, + TimedOut, + Skipped, + Final, + ObservationTransition, + Begin, + Step, + Concluded, + ObservationEventClass, + ClassRefused, + ClassBlocked, + ClassDwellEscalation, + observation_event_class, + observation_presentation, + observation_class_density, + ObservationDensity, + RoutineCollapsible, + SummaryAlwaysShown, + AnomalyExpanded, + observation_class_is_intrinsic_anomaly, + AttentionLevel, + Ambient, + Notable, + Anomaly, + AttentionBasis, + observation_attention, + DwellThreshold, + observation_dwell_threshold, + observation_escalation_depth, + observation_escalation_subject, + SchedulerHold, + PsiPressure, + CurrentHighWater, + observation_hold_resource, + ContendedResource, + MemoryBudget, + AdmissionWindow, + MaturationReserve, +} + +data observation_ci_render_note: String = "P1 of the progress-and-observation lane: the CI-log renderer, built first because the CI log is where the pain lives. Append-only, no repaint — a GitHub Actions log cannot be redrawn, so every line is final when written. This module PROJECTS the std.observation event vocabulary into lines; it computes no facts of its own and holds no telemetry source. Every number it prints arrives in an event or a heartbeat sample that the process already had (law 5), which is why the flagship acceptance can replay a captured run: the renderer is a pure function of the stream." + +data observation_ci_render_tone_note: String = "Tone contract (operator, 2026-07-24). Arm's-length lines are plain sentences a human reads without decoding — not dense key-equals-value chains, which belong in receipt boxes and files. Magnitudes are in human units: gibibytes, minutes, percent. The named negative example is the floor memory heartbeat's current shape, sixty-plus context-free byte dumps an hour, each carrying a raw current=16106717184 and no subject at all; that line is what the operator watched for ten minutes while a single module typechecked, and it is the exact shape this renderer must not reproduce." + +data ci_unit_authority_note: String = "The scale factors are CONSUMED from std.measure, never respelled here. A renderer carrying its own copy of how many bytes are in a gibibyte is a second unit authority hiding inside a formatting helper, and the fork is easy to miss precisely because it looks like arithmetic rather than modelling." + +data ci_one_decimal: Nat = 10 + +fn ci_gibibyte_tenths(b: ByteSize) -> Nat { + (byte_size_count(b) * ci_one_decimal) / gibibyte_scale_factor_bytes() +} + +fn ci_milliseconds_per_minute() -> Nat { + milliseconds_per_second() * seconds_per_minute() +} + +data ci_minute_switch_seconds: Nat = 90 + +data ci_minute_switch_note: String = "Where the sentence form switches from seconds to minutes is a DISPLAY policy, not a unit: ninety seconds reads better as ninety seconds than as one minute. It is named rather than buried as a literal so it stays visible as the judgement it is, distinct from the scale factors above, which are facts." + +fn ci_tenths_text(tenths: Nat) -> String { + concat(to_string(tenths / 10), concat(".", to_string(tenths % 10))) +} + +fn ci_human_bytes(b: ByteSize) -> String { + concat(ci_tenths_text(tenths: ci_gibibyte_tenths(b: b)), " GiB") +} + +fn ci_human_percent(bp: BasisPoint) -> String { + concat(ci_tenths_text(tenths: basis_point_count(bp: bp) / 10), "%") +} + +fn ci_human_duration(d: Millisecond) -> String { + let ms = millisecond_count(m: d) + if ms < milliseconds_per_second() { + concat(to_string(ms), "ms") + } else if ms < (ci_minute_switch_seconds * milliseconds_per_second()) { + concat(to_string(ms / milliseconds_per_second()), " seconds") + } else { + concat(to_string(ms / ci_milliseconds_per_minute()), " minutes") + } +} + +data ci_human_duration_note: String = "A second duration projection beside gunbc.ci_render.format_duration is deliberate and is not a fork: that one is the COMPACT technical form for receipt boxes and timing tables (230ms, 41s), this one is the SENTENCE form for arm's-length prose (10 minutes). Same fact, two surfaces, chosen by the tone contract rather than by taste. If a third caller wants a third spelling, that is the signal to unify them behind one carrier with a declared style axis rather than to add a fourth." + +fn ci_measured_bytes_text(m: Measured) -> String { + match m { + MeasuredValue { value: b } => ci_human_bytes(b: b) + MeasuredUnavailable { cause: c } => concat("unreadable (", concat(c, ")")) + } +} + +fn ci_measured_percent_text(m: Measured) -> String { + match m { + MeasuredValue { value: bp } => ci_human_percent(bp: bp) + MeasuredUnavailable { cause: c } => concat("unreadable (", concat(c, ")")) + } +} + +fn ci_glyph(id: SymbolId, tier: Tier) -> String { + match resolve_symbol(id: id, tier: tier) { + Present { value: g } => g + Absent => "?" + } +} + +type RenderedLine { + glyph: SymbolId + text: String + attention: AttentionLevel +} + +type LinePlacement + = InsideCollapsedGroup + | OutsideGroup + +data observation_placement_note: String = "Law 4's asymmetry made structural. Placement is DERIVED from the line's attention, never chosen at the call site, so an anomaly cannot be written into a collapsed group: the group is exactly where a reader will not look, and burying a refusal there is how a stopped line reads as silence. This is the construction that replaces the review instruction not to hide refusals." + +fn ci_line_placement(attention: AttentionLevel) -> LinePlacement { + match attention { + Ambient => InsideCollapsedGroup + Notable => OutsideGroup + Anomaly => OutsideGroup + } +} + +fn ci_render_line(line: RenderedLine, tier: Tier) -> String { + concat(ci_glyph(id: line.glyph, tier: tier), concat(" ", line.text)) +} + +fn ci_subject_leaf_text(subject: ObservationSubject) -> String { + let a = ci_subject_activity(subject: subject) + let detail = ci_subject_detail_text(subject: subject) + if detail != "" { + detail + } else if a.entry != "" { + a.entry + } else if a.batch != "" { + a.batch + } else if a.run != "" { + concat("run ", a.run) + } else { + "the run" + } +} + +data ci_leaf_text_note: String = "The deepest MEANINGFUL identity, not literally the last segment. A path ending in a phase segment renders as typecheck v2.compiler.normalized_tree rather than the bare word typecheck, because the phase alone names an activity without naming what it is being done to — which is the same half-answer the captured crawl window gave for ten minutes." + +type ObservationActivity { + run: String + batch: String + entry: String + module_path: String + phase: String +} + +data ci_activity_grain_note: String = "A subject path carries TWO grains a reader needs at once, and collapsing them loses the one that orients: the coarse phase the run is in (the batch label — witness discovery) and the fine activity inside it (typecheck of a named module). A projection that keeps only the deepest segment answers what is happening but not where, and one that keeps only the batch answers where but not what. The heartbeat sentence carries both because the crawl window proved both were missing." + +fn ci_subject_activity(subject: ObservationSubject) -> ObservationActivity { + fold( + subject.segments, + init: ObservationActivity { run: "", batch: "", entry: "", module_path: "", phase: "" }, + f: (acc, seg) => + match seg { + RunSegment { id: i } => + ObservationActivity { run: i, batch: acc.batch, entry: acc.entry, module_path: acc.module_path, phase: acc.phase } + BatchSegment { index: _, label: l } => + ObservationActivity { run: acc.run, batch: l, entry: acc.entry, module_path: acc.module_path, phase: acc.phase } + EntrySegment { source_path: p } => + ObservationActivity { run: acc.run, batch: acc.batch, entry: p, module_path: acc.module_path, phase: acc.phase } + ModuleSegment { module_path: m } => + ObservationActivity { run: acc.run, batch: acc.batch, entry: acc.entry, module_path: m, phase: acc.phase } + PhaseSegment { name: n } => + ObservationActivity { run: acc.run, batch: acc.batch, entry: acc.entry, module_path: acc.module_path, phase: n } + } + ) +} + +fn ci_subject_activity_text(subject: ObservationSubject) -> String { + let a = ci_subject_activity(subject: subject) + if a.batch != "" { + a.batch + } else if a.entry != "" { + a.entry + } else if a.run != "" { + concat("run ", a.run) + } else { + "the run" + } +} + +fn ci_subject_detail_text(subject: ObservationSubject) -> String { + let a = ci_subject_activity(subject: subject) + if a.module_path != "" && a.phase != "" { + concat(a.phase, concat(" ", a.module_path)) + } else if a.module_path != "" { + a.module_path + } else if a.phase != "" { + a.phase + } else { + "" + } +} + +fn ci_outcome_text(o: ObservationOutcome) -> String { + match o { + Done => "done" + Refused { diagnostic: d } => concat("refused: ", d) + Failed { error: e, output: _ } => concat("failed: ", e) + TimedOut { budget: b, elapsed: e } => + concat("timed out after ", concat(ci_human_duration(d: e), concat(" (budget ", concat(ci_human_duration(d: b), ")")))) + Skipped { reason: r } => concat("skipped: ", r) + Final { shown_failures: n } => concat("finished with ", concat(to_string(n), " problems shown")) + } +} + +fn ci_duration_suffix(wall: Measured) -> String { + match wall { + MeasuredValue { value: w } => concat(" in ", ci_human_duration(d: w)) + MeasuredUnavailable { cause: c } => concat(" (duration unmeasured: ", concat(c, ")")) + } +} + +data ci_duration_absence_note: String = "An unmeasured duration NAMES ITSELF rather than vanishing from the line. Dropping the clause would have been the model own fabrication rule broken by its first renderer: observation_measured_note says an absent measurement never prints a zero and never omits the field, because a silently omitted number reads as a fact that was never in question. A reader who sees no duration assumes nobody timed it; a reader who sees the cause knows why." + +data ci_outcome_duration_note: String = "Every outcome line carries its duration. The reference implementation studied for this lane omits them, which is why a reader cannot tell a fast green from a slow one without diffing timestamps by hand — and the slow green is the one that becomes next month's wall." + +fn ci_event_line(event: ObservationEvent, basis: AttentionBasis) -> RenderedLine { + let attention = observation_attention(event: event, basis: basis) + let presentation = observation_presentation(c: observation_event_class(event: event)) + let subject = ci_subject_leaf_text(subject: event.subject) + match event.transition { + Begin => + RenderedLine { glyph: presentation.glyph, text: concat("started ", subject), attention: attention } + Step { k: k, n: n } => + RenderedLine { + glyph: presentation.glyph, + text: concat(subject, concat(": ", concat(to_string(k), concat(" of ", to_string(n))))), + attention: attention + } + Concluded { outcome: o } => + RenderedLine { + glyph: presentation.glyph, + text: concat(subject, concat(" ", concat(ci_outcome_text(o: o), ci_duration_suffix(wall: event.wall)))), + attention: attention + } + } +} + +type HeartbeatSample { + elapsed: Millisecond + active: ObservationSubject + entry_index: Nat + entry_total: Nat + rss: Measured + swap: Measured + pressure: Measured +} + +data ci_heartbeat_identity_note: String = "Law 2: the heartbeat carries IDENTITY, not just vitals — and identity comes first in the sentence, vitals last. The ordering is the whole point. A reader scanning a wall of heartbeats needs to know WHAT is running; the memory numbers only matter once they know that. The captured crawl window is the counter-example in the record: sixty heartbeats that named no subject at all while one module typechecked for ten minutes, so the operator could see the process was alive and could not see what it was doing." + +fn ci_heartbeat_line(sample: HeartbeatSample) -> RenderedLine { + let detail = ci_subject_detail_text(subject: sample.active) + let position = concat(": entry ", concat(to_string(sample.entry_index), concat(" of ", to_string(sample.entry_total)))) + let identity = concat( + "still in ", + concat(ci_subject_activity_text(subject: sample.active), + concat(position, if detail == "" { "" } else { concat(", now ", detail) })) + ) + let vitals = concat( + "memory ", concat(ci_measured_bytes_text(m: sample.rss), + concat(", swap ", concat(ci_measured_bytes_text(m: sample.swap), + concat(", pressure ", ci_measured_percent_text(m: sample.pressure))))) + ) + RenderedLine { + glyph: StatusPulse, + text: concat(ci_human_duration(d: sample.elapsed), concat(" in — ", concat(identity, concat(". ", vitals)))), + attention: Ambient + } +} + +fn ci_hold_cause_text(hold: SchedulerHold) -> String { + match hold { + PsiPressure { avg10: bp } => + concat("blocked on memory reclaim (pressure ", concat(ci_human_percent(bp: bp), ")")) + CurrentHighWater { current: c, high_water: hw } => + concat("blocked on the memory high-water line (", concat(ci_human_bytes(b: c), concat(" of ", concat(ci_human_bytes(b: hw), ")")))) + _ => "blocked on scheduler admission" + } +} + +data ci_escalation_note: String = "Law 3 rendered: quiet past T names the child, past 2T the grandchild, recursively to the leaf. The escalation APPENDS — it never repaints, because a CI log cannot be redrawn — so the reader watches attention walk down the tree in real time. Depth comes from std.observation, not from a threshold invented here." + +fn ci_escalation_line(quiet_subject: ObservationSubject, active_leaf: ObservationSubject, quiet: Millisecond, t: DwellThreshold) -> RenderedLine { + let surfaced = observation_escalation_subject(quiet_subject: quiet_subject, active_leaf: active_leaf, quiet: quiet, t: t) + RenderedLine { + glyph: StatusDwell, + text: concat( + ci_subject_leaf_text(subject: surfaced), + concat(" has been quiet for ", concat(ci_human_duration(d: quiet), " — still working")) + ), + attention: Notable + } +} + +fn ci_escalation_blocked_line(quiet_subject: ObservationSubject, active_leaf: ObservationSubject, quiet: Millisecond, t: DwellThreshold, hold: SchedulerHold) -> RenderedLine { + let surfaced = observation_escalation_subject(quiet_subject: quiet_subject, active_leaf: active_leaf, quiet: quiet, t: t) + RenderedLine { + glyph: StatusBlocked, + text: concat( + ci_subject_leaf_text(subject: surfaced), + concat(" quiet for ", concat(ci_human_duration(d: quiet), concat(" — ", ci_hold_cause_text(hold: hold)))) + ), + attention: Anomaly + } +} + +data ci_escalation_exponents: List = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] + +type EscalationScan { + count: Nat + threshold: Nat +} + +data ci_escalation_emission_note: String = "The escalation law has two rates and they are not the same number. The REVEAL DEPTH grows linearly — quiet over T — because the tree is walked one level per threshold. The EMISSION POINTS double — T, 2T, 4T, 8T — so a window that stays quiet does not turn into a per-minute drumbeat saying the same thing; each successive line costs twice the silence to earn and carries strictly more information than the last. That is law 4 and law 3 agreeing rather than competing: attention escalates, but the log stays quiet at arm's length. The exponent roster is bounded by construction, so a window that never ends emits a bounded number of lines instead of growing without limit." + +fn ci_escalation_emission_count(quiet: Millisecond, t: DwellThreshold) -> Nat { + let q = millisecond_count(m: quiet) + let scanned = fold( + ci_escalation_exponents, + init: EscalationScan { count: 0, threshold: millisecond_count(m: t.period) }, + f: (acc, k) => + if acc.threshold <= q { + EscalationScan { count: acc.count + 1, threshold: acc.threshold * 2 } + } else { + acc + } + ) + scanned.count +} + +data ci_responsiveness_note: String = "The responsiveness law stated as a predicate the flagship can execute against a real captured window: a subject quiet past T must have produced at least one line naming it. The captured crawl window of run 30044816605 fails this by construction — ten minutes quiet, zero lines — which is what makes it the right acceptance fixture rather than a synthetic one. A planted silent phase reds this predicate; that is the RED control, and it is the same shape as the historical defect." + +fn ci_window_is_responsive(quiet: Millisecond, t: DwellThreshold) -> Bool { + if millisecond_count(m: quiet) <= millisecond_count(m: t.period) { + true + } else { + ci_escalation_emission_count(quiet: quiet, t: t) > 0 + } +} + +fn ci_group_open(title: String) -> String { + concat(annotation_group_prefix, title) +} + +fn ci_group_close() -> String { + annotation_endgroup +} + +fn ci_collapsed_summary(green_count: Nat) -> String { + concat("… ", concat(to_string(green_count), " more finished cleanly")) +} + +data ci_collapsed_summary_note: String = "Law 4's quiet half: sub-threshold work collapses to a count rather than a line each. The count is a projection of the events that were collapsed, so the summary is true by construction and the reader can always expand the group to see them." + +type FloorRunSummary { + entries_complete: Nat + entries_total: Nat + refusals: Nat + failures: Nat + wall: Millisecond +} + +fn ci_run_summary_glyph(summary: FloorRunSummary) -> SymbolId { + if summary.failures > 0 { + NodeFailed + } else if summary.refusals > 0 { + StatusRefused + } else { + NodeCompleted + } +} + +data ci_summary_glyph_note: String = "The run summary keeps Refused and Failed apart, because a summary that collapses them undoes at the last line exactly what the outcome sum spends the model establishing. A run that only refused stopped deliberately and its glyph says so; a run carrying any failure is broken, and breakage dominates the summary glyph because it is the stronger claim about the run. Both counts appear in the text either way, so the glyph chooses emphasis and never hides a number." + +fn ci_run_summary_line(summary: FloorRunSummary) -> RenderedLine { + let problems = summary.refusals + summary.failures + RenderedLine { + glyph: ci_run_summary_glyph(summary: summary), + text: concat( + to_string(summary.entries_complete), + concat(" of ", concat(to_string(summary.entries_total), + concat(" entries finished in ", concat(ci_human_duration(d: summary.wall), + concat(", ", concat(to_string(summary.refusals), + concat(" refused and ", concat(to_string(summary.failures), " failed")))))))) + ), + attention: if problems == 0 { Ambient } else { Anomaly } + } +} + +data ci_final_restate_note: String = "Refusals and failures are re-rendered at the end of the run as well as at the moment they happen. A GitHub Actions log is routinely truncated in the middle and is read from the bottom, so a refusal that appeared only at its own timestamp is a refusal the reader will not find. Restating it is not duplication of the FACT — it is the same event projected to a second position, which is what an append-only medium requires." + +fn ci_refusal_restate_line(subject: ObservationSubject, diagnostic: NonEmptyStr) -> RenderedLine { + RenderedLine { + glyph: StatusRefused, + text: concat(observation_subject_render(subject: subject), concat(" — ", neutralize_workflow_commands(text: diagnostic))), + attention: Anomaly + } +} + +data ci_subject_text_guard_note: String = "A restated diagnostic is text this repo relays from a SUBJECT process, so it goes through extdeps.github.log_annotations.neutralize_workflow_commands: a child whose captured stderr happens to begin with the workflow-command prefix would otherwise mint annotations in the parent run that no gunbc code authored. Consuming the existing guard rather than re-deriving it is the point — the neutralization rule is that module's fact, not this renderer's." + + +data ci_measurement_rss_note: String = "Measurement-feed projection (wiring flip): peak-RSS and cgroup samples are not progress Begin/Concluded pairs — they are Measured facts about the run. This line is Ambient, identity-first (the label), vitals in human GiB via ci_measured_bytes_text, never a raw byte dump. Placement consumers and receipts still read the typed facts; this is the log surface only." + +fn ci_measurement_rss_line(label: NonEmptyStr, rss: Measured) -> RenderedLine { + RenderedLine { + glyph: StatusPulse, + text: concat(label, concat(" — ", ci_measured_bytes_text(m: rss))), + attention: Ambient + } +} + +data observation_ci_render_disposition: Disposition = Terminal { + reason: "The CI-log renderer is Terminal, not a scaffold: append-only line projection is what the GitHub Actions medium IS, not an interim stand-in for a richer one. The TTY renderer is a sibling projection of the same carriers rather than this module's successor, and the dashboard is a third. What dissolves later is the hand-rolled emit sites this renderer replaces, each counted by the P3 census." +} diff --git a/dag/gunbc/observation_emit_census.dag b/dag/gunbc/observation_emit_census.dag new file mode 100644 index 00000000000..572c91d4454 --- /dev/null +++ b/dag/gunbc/observation_emit_census.dag @@ -0,0 +1,173 @@ +module gunbc.observation_emit_census + +import std.types { String, NonEmptyStr, Bool, List } +import std.nat { Nat } +import std.disposition { Disposition, Terminal, Scaffold, SingleAuthority } +import std.decl_ref { DeclarationRef, WholeDeclaration } + +data observation_emit_census_note: String = "P3 of the progress-and-observation lane: the census wall. Every place the floor emits a progress line today is either a PROJECTION of the observation event stream (migrated, P1/P2) or a COUNTED FRONTIER ROW carrying a reason and a dissolve-on — the same discipline the site subsumption lane uses for unthemed colours. This module is the census authority: the classification is a closed sum, so a site cannot be half-classified, and the roster below names the structured-tag emit families that exist in the seed today. The executable hygiene witness is bidirectional: it reds when a rostered marker vanishes AND when a known live tag family is missing from the roster, so growth after the snapshot cannot silently understate." + +data observation_emit_census_sequencing_note: String = "Sequencing, per the operator ruling and live-log review (run 30142403230): tagged migrations landed (floor-memory, gantt, governor, typecheck-attribution, measurement, shell with named-intent subjects; Ambient ShellTrace Suppressed at Normal). Post-census growth tags ([floor-drain], [gate-warm-cost], [receipt], [file], and #7205's [resolve-split]/[assembly-split]) are CountedFrontierSite rows so the roster cannot silently understate. Unmarked raw eprintln residue in claim_executor remains counted separately." + +type EmitSiteDisposition + = MigratedToObservation { via: NonEmptyStr } + | CountedFrontierSite { reason: NonEmptyStr, dissolve_on: NonEmptyStr } + +data emit_site_disposition_note: String = "A closed sum, so a site is exactly one of two things and never silently neither. MigratedToObservation names the projection that replaced the raw emit — the evidence the site now speaks the event vocabulary. CountedFrontierSite carries a reason it has not migrated yet and the trigger that will dissolve it, so the debt is countable and prioritizable rather than an open-ended intention. There is no third arm, which is the point: an emit site the census has not looked at cannot be represented here, so the roster's completeness is what the witness checks against the seed." + +type CensusedEmitSite { + marker: NonEmptyStr + source_file: NonEmptyStr + disposition: EmitSiteDisposition +} + +data floor_memory_site: CensusedEmitSite = CensusedEmitSite { + marker: "[floor-memory]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: MigratedToObservation { + via: "gunbc.observation_ci_render.ci_heartbeat_line (seed mirror render_heartbeat_line_mirror; HeartbeatFeed subject)" as NonEmptyStr + } +} + +data typecheck_attribution_site: CensusedEmitSite = CensusedEmitSite { + marker: "[typecheck-attribution]" as NonEmptyStr, + source_file: "src/v1/stage0/src/cli_run.rs" as NonEmptyStr, + disposition: MigratedToObservation { + via: "gunbc.observation_ci_render.ci_event_line (seed typecheck_begin_line / typecheck_concluded_line; mirror render_typecheck_*_line_mirror; ModuleSegment+PhaseSegment)" as NonEmptyStr + } +} + +data gantt_site: CensusedEmitSite = CensusedEmitSite { + marker: "[gantt]" as NonEmptyStr, + source_file: "src/v1/stage0/src/v1_rt.rs" as NonEmptyStr, + disposition: MigratedToObservation { + via: "gunbc.observation_ci_render.ci_event_line (seed phase_begin_line / phase_concluded_line; mirror render_phase_begin_line_mirror / render_phase_concluded_line_mirror; PhaseSegment)" as NonEmptyStr + } +} + +data governor_site: CensusedEmitSite = CensusedEmitSite { + marker: "[governor]" as NonEmptyStr, + source_file: "src/v1/stage0/src/memory_governor.rs" as NonEmptyStr, + disposition: MigratedToObservation { + via: "gunbc.observation_ci_render.ci_hold_cause_text (seed seed_governor_hold_line; mirror render_governor_hold_line_mirror; SchedulerHold/BlockedOn)" as NonEmptyStr + } +} + +data measurement_site: CensusedEmitSite = CensusedEmitSite { + marker: "[measurement]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: MigratedToObservation { + via: "gunbc.observation_ci_render.ci_measurement_rss_line (seed seed_peak_rss_line; mirror render_peak_rss_line_mirror; identity-first label, human GiB)" as NonEmptyStr + } +} + +data shell_site: CensusedEmitSite = CensusedEmitSite { + marker: "[shell]" as NonEmptyStr, + source_file: "src/v1/stage0/src/v1_interpreter.rs" as NonEmptyStr, + disposition: MigratedToObservation { + via: "gunbc.observation_ci_render.ci_event_line (seed shell_effect_begin_line / shell_effect_done_line / shell_effect_failed_line; mirror render_shell_effect_*_line_mirror; named-intent subject; Ambient ShellTrace Suppressed-at-Normal; Anomaly Failed via disposition alone)" as NonEmptyStr + } +} + +data floor_drain_site: CensusedEmitSite = CensusedEmitSite { + marker: "[floor-drain]" as NonEmptyStr, + source_file: "src/v1/stage0/src/cli_run.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "schedule-retention / typed-cache drain diagnostics born after the initial census snapshot — retention kit instrumentation, not yet an observation projection" as NonEmptyStr, + dissolve_on: "re-census into MigratedToObservation once drain events speak the observation vocabulary (or retire the tag into a named group summary)" as NonEmptyStr + } +} + +data gate_warm_cost_site: CensusedEmitSite = CensusedEmitSite { + marker: "[gate-warm-cost]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "per-gate warm-cost TSV probe lines — the instrument producing the fleet receipt basis, still a raw tag family" as NonEmptyStr, + dissolve_on: "fold the TSV emit into an observation MeasurementSegment / receipt projection and retire the bracket tag" as NonEmptyStr + } +} + +data receipt_site: CensusedEmitSite = CensusedEmitSite { + marker: "[receipt]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "floor resolve / batch-wall / materialization receipt lines — data, not outcomes; still bracket-tagged" as NonEmptyStr, + dissolve_on: "collapse into the floor-receipts observation group as Ambient MeasurementSegment / StatusPulse lines" as NonEmptyStr + } +} + +data file_trace_site: CensusedEmitSite = CensusedEmitSite { + marker: "[file]" as NonEmptyStr, + source_file: "src/v1/stage0/src/v1_interpreter.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "Filesystem transport host-effect traces — sibling of the shell-echo class, still raw-tagged" as NonEmptyStr, + dissolve_on: "same named-intent ObservationEvent path as shell (Filesystem.Read/Write/…) once shell grain is proven" as NonEmptyStr + } +} + +data resolve_split_site: CensusedEmitSite = CensusedEmitSite { + marker: "[resolve-split]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "per-entry resolve-stage timing split (#7205 / #6848 cost instrumentation) — load/parse/resolve/normalize/typecheck/parent_envs/assembly/ownership" as NonEmptyStr, + dissolve_on: "fold into a MeasurementSegment / StatusPulse receipt projection under the floor-receipts group" as NonEmptyStr + } +} + +data assembly_split_site: CensusedEmitSite = CensusedEmitSite { + marker: "[assembly-split]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "reconcile-assembly sub-split (#7205) — schedule/probe/registry/services/rewire/emit_info/residue timings that priced the type-export-set memo" as NonEmptyStr, + dissolve_on: "fold into the same MeasurementSegment receipt projection as resolve-split" as NonEmptyStr + } +} + +data observation_emit_roster: List = [ + floor_memory_site, + typecheck_attribution_site, + gantt_site, + governor_site, + measurement_site, + shell_site, + floor_drain_site, + gate_warm_cost_site, + receipt_site, + file_trace_site, + resolve_split_site, + assembly_split_site +] + +data observation_emit_roster_completeness_disposition: Disposition = Scaffold { + dissolves_to: SingleAuthority, + bind: DeclarationRef { + module_path: "gunbc.observation_emit_census", + decl_name: "observation_emit_roster", + field: WholeDeclaration + } +} + +data observation_emit_roster_completeness_note: String = "Roster is bidirectional: every rostered marker must still exist in the seed, AND every known live tag family born after the initial snapshot must appear on the roster (operator finding 3, run 30142403230). Frontier count = 6 (floor-drain, gate-warm-cost, receipt, file, resolve-split, assembly-split). Six families MigratedToObservation. The unmarked eprintln sites remain the per-print completeness scaffold." + +fn emit_site_is_frontier(site: CensusedEmitSite) -> Bool { + match site.disposition { + MigratedToObservation { via: _ } => false + CountedFrontierSite { reason: _, dissolve_on: _ } => true + } +} + +fn emit_site_dissolve_on(site: CensusedEmitSite) -> String { + match site.disposition { + MigratedToObservation { via: v } => v + CountedFrontierSite { reason: _, dissolve_on: d } => d + } +} + +fn observation_emit_frontier_count() -> Nat { + fold(observation_emit_roster, init: 0, f: (acc, site) => + if emit_site_is_frontier(site: site) { acc + 1 } else { acc }) +} + +data observation_emit_census_disposition: Disposition = Terminal { + reason: "The census MODEL is Terminal — a closed classification of emit sites is the wall's authority, not an interim stand-in. What is a scaffold is the ROSTER's completeness (declared above), because the site list grows when post-rewrite sites are enumerated. The model that classifies them does not change; only the rows do." +} diff --git a/dag/gunbc/observation_seed_render.dag b/dag/gunbc/observation_seed_render.dag new file mode 100644 index 00000000000..3e6861c65e5 --- /dev/null +++ b/dag/gunbc/observation_seed_render.dag @@ -0,0 +1,257 @@ +module gunbc.observation_seed_render + +import std.types { String, NonEmptyStr, Bool } +import std.nat { Nat } +import std.symbols { Tier, Emoji, Unicode, StatusBlocked } +import std.measure { millisecond, byte_size, basis_point } +import std.disposition { Disposition, Terminal } +import std.observation { + ObservationSubject, + BatchSegment, + PhaseSegment, + ModuleSegment, + MeasuredValue, + MeasuredUnavailable, + ObservationEvent, + Begin, + Concluded, + Done, + Failed, + AttentionBasis, + Anomaly, + SchedulerHold, + PsiPressure, + CurrentHighWater, +} +import gunbc.observation_ci_render { + ci_event_line, + ci_render_line, + ci_heartbeat_line, + ci_hold_cause_text, + ci_measurement_rss_line, + HeartbeatSample, + RenderedLine, +} + +data observation_seed_render_note: String = "The seed to .dag render boundary. The v1 seed (claim_executor and its siblings) emits progress lines, but the FORMAT is a single authority in gunbc.observation_ci_render — a raw eprintln in the seed forks that format the way a hand-rolled duration string forks std.measure. This module is the boundary the seed calls across, exactly as cli_run.install_output_policy calls output_policy.resolve_channel_policy: primitive arguments in, a rendered line out, no format logic on the seed side. It constructs the ObservationEvent the seed's occurrence corresponds to (a phase concluding is a Concluded event on a PhaseSegment subject) and projects it through the existing renderer, so the seed cannot drift from the model. It holds no telemetry source and computes no facts of its own; every field it fills came in as an argument the seed already had (observation law 5)." + +data observation_seed_render_basis_note: String = "The attention basis governs PLACEMENT (ambient collapses, an anomaly surfaces), not the rendered string — ci_render_line reads only the glyph and text. It is supplied here, not omitted, so a future placement consumer reads a real level rather than a fabricated one, and it is grounded in the clamp overhead the floor already models (the non-per-unit floor time gunbc.ci_spec budgets as gunbc_ci_floor_batch_clamp_params overhead_seconds) rather than a threshold invented in a formatting helper: a prelude phase completing inside the whole overhead budget is routine, so it reads Ambient. The seed passes that overhead in as overhead_ms — this module does not reach for it, keeping the module a pure projection." + +fn seed_tier(emoji: Bool) -> Tier { + if emoji { Emoji } else { Unicode } +} + +fn seed_phase_basis(overhead_ms: Nat) -> AttentionBasis { + AttentionBasis { average: millisecond(count: overhead_ms), maximum: millisecond(count: overhead_ms) } +} + +fn seed_phase_subject(phase: NonEmptyStr) -> ObservationSubject { + ObservationSubject { segments: [ PhaseSegment { name: phase } ] } +} + +data seed_phase_begin_note: String = "A phase Begin is the matched half of observation law 1 for phase-grain subjects. The floor prelude historically only printed Concluded (the mark fires when the phase ends); compile-path gantt marks already pair .begin/.done, so the seed boundary exposes both arms. Wall at Begin is unmeasured by construction — elapsed does not exist yet — and RSS stays on heartbeat/measurement (ci_event_line does not project event.rss). The compile-path seed renders through a Rust MIRROR of these fns rather than the interpreter: trace_mark runs inside compile itself, and calling back into the interpreter to render would recurse. The mirror is proven byte-equal to this oracle by the seed RED (same pattern as seed_heartbeat_line / render_heartbeat_line_mirror)." + +fn phase_begin_line(phase: NonEmptyStr, overhead_ms: Nat, emoji: Bool) -> String { + let event = ObservationEvent { + subject: seed_phase_subject(phase: phase), + transition: Begin, + wall: MeasuredUnavailable { cause: "begin has no elapsed yet" as NonEmptyStr }, + rss: MeasuredUnavailable { cause: "not sampled at a phase boundary" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +fn phase_concluded_line(phase: NonEmptyStr, elapsed_ms: Nat, overhead_ms: Nat, emoji: Bool) -> String { + let event = ObservationEvent { + subject: seed_phase_subject(phase: phase), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: elapsed_ms) }, + rss: MeasuredUnavailable { cause: "not sampled at a phase boundary" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +data seed_heartbeat_unreadable_cause: NonEmptyStr = "cgroup field unreadable" as NonEmptyStr + +data seed_heartbeat_note: String = "The floor-memory heartbeat's projection. The heartbeat runs on a detached liveness thread in a memory-constrained context, so the seed renders it through a Rust MIRROR of ci_heartbeat_line rather than calling the interpreter there — an interpreter render on that thread would build a duplicate module index, consuming the very memory the heartbeat watches (DESIGN section 2), and the thread's whole point is to stay alive when the main interpreter is busy. This fn is the ORACLE that keeps the mirror honest: the seed's Rust test runs this (through the interpreter) and its mirror on the same samples and asserts byte-equality, so the format authority stays here in the one renderer and any drift reds. The subject is batch-grain by construction — the floor walks entries in parallel, so there is no single active module to name; the primitive interface carries no module or phase, which is why the mirror is a COMPLETE reflection of this fn over the seed's real input space, never a fabricated per-module detail (observation law 2: identity, never invented). Wired (4b): claim_executor.render_heartbeat_line_mirror + HeartbeatFeed." + +fn seed_heartbeat_line( + elapsed_ms: Nat, + batch_index: Nat, + batch_label: NonEmptyStr, + entry_index: Nat, + entry_total: Nat, + rss_bytes: Nat, + rss_available: Bool, + swap_bytes: Nat, + swap_available: Bool, + pressure_bp: Nat, + pressure_available: Bool, + emoji: Bool +) -> String { + ci_render_line( + line: ci_heartbeat_line(sample: HeartbeatSample { + elapsed: millisecond(count: elapsed_ms), + active: ObservationSubject { segments: [ BatchSegment { index: batch_index, label: batch_label } ] }, + entry_index: entry_index, + entry_total: entry_total, + rss: if rss_available { MeasuredValue { value: byte_size(rss_bytes) } } else { MeasuredUnavailable { cause: seed_heartbeat_unreadable_cause } }, + swap: if swap_available { MeasuredValue { value: byte_size(swap_bytes) } } else { MeasuredUnavailable { cause: seed_heartbeat_unreadable_cause } }, + pressure: if pressure_available { MeasuredValue { value: basis_point(count: pressure_bp) } } else { MeasuredUnavailable { cause: seed_heartbeat_unreadable_cause } } + }), + tier: seed_tier(emoji: emoji) + ) +} + +data seed_governor_hold_note: String = "Governor hold projection (wiring flip): HoldReason is already lockstep with SchedulerHold; the emit side now projects through ci_hold_cause_text rather than a bespoke [governor] printer. Attention is Anomaly (a hold is never Ambient). The seed renders through a Rust MIRROR in memory_governor — the governor is host physics without a source-root resolve context, same class as the heartbeat mirror. Oracle RED: seed_psi_hold_line / seed_high_water_hold_line byte-equal to render_governor_hold_line_mirror." + +fn seed_governor_hold_line(hold: SchedulerHold, emoji: Bool) -> String { + ci_render_line( + line: RenderedLine { + glyph: StatusBlocked, + text: ci_hold_cause_text(hold: hold), + attention: Anomaly + }, + tier: seed_tier(emoji: emoji) + ) +} + +fn seed_psi_hold_line(avg10_bp: Nat, emoji: Bool) -> String { + seed_governor_hold_line( + hold: PsiPressure { avg10: basis_point(count: avg10_bp) }, + emoji: emoji + ) +} + +fn seed_high_water_hold_line(current_bytes: Nat, high_water_bytes: Nat, emoji: Bool) -> String { + seed_governor_hold_line( + hold: CurrentHighWater { + current: byte_size(current_bytes), + high_water: byte_size(high_water_bytes) + }, + emoji: emoji + ) +} + +data seed_typecheck_attribution_note: String = "Per-module typecheck attribution (wiring flip): a Concluded event on ModuleSegment+PhaseSegment\{typecheck\} carrying the measured wall, rendered by ci_event_line — written as the module concludes rather than batched to walk end. The 2s threshold that kept the floor log quiet is preserved at the emit site (pathology lane only). Hot-path mirror in cli_run (interpreter render per module under resolve would dominate the wall); oracle RED keeps the mirror byte-equal to this seed." + +fn seed_typecheck_subject(module_path: NonEmptyStr) -> ObservationSubject { + ObservationSubject { + segments: [ + ModuleSegment { module_path: module_path }, + PhaseSegment { name: "typecheck" as NonEmptyStr } + ] + } +} + +fn typecheck_begin_line(module_path: NonEmptyStr, overhead_ms: Nat, emoji: Bool) -> String { + let event = ObservationEvent { + subject: seed_typecheck_subject(module_path: module_path), + transition: Begin, + wall: MeasuredUnavailable { cause: "begin has no elapsed yet" as NonEmptyStr }, + rss: MeasuredUnavailable { cause: "not sampled at a typecheck boundary" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +fn typecheck_concluded_line(module_path: NonEmptyStr, elapsed_ms: Nat, overhead_ms: Nat, emoji: Bool) -> String { + let event = ObservationEvent { + subject: seed_typecheck_subject(module_path: module_path), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: elapsed_ms) }, + rss: MeasuredUnavailable { cause: "not sampled at a typecheck boundary" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +data seed_measurement_unreadable_cause: NonEmptyStr = "no /proc/self/status" as NonEmptyStr + +data seed_measurement_note: String = "Measurement-feed projection (wiring flip): peak-RSS samples become ci_measurement_rss_line — identity-first label, human GiB, MeasuredUnavailable names its cause. Hot-path mirrors in claim_executor / claim_batch (same class as heartbeat). The typed receipt consumers are unchanged; only the log dialect migrates." + +fn seed_peak_rss_line(label: NonEmptyStr, rss_bytes: Nat, rss_available: Bool, emoji: Bool) -> String { + ci_render_line( + line: ci_measurement_rss_line( + label: label, + rss: if rss_available { + MeasuredValue { value: byte_size(rss_bytes) } + } else { + MeasuredUnavailable { cause: seed_measurement_unreadable_cause } + } + ), + tier: seed_tier(emoji: emoji) + ) +} + +data seed_shell_effect_note: String = "Shell host-effect projection (operator live-log 2026-07-25 acceptance): subjects are NAMED INTENTS — the typed service.op key (`git.Core.HeadCommit`, `shell.Env.Get`), never raw `$ argv`. Argv is demoted to Failed.error (`$ (exit=N)`, whitespace-collapsed) and the Verbose/title channel. Routine Begin/Concluded\{Done\} are Ambient (ShellTrace-gated; Suppressed at Normal so scaffolding collapses); failing Concluded\{Failed\} is Anomaly (OutsideGroup by law 4), gated by divergence disposition alone — channel Suppressed must never silence a failure. Hot-path mirrors in v1_interpreter; oracle RED keeps them honest." + +fn seed_shell_subject(intent: NonEmptyStr) -> ObservationSubject { + ObservationSubject { segments: [ PhaseSegment { name: intent } ] } +} + +fn shell_effect_begin_line(intent: NonEmptyStr, overhead_ms: Nat, emoji: Bool) -> String { + let event = ObservationEvent { + subject: seed_shell_subject(intent: intent), + transition: Begin, + wall: MeasuredUnavailable { cause: "begin has no elapsed yet" as NonEmptyStr }, + rss: MeasuredUnavailable { cause: "not sampled at shell spawn" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +fn shell_effect_done_line(intent: NonEmptyStr, elapsed_ms: Nat, overhead_ms: Nat, emoji: Bool) -> String { + let event = ObservationEvent { + subject: seed_shell_subject(intent: intent), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: elapsed_ms) }, + rss: MeasuredUnavailable { cause: "not sampled at shell completion" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +fn shell_effect_failed_line( + intent: NonEmptyStr, + argv_collapsed: NonEmptyStr, + exit_code: Nat, + elapsed_ms: Nat, + overhead_ms: Nat, + emoji: Bool +) -> String { + let error = concat( + "$ ", + concat(argv_collapsed, concat(" (exit=", concat(to_string(exit_code), ")"))) + ) as NonEmptyStr + let event = ObservationEvent { + subject: seed_shell_subject(intent: intent), + transition: Concluded { + outcome: Failed { error: error, output: "" } + }, + wall: MeasuredValue { value: millisecond(count: elapsed_ms) }, + rss: MeasuredUnavailable { cause: "not sampled at shell completion" as NonEmptyStr } + } + ci_render_line( + line: ci_event_line(event: event, basis: seed_phase_basis(overhead_ms: overhead_ms)), + tier: seed_tier(emoji: emoji) + ) +} + +data observation_seed_render_disposition: Disposition = Terminal { + reason: "The seed to .dag render boundary is Terminal, not a scaffold: a boundary the seed calls across to keep the format single-authority is the mechanism, not an interim stand-in. What dissolves is each raw eprintln the seed replaces with a call here — counted by gunbc.observation_emit_census — and, further out, the seed itself as v2 realizes the witnesses natively; neither changes what this module models, which is that a seed occurrence is an observation event projected by the one renderer." +} diff --git a/dag/gunbc/observation_tty_render.dag b/dag/gunbc/observation_tty_render.dag new file mode 100644 index 00000000000..1f9fd4fd035 --- /dev/null +++ b/dag/gunbc/observation_tty_render.dag @@ -0,0 +1,176 @@ +module gunbc.observation_tty_render + +import std.types { String, NonEmptyStr, Bool, List } +import std.nat { Nat } +import std.measure { Millisecond, ByteSize, millisecond, millisecond_count } +import std.symbols { SymbolId, Tier, Emoji, Unicode, Ascii, Spinner0, StatusBlocked, StatusFinal, NodeRunning, NodeCompleted } +import std.render { CursorAction, Overwrite, Append, RenderCapability } +import std.disposition { Disposition, Terminal } +import extdeps.render.glyphs { resolve_symbol, reward_animal_at } +import gunbc.observation_ci_render { + RenderedLine, + ci_render_line, + ci_event_line, + ci_human_duration, + ci_human_bytes, + ci_subject_activity_text, + ci_subject_detail_text, + ci_subject_leaf_text, + ci_hold_cause_text, +} +import std.observation { + ObservationSubject, + ObservationEvent, + ObservationTransition, + Begin, + Step, + Concluded, + ObservationOutcome, + Done, + Refused, + Failed, + TimedOut, + Skipped, + Final, + ObservationEventClass, + observation_event_class, + observation_class_density, + ObservationDensity, + RoutineCollapsible, + SummaryAlwaysShown, + AnomalyExpanded, + observation_presentation, + AttentionLevel, + Ambient, + Notable, + Anomaly, + AttentionBasis, + observation_attention, + BlockedOn, + ContendedResource, + MemoryBudget, + AdmissionWindow, + MaturationReserve, + ContentionRemaining, + RemainingUnknown, + RemainingBounded, + SchedulerHold, + PsiPressure, +} + +data observation_tty_render_note: String = "P2 of the progress-and-observation lane: the interactive TTY renderer, a SIBLING projection of the same std.observation carriers the CI renderer projects — not a second model and not the CI renderer's successor. The two differ only where the medium differs. A terminal can be repainted, so routine progress overwrites one line in place and stays quiet; a CI log cannot, so it appends. Everything the two share — the event vocabulary, the density authority, the glyph table, the duration and byte and percent projections, the hold-cause text — is imported from the CI renderer or the model, never re-derived. That sharing is the proof that one event stream drives N renderers (DESIGN section 2); a TTY renderer that re-spelled ci_human_duration would be the fork this lane exists to avoid." + +data observation_tty_upgrades_note: String = "The three upgrades over the reference implementation studied for this lane, all inherited from the model rather than re-earned here: outcome lines carry durations (ci_event_line already appends them), Refused is distinct from Failed (the outcome sum and the glyph authority keep them apart), and dwell escalation is recursive rather than one-level (observation_escalation_subject walks the whole tree). The reference has none of these; the TTY renderer gets them for free by projecting the same carriers." + +type PreambleFields { + run_id: NonEmptyStr + trigger: NonEmptyStr + diff_summary: NonEmptyStr + entry_total: Nat +} + +data observation_tty_preamble_note: String = "The preamble is REQUIRED — no anonymous process. The reference implementation makes this a construction rule (a titled, described process or nothing), and it is kept: a TTY run opens with what is running, why it was triggered, and how much work it faces, so the first thing on screen answers the question the crawl window could not. The diff-to-runs selection summary is the centre of it (section 6b), but its declaration-grain form rides the namespace lane; this carrier holds the fields that exist today and grows without a format change when decl-grain selection lands." + +fn tty_preamble_line(fields: PreambleFields) -> RenderedLine { + RenderedLine { + glyph: NodeRunning, + text: concat( + "run ", concat(fields.run_id, + concat(" — ", concat(fields.trigger, + concat(" — ", concat(fields.diff_summary, + concat(" — ", concat(to_string(fields.entry_total), " entries to run"))))))) + ), + attention: Notable + } +} + +data observation_tty_repaint_note: String = "Whether a line repaints in place or scrolls is DERIVED from its attention, one decision function, never chosen at the call site. Routine progress overwrites — the spinner and the current-entry counter update on one line so the terminal does not fill with green. An anomaly appends, because it must survive on screen and be scrollable back to; overwriting a refusal would erase it the instant the next line arrived, which is the terminal form of burying it in a collapsed group. The summary appends too — it is the last thing the reader wants to keep. This is the same law-4 asymmetry the CI renderer expresses through group placement, expressed here through cursor action, because that is what the medium offers." + +fn tty_cursor_action(attention: AttentionLevel, cap: RenderCapability) -> CursorAction { + if cap.cursor_addressable { + match attention { + Ambient => Overwrite + Notable => Append + Anomaly => Append + } + } else { + Append + } +} + +data observation_tty_group_note: String = "The live group line is a repainting projection of the currently-running subject: a spinner, the phase, and the k-of-n counter, overwritten each tick. It names the current entry the way the reference's group line does, and it carries the same identity-first discipline as the heartbeat — the spinner says alive, the text says what is alive." + +fn tty_group_line(active: ObservationSubject, k: Nat, n: Nat, spinner: SymbolId) -> RenderedLine { + let detail = ci_subject_detail_text(subject: active) + RenderedLine { + glyph: spinner, + text: concat( + ci_subject_activity_text(subject: active), + concat(" ", concat(to_string(k), concat(" of ", concat(to_string(n), + if detail == "" { "" } else { concat(" — ", detail) })))) + ), + attention: Ambient + } +} + +data observation_tty_blocked_inline_note: String = "BlockedOn renders INLINE on the TTY, with its holder and its remaining bound when the governor has one — the reference's contention feature, kept and upgraded to derived facts. A bounded remaining prints the time; an unknown remaining prints why it is unknown rather than a fabricated estimate, because a made-up ETA is the same fabrication the model forbids for any other measurement." + +fn tty_remaining_text(remaining: ContentionRemaining) -> String { + match remaining { + RemainingBounded { by: b } => concat("about ", concat(ci_human_duration(d: b), " remaining")) + RemainingUnknown { cause: c } => concat("no estimate (", concat(c, ")")) + } +} + +fn tty_blocked_line(blocked: BlockedOn) -> RenderedLine { + let holders = concat(to_string(count(blocked.holders)), " holder(s)") + RenderedLine { + glyph: StatusBlocked, + text: concat( + "blocked — ", + concat(holders, concat(", ", tty_remaining_text(remaining: blocked.remaining))) + ), + attention: Anomaly + } +} + +data observation_tty_reward_note: String = "The reward animal renders on Final — the reference's flourish, kept, and kept CORRECTLY: it is drawn from the reward-animal rows in the one glyph authority, selected deterministically by the run's identity, never a random draw. A random animal would make the event stream unreplayable, which would defeat the same replay the flagship depends on. On a non-emoji terminal the row's ascii form renders, so the flourish degrades to a word rather than a mojibake box." + +fn tty_final_line(shown_failures: Nat, run_index: Int, tier: Tier) -> RenderedLine { + let animal = match reward_animal_at(index: run_index, tier: tier) { + Present { value: a } => a + Absent => "" + } + RenderedLine { + glyph: StatusFinal, + text: if shown_failures == 0 { + concat("all clean ", animal) + } else { + concat(to_string(shown_failures), " problems shown above") + }, + attention: if shown_failures == 0 { Ambient } else { Anomaly } + } +} + +data observation_tty_event_note: String = "An event line on the TTY is the SAME projection the CI renderer produces — ci_event_line, imported, not re-implemented — paired with the cursor action its attention derives. The renderer differs from its CI sibling in HOW a line reaches the screen (overwrite vs append), never in WHAT the line says. That is the single-stream property made concrete: change the event and both surfaces move together, because both call the same projection." + +type TtyEmission { + line: RenderedLine + cursor: CursorAction +} + +fn tty_event_emission(event: ObservationEvent, basis: AttentionBasis, cap: RenderCapability) -> TtyEmission { + let line = ci_event_line(event: event, basis: basis) + TtyEmission { + line: line, + cursor: tty_cursor_action(attention: line.attention, cap: cap) + } +} + +fn tty_render_emission(emission: TtyEmission, tier: Tier) -> String { + ci_render_line(line: emission.line, tier: tier) +} + +data observation_tty_render_disposition: Disposition = Terminal { + reason: "The TTY renderer is Terminal — an in-place-repaint projection is what an interactive terminal IS. It is a peer of the CI-log renderer and the later dashboard, all projecting one event stream; none is a scaffold for another. What dissolves is the hand-rolled TTY progress code the P3 census will enumerate, not this projection." +} diff --git a/dag/gunbc/output_policy.dag b/dag/gunbc/output_policy.dag index 074d7de12bc..52d8cd526d9 100644 --- a/dag/gunbc/output_policy.dag +++ b/dag/gunbc/output_policy.dag @@ -53,7 +53,7 @@ fn channel_decision(channel: OutputChannel, verbosity: Verbosity) -> OutputDecis } ShellTrace => match verbosity { Quiet => Suppressed - Normal => Condensed + Normal => Suppressed Verbose => Full } Instrumentation => match verbosity { @@ -104,7 +104,9 @@ fn is_condensed(channel: OutputChannel, verbosity: Verbosity) -> Bool { data effect_expectation_note: String = "channel_decision above answers how loud a CHANNEL is. It cannot answer whether a given effect's captured streams are worth surfacing, because that needs a fact it was never given: what the caller EXPECTED. Before this row 'failure' was hand-coded downstream as exit != 0 — an untyped proxy that silently assumes every effect expects success (the empty set). Naming the expectation makes the rule one line: DIVERGENCE SURFACES, AGREEMENT COUNTS, where failure means observed differs from declared-expected at the grain that declared it. That closes two leaks in one dispatch. The noisy one: a red-control child failing exactly as declared is AGREEMENT, so its stderr is counted, not replayed — and a subject stderr carrying '::error::' stops minting annotations in the parent run. The silent one, which nobody has hit yet and which is the real payoff: a red control that SUCCEEDS (the wall failing to fire) is divergence, and today logs nothing at all — the single most diagnosis-worthy event in the system is currently the quietest. There is deliberately no ExpectAny arm: an effect whose expectation is unknown would make every observation agree, restoring the empty set (DESIGN 5 — a failure arm must refuse, never widen)." -type ExpectedOutcome = ExpectSuccess | ExpectFailure +data outcome_is_data_note: String = "OutcomeIsData is the third arm (operator ruling 2026-07-25), and it exists because ExpectSuccess|ExpectFailure mis-models a whole class of site: a PROBE whose exit code is an ANSWER, not a verdict. `test -x` returning 1 means 'subject absent' — ambient in BOTH contexts, because production's ensure-bins responds by building and a witness's discriminating arm responds by asserting. The effect layer's anomaly glyph there is a second, cruder judgment of an event whose real adjudicator is typed logic downstream, so the honest disposition is: render ambient regardless of exit; only DISPATCH failure (the probe itself could not run) is anomalous. That distinction — subject-absent versus probe-broken — is the DaemonUnsupervised lesson applied to exit codes. Loudness does not disappear under this arm, it MOVES to the layer holding the context, which is this model's whole thesis. It is annotatable on a shared helper's own call node, once, correct for every caller — which is why it dissolves the apparent trilemma between threading a test concern through production signatures and an ambient frame that can be left open. ADMISSIBILITY GUARD (§5, what stops this becoming a silence-anything sticker): OutcomeIsData is admissible ONLY where the exit provably feeds a typed consumer — the annotated helper returns a typed observation or verdict, never unit. Both clusters it was ruled for qualify today: witness_bin_artifact_readiness returns its WitnessBinReadiness arms, and belt_actuate_spawn_member's exit feeds the belt's per-variant status mapping. The guard is checkable, so it belongs in the wiring test and eventually a lens; until that lands it is a REVIEW obligation, not a mechanical one, and this note is where the criterion is written down." + +type ExpectedOutcome = ExpectSuccess | ExpectFailure | OutcomeIsData type ObservedOutcome = ObservedSuccess | ObservedFailure { exit: Int } @@ -130,6 +132,10 @@ fn outcome_diverges(expected: ExpectedOutcome, observed: ObservedOutcome) -> Boo ObservedSuccess => true ObservedFailure { exit: exit } => false } + OutcomeIsData => match observed { + ObservedSuccess => false + ObservedFailure { exit: exit } => false + } } } @@ -141,7 +147,7 @@ fn divergence_disposition(diverges: Bool) -> StreamDisposition { } } -data effect_stream_dispatch_note: String = "Verbosity and divergence compose rather than compete: Quiet suppresses the stream whatever the outcome, Verbose surfaces it whatever the outcome (the operator asked for everything), and the divergence rule decides the Normal middle — which is where CI runs. So the channel's existing authority is consumed, not re-derived, and the new axis only refines the arm that was previously fixed." +data effect_stream_dispatch_note: String = "Observation dissolve (operator live-log 2026-07-25): Ambient ShellTrace silence at Normal must not force StreamSuppressed — Anomaly Failed still surfaces via divergence alone. Quiet no longer swallows divergence either (DESIGN §5: a failing effect is never silent). Verbose still surfaces every stream (Full). The channel grades Ambient scaffolding; the stream policy grades subject content; they compose rather than conflate." fn effect_stream_disposition( channel: OutputChannel, @@ -150,7 +156,7 @@ fn effect_stream_disposition( observed: ObservedOutcome ) -> StreamDisposition { match channel_decision(channel: channel, verbosity: verbosity) { - Suppressed => StreamSuppressed + Suppressed => divergence_disposition(diverges: outcome_diverges(expected: expected, observed: observed)) Condensed => divergence_disposition(diverges: outcome_diverges(expected: expected, observed: observed)) Full => SurfaceContent } @@ -173,6 +179,8 @@ type EffectStreamPolicy { expect_success_observed_failure: StreamDisposition expect_failure_observed_success: StreamDisposition expect_failure_observed_failure: StreamDisposition + outcome_is_data_observed_success: StreamDisposition + outcome_is_data_observed_failure: StreamDisposition subject_line_guard: String } @@ -194,6 +202,14 @@ fn shell_trace_stream_policy(verbosity: Verbosity) -> EffectStreamPolicy { channel: ShellTrace, verbosity: verbosity, expected: ExpectFailure, observed: observed_outcome_from_exit(exit: 1) ) + outcome_is_data_observed_success: effect_stream_disposition( + channel: ShellTrace, verbosity: verbosity, + expected: OutcomeIsData, observed: ObservedSuccess + ) + outcome_is_data_observed_failure: effect_stream_disposition( + channel: ShellTrace, verbosity: verbosity, + expected: OutcomeIsData, observed: observed_outcome_from_exit(exit: 1) + ) subject_line_guard: subject_text_line_guard } } @@ -201,3 +217,5 @@ fn shell_trace_stream_policy(verbosity: Verbosity) -> EffectStreamPolicy { fn resolve_shell_trace_stream_policy(verbose: Bool, quiet: Bool) -> EffectStreamPolicy { shell_trace_stream_policy(verbosity: resolve_verbosity(env: VerbosityEnv { verbose: verbose, quiet: quiet })) } + +data shell_trace_observation_convergence_note: String = "DISSOLVED (operator live-log 2026-07-25, run 30142403230): shell host-effects now emit as ObservationEvents with named-intent subjects (service.op); Ambient Begin/Done are ShellTrace-gated (Suppressed at Normal — silent scaffolding); Anomaly Failed surfaces from divergence disposition alone (channel Suppressed no longer forces StreamSuppressed). Derived attention + ci_line_placement supersede the dual-class conflation. Keep this note as the dissolve receipt; delete when a later census no longer needs the history." diff --git a/dag/gunbc/roadmap_belt_actuate.dag b/dag/gunbc/roadmap_belt_actuate.dag index 61878d4942f..25520f6b98e 100644 --- a/dag/gunbc/roadmap_belt_actuate.dag +++ b/dag/gunbc/roadmap_belt_actuate.dag @@ -1,5 +1,6 @@ module gunbc.roadmap_belt_actuate +import gunbc.output_policy { OutcomeIsData } import std.types { String, Int, Bool, List, NonEmptyStr, FilePath } import gunbc.roadmap_model { RoadmapNode } import gunbc.roadmap_spawner { SpawnFrontier, spawn_frontier, doc_all_nodes } @@ -84,8 +85,10 @@ type BeltExecOutcome data belt_program_available_note: String = "program-on-PATH probe via shell.Which.Check exit_success (Bool), NOT its .path field: the Which .path realization is a bare-stdout string wet (the Optional-over-Value::Null realization fork — DESIGN open thread), so the belt keys on the exit Bool and lets WitnessBin.Run's `env -C \{workdir\} \{bin_path\} args` resolve the bare program name via PATH (env execvp searches PATH). This keeps 'program missing' (refuse) distinct from 'program present, exited nonzero' (fail) without touching the Optional straddle." +data belt_outcome_is_data_note: String = "The three probes in this module are annotated OutcomeIsData (operator ruling 2026-07-25): their exit codes are ANSWERS the belt adjudicates, not faults. belt_program_available's which-miss is the 'program missing' pole this module's belt_program_available_note already keeps deliberately distinct from 'present but exited nonzero'; belt_exec's nonzero exit becomes ExecFailed; belt_observe's becomes a typed BeltObserve arm. In every case the belt renders the fault loudly through its own per-variant status mapping, so the effect layer's anomaly glyph was a second, cruder judgment of an event that already has a real adjudicator. ADMISSIBILITY (§5 guard, gunbc.output_policy outcome_is_data_note): each annotated fn returns a typed value whose arms ARE the adjudication — Bool for the availability probe (consumed as the refuse/fail discriminator, never discarded), BeltExecOutcome, BeltObserve — never unit. Dispatch failure stays anomalous: it is not an exit code and does not travel this arm." + fn belt_program_available(program: String) -> Bool { - shell.Which.Check(command: program as NonEmptyStr).exists + shell.Which.Check(command: program as NonEmptyStr, expect: OutcomeIsData).exists } fn belt_exec(cmd: HostExecArgv, workdir: String) -> BeltExecOutcome { @@ -94,6 +97,7 @@ fn belt_exec(cmd: HostExecArgv, workdir: String) -> BeltExecOutcome { workdir: workdir as FilePath, bin_path: cmd.program as FilePath, args: cmd.args, + expect: OutcomeIsData, ) if result.success { ExecOk { stdout: result.stdout } @@ -343,6 +347,7 @@ fn belt_observe() -> BeltObserve { workdir: belt_observe_workdir, bin_path: ls.program as FilePath, args: ls.args, + expect: OutcomeIsData, ) belt_observe_from_result( success: result.success, diff --git a/dag/std/measure.dag b/dag/std/measure.dag index 7a829d39555..34914571db5 100644 --- a/dag/std/measure.dag +++ b/dag/std/measure.dag @@ -85,6 +85,10 @@ fn kibi_factor() -> Nat { 1024 } +fn milliseconds_per_second() -> Nat { + 1000 +} + fn seconds_per_minute() -> Nat { 60 } diff --git a/dag/std/observation.dag b/dag/std/observation.dag new file mode 100644 index 00000000000..3c3165679a6 --- /dev/null +++ b/dag/std/observation.dag @@ -0,0 +1,600 @@ +module std.observation + +import std.types { String, NonEmptyStr, Bool, List } +import std.nat { Nat } +import std.measure { Millisecond, ByteSize, BasisPoint, measure_le, millisecond, millisecond_count } +import std.decl_ref { DeclarationRef, WholeDeclaration } +import std.symbols { + SymbolId, + NodeRunning, + NodeCompleted, + NodeFailed, + NodeSkipped, + StatusInfo, + StatusWarning, + StatusRefused, + StatusBlocked, + StatusTimedOut, + StatusDwell, + StatusFinal, + ChangeAdded, + ChangeModified, + ChangeRemoved, +} +import std.change { + KeyedDiffHunk, + KeyedDiffAddedHunk, + KeyedDiffRemovedHunk, + KeyedDiffModifiedHunk, +} +import std.effect_grant { path_is_prefix } +import std.disposition { Disposition, Terminal, Scaffold, SingleAuthority } +import std.temporal_effect { + EffectStallBudget, + StallBudgetVerdict, + StallWithinBudget, + StallBudgetExceeded, + stall_budget_verdict, +} + +data observation_module_note: String = "P0 of the progress-and-observation lane (docs/plans/progress-observation-design.md, operator-signed 2026-07-23; CI-rework delta in that doc section 6b). One event vocabulary emitted by the process and consumed by every renderer — a progress line is a PROJECTION of a fact the process already has, never a hand-authored string with a hand-chosen level. This module carries the model, the five laws as rows, and the derived attention/presentation assignment; it renders nothing (renderers are P1 and later). Displaced cost priced in the design doc: the 10-minute silent crawl window of run 30044816605, a 55-minute triage cycle burned on a receipt that only wrote at walk end, and workers hand-rolling the discipline ad hoc." + +data observation_homonym_note: String = "Naming diligence (DESIGN section 3): std.upsert_decision.ObservationVerdict is a DIFFERENT concept that shares the word — it classifies a desired-versus-observed STATE for reconcile/upsert (Converged, Drifted, Conflict). This module observes a PROCESS advancing through time. Neither is a nickname for the other; they are not to be consolidated. The distinguishing question: upsert asks what is true of the world, observation asks how far the work has got." + +type ObservationSegment + = RunSegment { id: NonEmptyStr } + | BatchSegment { index: Nat, label: NonEmptyStr } + | EntrySegment { source_path: NonEmptyStr } + | ModuleSegment { module_path: NonEmptyStr } + | PhaseSegment { name: NonEmptyStr } + +data observation_segment_grounding_note: String = "The subject is a CONTAINMENT PATH — run contains batch contains entry contains module contains phase, the tree the floor already walks — decomposed into typed segments rather than carried as one opaque string (DESIGN section 2 deep decomposition). Two segments name identities other authorities own, and are carried here in the same representation std already uses for them, not in a fresh one: module_path matches std.decl_ref.DeclarationRef.module_path, and source_path is a repo-relative source path. Convergence, not fork: module_path becomes v2.std.qualified_name.QualifiedName and source_path becomes the typed SourceRef when the namespace and module-identity lanes land those carriers (DESIGN open threads). Until then the representation is std's existing one, so there is nothing new to migrate." + +type ObservationSubject { + segments: List +} + +fn observation_segment_key(seg: ObservationSegment) -> String { + match seg { + RunSegment { id: i } => concat("run=", i) + BatchSegment { index: n, label: l } => concat("batch=", concat(to_string(n), concat(":", l))) + EntrySegment { source_path: p } => concat("entry=", p) + ModuleSegment { module_path: m } => concat("module=", m) + PhaseSegment { name: n } => concat("phase=", n) + } +} + +fn observation_subject_path(subject: ObservationSubject) -> List { + map(subject.segments, seg => observation_segment_key(seg: seg)) +} + +data observation_prefix_reuse_note: String = "Ancestry reuses std.effect_grant.path_is_prefix — the ONE prefix relation already carrying the containment question for effect targets, rather than a second walk written here (DESIGN section 3). Observation is one more consumer of the containment structure the resolver walks, content-addressing hashes, and effect admissibility tests. The subject tree is deliberately NOT added as a std.effect_grant.NamespaceTree variant in this phase: that enum is another lane's authority and the seed does not check match exhaustiveness, so a new variant would fall through silently in its existing consumers. Dissolve-on: the SymbolIndex-backed position supersedes the path-string interim, per the convergence commitment already declared in std.effect_grant.namespace_tree_note." + +fn observation_subject_contains(ancestor: ObservationSubject, descendant: ObservationSubject) -> Bool { + path_is_prefix( + prefix: observation_subject_path(subject: ancestor), + path: observation_subject_path(subject: descendant) + ) +} + +type ObservationGrain + = RunGrain + | BatchGrain + | EntryGrain + | ModuleGrain + | PhaseGrain + +fn observation_grain_of_segment(seg: ObservationSegment) -> ObservationGrain { + match seg { + RunSegment { id: _ } => RunGrain + BatchSegment { index: _, label: _ } => BatchGrain + EntrySegment { source_path: _ } => EntryGrain + ModuleSegment { module_path: _ } => ModuleGrain + PhaseSegment { name: _ } => PhaseGrain + } +} + +fn observation_grain_depth(g: ObservationGrain) -> Nat { + match g { + RunGrain => 0 + BatchGrain => 1 + EntryGrain => 2 + ModuleGrain => 3 + PhaseGrain => 4 + } +} + +fn observation_subject_grain(subject: ObservationSubject) -> ObservationGrain { + fold(subject.segments, init: RunGrain, f: (acc, seg) => observation_grain_of_segment(seg: seg)) +} + +fn observation_grain_at_or_above_entry(g: ObservationGrain) -> Bool { + observation_grain_depth(g: g) <= observation_grain_depth(g: EntryGrain) +} + +type Measured + = MeasuredValue { value: T } + | MeasuredUnavailable { cause: NonEmptyStr } + +data observation_measured_note: String = "Absence of a measurement is a NAMED state carrying its cause, never a fabricated zero (DESIGN section 5). This mirrors the discipline the floor heartbeat already keeps in the seed: an unreadable cgroup field prints unreadable and refuses to invent a number. A renderer projecting MeasuredUnavailable prints the cause; it never prints 0 and never omits the field, because a silently omitted number is the same fabrication one layer up." + +type ObservationOutcome + = Done + | Refused { diagnostic: NonEmptyStr } + | Failed { error: NonEmptyStr, output: String } + | TimedOut { budget: Millisecond, elapsed: Millisecond } + | Skipped { reason: NonEmptyStr } + | Final { shown_failures: Nat } + +data observation_refused_distinct_note: String = "Refused is DISTINCT from Failed, and this is the point of the sum, not a nicety. A typed refusal is the house's fail-closed spine (DESIGN section 5): the line stopped deliberately, located, with a diagnostic — the stopped line the operator is meant to analyse. A failure is the process breaking. The reference implementation studied for this lane conflates them behind one red glyph, which is exactly how a deliberate refusal reads as a crash and a crash reads as policy. They get different glyphs and different rendering; a renderer that maps both to one symbol is a regression the presentation census reds." + +type ObservationTransition + = Begin + | Step { k: Nat, n: Nat } + | Concluded { outcome: ObservationOutcome } + +type ObservationEvent { + subject: ObservationSubject + transition: ObservationTransition + wall: Measured + rss: Measured +} + +type SchedulerHold + = WindowFull { active: Nat, target: Nat } + | CurrentHighWater { current: ByteSize, high_water: ByteSize } + | PsiPressure { avg10: BasisPoint } + | SwapGrowth { delta: ByteSize } + | AwaitFirstCost { undigested: Nat } + | InsufficientHeadroom { current: ByteSize, share: ByteSize, high_water: ByteSize } + | AdmissionCeiling { current: ByteSize, ceiling: ByteSize } + +data observation_scheduler_hold_note: String = "SchedulerHold is the substrate authority for why admission was withheld this poll; the seed's v1_compiler memory_governor HoldReason is the realization that must stay in lockstep with it (the .dag graph is the authority and Rust is the seed). The lockstep witness proves arity and variant names match by reading the live seed, so a variant added on one side and not the other reds instead of drifting." + +data observation_psi_pressure_unit_note: String = "avg10 is a MAGNITUDE WITH A SCALE and is carried as one: std.measure.BasisPoint, not a bare scalar. The reading is a utilization ratio — the share of a ten-second window in which work stalled on memory — which is the exact semantic axis basis_point_unit_note names for that carrier, so this is a consumer of an existing authority rather than a new unit. Resolution is exact at the seed's fidelity: the governor reads one decimal of a percentage, and one basis point is a hundredth of a percentage point, so 37.5 percent is 3750 with nothing truncated. NO NEW CARRIER IS MINTED HERE DELIBERATELY: basis_point_dissolve_on already warns that a third dimensionless-ratio use-case minting a third nickname is the failure mode, and this module is that third use-case. When the Ratio unification that row calls for lands, this field follows Percent and BasisPoint onto it and needs no change of meaning." + +type ContendedResource + = MemoryBudget + | AdmissionWindow + | MaturationReserve + +type ContentionRemaining + = RemainingUnknown { cause: NonEmptyStr } + | RemainingBounded { by: Millisecond } + +type BlockedOn { + resource: ContendedResource + holders: List + remaining: ContentionRemaining +} + +fn observation_hold_resource(hold: SchedulerHold) -> ContendedResource { + match hold { + WindowFull { active: _, target: _ } => AdmissionWindow + CurrentHighWater { current: _, high_water: _ } => MemoryBudget + PsiPressure { avg10: _ } => MemoryBudget + SwapGrowth { delta: _ } => MemoryBudget + AwaitFirstCost { undigested: _ } => MaturationReserve + InsufficientHeadroom { current: _, share: _, high_water: _ } => MemoryBudget + AdmissionCeiling { current: _, ceiling: _ } => MaturationReserve + } +} + +data observation_blocked_on_derived_note: String = "BlockedOn is DERIVED from scheduler and governor state and has no hand-set constructor path in any renderer (design doc section 2). The governor already knows why it held; the crawl window that priced this lane was invisible only because that knowledge never reached the log. holders are the subjects currently occupying the contended resource — existing scheduler state, not a new telemetry source (design doc section 7 forbids new sources in P0 and P1). remaining is a named coproduct because the governor does not always have a bound to give: RemainingUnknown carries why, so the renderer prints the cause rather than an invented estimate." + +fn observation_blocked_on(hold: SchedulerHold, holders: List, remaining: ContentionRemaining) -> BlockedOn { + BlockedOn { + resource: observation_hold_resource(hold: hold), + holders: holders, + remaining: remaining + } +} + +type AttentionLevel + = Ambient + | Notable + | Anomaly + +type AttentionBasis { + average: Millisecond + maximum: Millisecond +} + +data observation_attention_basis_note: String = "AttentionLevel is DERIVED, never chosen at an emit site — there are no per-site log levels in this model (law 5). The basis is the pair of signed cost constants the CI clamp model carries (design doc section 6b): over the signed average is Notable, over the signed maximum is Anomaly. No threshold is invented here — the basis is a parameter supplied by the consumer that owns those constants, so this module cannot smuggle a number of its own. A refusal, failure, timeout, or orphaned begin is an Anomaly on its own terms, independent of any basis." + +fn observation_outcome_is_anomaly(o: ObservationOutcome) -> Bool { + match o { + Done => false + Refused { diagnostic: _ } => true + Failed { error: _, output: _ } => true + TimedOut { budget: _, elapsed: _ } => true + Skipped { reason: _ } => false + Final { shown_failures: _ } => false + } +} + +fn observation_attention_of_elapsed(elapsed: Millisecond, basis: AttentionBasis) -> AttentionLevel { + if measure_le(a: elapsed, b: basis.average) { + Ambient + } else if measure_le(a: elapsed, b: basis.maximum) { + Notable + } else { + Anomaly + } +} + +fn observation_attention(event: ObservationEvent, basis: AttentionBasis) -> AttentionLevel { + match event.transition { + Begin => Ambient + Step { k: _, n: _ } => + match event.wall { + MeasuredValue { value: w } => observation_attention_of_elapsed(elapsed: w, basis: basis) + MeasuredUnavailable { cause: _ } => Ambient + } + Concluded { outcome: o } => + if observation_outcome_is_anomaly(o: o) { + Anomaly + } else { + match event.wall { + MeasuredValue { value: w } => observation_attention_of_elapsed(elapsed: w, basis: basis) + MeasuredUnavailable { cause: _ } => Ambient + } + } + } +} + +type ObservationEventClass + = ClassBegin + | ClassStep + | ClassDone + | ClassRefused + | ClassFailed + | ClassTimedOut + | ClassSkipped + | ClassFinal + | ClassBlocked + | ClassDwellEscalation + +data observation_event_classes: List = [ + ClassBegin, + ClassStep, + ClassDone, + ClassRefused, + ClassFailed, + ClassTimedOut, + ClassSkipped, + ClassFinal, + ClassBlocked, + ClassDwellEscalation +] + +fn observation_outcome_class(o: ObservationOutcome) -> ObservationEventClass { + match o { + Done => ClassDone + Refused { diagnostic: _ } => ClassRefused + Failed { error: _, output: _ } => ClassFailed + TimedOut { budget: _, elapsed: _ } => ClassTimedOut + Skipped { reason: _ } => ClassSkipped + Final { shown_failures: _ } => ClassFinal + } +} + +fn observation_event_class(event: ObservationEvent) -> ObservationEventClass { + match event.transition { + Begin => ClassBegin + Step { k: _, n: _ } => ClassStep + Concluded { outcome: o } => observation_outcome_class(o: o) + } +} + +type ObservationPresentation { + glyph: SymbolId + collapsible: Bool + expands_fully: Bool +} + +data observation_collapse_axes_note: String = "collapsible and expands_fully are two axes, not one flag with a middle value, because three states are real and conflating them loses one. Routine progress collapses and does not expand. An intrinsic anomaly neither collapses nor stays terse: it expands fully and names itself. The run's Final summary is the third state — it never collapses, because a summary that hid itself would defeat its purpose, yet it is NOT an anomaly and must not be rendered as one. A single collapsible flag would force Final to borrow the anomaly's presentation, which is the state-space conflation the house splits into named variants." + +data observation_presentation_note: String = "Event class to presentation is a TOTAL assignment over the class roster, censused and walled the way the register censuses unthemed colors. The presentation carries only the SymbolId: the glyph text per tier and the register colour role are read from the ONE glyph authority (std.symbols identity, extdeps.render.glyphs rows), never restated here, so perturbing a glyph row moves every renderer and a renderer that hardcodes a glyph is provably not reading the table. collapsible and expands_fully carry law 4: sub-threshold work collapses, an anomaly expands fully and names itself, and the two flags are never both permissive for an intrinsically anomalous class." + +type ObservationDensity + = RoutineCollapsible + | SummaryAlwaysShown + | AnomalyExpanded + +data observation_density_single_authority_note: String = "The three display states of law 4 live in ONE table. Density is the authority; collapsible, expands_fully and intrinsic-anomaly are all DERIVED from it. The earlier shape carried two hand-written tables — a class-to-anomaly predicate beside a class-to-presentation assignment — with a witness asserting they agreed. That witness was validation standing exactly where construction was available (DESIGN section 5): it conceded the two tables could disagree and merely promised to notice. They cannot disagree now, because there is only one of them, and the review finding that named this duplication is what collapsed it." + +fn observation_class_density(c: ObservationEventClass) -> ObservationDensity { + match c { + ClassBegin => RoutineCollapsible + ClassStep => RoutineCollapsible + ClassDone => RoutineCollapsible + ClassRefused => AnomalyExpanded + ClassFailed => AnomalyExpanded + ClassTimedOut => AnomalyExpanded + ClassSkipped => RoutineCollapsible + ClassFinal => SummaryAlwaysShown + ClassBlocked => AnomalyExpanded + ClassDwellEscalation => AnomalyExpanded + } +} + +fn observation_density_collapsible(d: ObservationDensity) -> Bool { + match d { + RoutineCollapsible => true + SummaryAlwaysShown => false + AnomalyExpanded => false + } +} + +fn observation_density_expands_fully(d: ObservationDensity) -> Bool { + match d { + RoutineCollapsible => false + SummaryAlwaysShown => false + AnomalyExpanded => true + } +} + +fn observation_class_is_intrinsic_anomaly(c: ObservationEventClass) -> Bool { + observation_density_expands_fully(d: observation_class_density(c: c)) +} + +fn observation_class_glyph(c: ObservationEventClass) -> SymbolId { + match c { + ClassBegin => NodeRunning + ClassStep => NodeRunning + ClassDone => NodeCompleted + ClassRefused => StatusRefused + ClassFailed => NodeFailed + ClassTimedOut => StatusTimedOut + ClassSkipped => NodeSkipped + ClassFinal => StatusFinal + ClassBlocked => StatusBlocked + ClassDwellEscalation => StatusDwell + } +} + +fn observation_presentation(c: ObservationEventClass) -> ObservationPresentation { + let density = observation_class_density(c: c) + ObservationPresentation { + glyph: observation_class_glyph(c: c), + collapsible: observation_density_collapsible(d: density), + expands_fully: observation_density_expands_fully(d: density) + } +} + +data observation_change_presentation_note: String = "Change kind is NOT re-declared here. std.change.KeyedDiffHunk already carries the added, removed and modified discriminant — it is the house's one diff carrier — so a fresh three-variant enum on this module would be a nickname for it, duplicated again in everything derived from it. What observation owns is the PRESENTATION of that discriminant: a symbol whose register colour role follows the git-diff convention the operator signed, and a text tag. The tag is not optional decoration: colour is never the only channel, so a pipe, a colourless terminal, or a screen reader loses nothing. A presentation with an empty tag is the failure the witness reds." + +type ChangePresentation { + symbol: SymbolId + tag: NonEmptyStr +} + +fn observation_change_presentation(hunk: KeyedDiffHunk) -> ChangePresentation { + match hunk { + KeyedDiffAddedHunk { hunk_key: _, to: _ } => + ChangePresentation { symbol: ChangeAdded, tag: "new" as NonEmptyStr } + KeyedDiffRemovedHunk { hunk_key: _, from: _ } => + ChangePresentation { symbol: ChangeRemoved, tag: "removed" as NonEmptyStr } + KeyedDiffModifiedHunk { hunk_key: _, from: _, to: _ } => + ChangePresentation { symbol: ChangeModified, tag: "edited" as NonEmptyStr } + } +} + +type SelectionNoOp + = DocsPolicy + | Uncovered { declaration: NonEmptyStr } + | NoDeclsTouched + | GeneratedArtifact { drift_gate_authority: NonEmptyStr } + | DepartedPath { cause: NonEmptyStr } + +data observation_selection_no_op_note: String = "A touched file that selected no work is a CLOSED SUM, never a bare nothing. Nothing meaning five different things is the state-space conflation the house splits into named variants, and here the remedies genuinely differ: docs select no executable work and that is correct and quiet; a touched declaration no witness references is a COVERAGE GAP surfaced at review time, so it renders as a visible nudge rather than a comfort; only comments or whitespace intersected is fine and says so; a generated artifact is checked by the drift gate against its authority, not by witnesses, so nothing must never appear beside it; and a departed path is NOT a no-op at all — a deleted module cannot be scoped, so selection widens to baseline and the line says why. Because the sum is closed and the presentation assignment below is total, an unlabelled nothing is UNWRITABLE rather than censused after the fact — construction where a census would have conceded the bad line was writable." + +data selection_noop_generated_artifact_mint_note: String = "Mint helper for the SelectionNoOp.GeneratedArtifact arm. The bare constructor name `GeneratedArtifact` collides under whole-tree compile-clean with v2.std.artifact.GeneratedArtifact and gunbc.generated_artifact.GeneratedArtifact — three unrelated carriers that share a noun. Callers outside this module construct via this helper so the variant resolves in the defining module where the name is unambiguous." + +fn selection_noop_generated_artifact(drift_gate_authority: NonEmptyStr) -> SelectionNoOp { + GeneratedArtifact { drift_gate_authority: drift_gate_authority } +} + +fn observation_no_op_is_actually_a_widen(n: SelectionNoOp) -> Bool { + match n { + DocsPolicy => false + Uncovered { declaration: _ } => false + NoDeclsTouched => false + GeneratedArtifact { drift_gate_authority: _ } => false + DepartedPath { cause: _ } => true + } +} + +fn observation_no_op_attention(n: SelectionNoOp) -> AttentionLevel { + match n { + DocsPolicy => Ambient + Uncovered { declaration: _ } => Notable + NoDeclsTouched => Ambient + GeneratedArtifact { drift_gate_authority: _ } => Ambient + DepartedPath { cause: _ } => Notable + } +} + +fn observation_no_op_symbol(n: SelectionNoOp) -> SymbolId { + match n { + DocsPolicy => NodeSkipped + Uncovered { declaration: _ } => StatusWarning + NoDeclsTouched => NodeSkipped + GeneratedArtifact { drift_gate_authority: _ } => StatusInfo + DepartedPath { cause: _ } => StatusWarning + } +} + +fn observation_no_op_explanation(n: SelectionNoOp) -> NonEmptyStr { + match n { + DocsPolicy => "documentation never selects executable work" as NonEmptyStr + Uncovered { declaration: d } => concat("no witness covers this change: ", d) as NonEmptyStr + NoDeclsTouched => "only comments or whitespace changed" as NonEmptyStr + GeneratedArtifact { drift_gate_authority: a } => concat("generated artifact — checked by drift against ", a) as NonEmptyStr + DepartedPath { cause: c } => concat("a deleted module cannot be scoped, so selection widened: ", c) as NonEmptyStr + } +} + +data observation_human_units_contract: String = "Rendered magnitudes are in human units, and a raw telemetry dump is a census violation rather than a projection. The named negative example is the floor memory heartbeat's current shape: sixty-plus identical context-free byte dumps an hour, current=16111669248 with no subject attached, which is precisely the line that was present throughout the ten-minute crawl window and told the operator nothing. A projection states the magnitude at a scale a human reads and attaches the subject it belongs to. This contract is stated on the model so the P3 census has an authority to check against; the renderers that satisfy it are P1 and later." + +data observation_heartbeat_period: Millisecond = millisecond(count: 60000) + +data observation_heartbeat_period_note: String = "The heartbeat cadence is a measured fact of the running system, not a preference: the floor's memory heartbeat already samples and prints once per minute, and the lockstep witness reads that period out of the live seed rather than trusting this row. It is stated here because the dwell threshold is derived from it." + +type DwellThreshold { + period: Millisecond + basis: NonEmptyStr +} + +data observation_dwell_threshold: DwellThreshold = DwellThreshold { + period: observation_heartbeat_period, + basis: "one heartbeat period — the log's existing clock. Law 2 requires that no subject stay quiet past T without a line naming it, and the heartbeat is already the cadence at which the log speaks, so T is DERIVED from the emitter's own period rather than picked. Escalation at 2T and 4T rides the same clock, which is why depth is a division and not a table of tuned constants." +} + +data observation_dwell_threshold_disposition: Disposition = Scaffold { + dissolves_to: SingleAuthority, + bind: DeclarationRef { + module_path: "std.observation", + decl_name: "observation_dwell_threshold", + field: WholeDeclaration + } +} + +data observation_dwell_recalibration_note: String = "Declared scaffold, with its trigger: the design doc says T's basis is the measured quiet-time distribution. That distribution is not yet collected — the gauntlet and falsifier whole-corpus runs are the context that will produce it (design doc section 6b names that lane as the escalation law's primary home). Deriving T from the heartbeat period is a grounded interim, not taste, because the period is itself measured and law 2 is stated in terms of it. Dissolve-on: the first measured quiet-time distribution from a gauntlet cadence replaces the basis, and this row's period becomes a projection of it." + +fn observation_escalation_depth(quiet: Millisecond, t: DwellThreshold) -> Nat { + millisecond_count(m: quiet) / millisecond_count(m: t.period) +} + +data observation_escalation_note: String = "Law 3: attention escalates by dwell time, RECURSIVELY. A subject quiet past T surfaces its current child, past 2T the grandchild, and so on to the leaf — the collapse rule of law 4 run in reverse under time pressure, over the same tree and the same events. Depth is floor of quiet over T, so one law covers every level instead of a special case per level; the reference implementation expands one level only, and only on a TTY, which is why the CI crawl window stayed silent." + +fn observation_escalation_reveal_depth(quiet_subject: ObservationSubject, quiet: Millisecond, t: DwellThreshold) -> Nat { + count(quiet_subject.segments) + observation_escalation_depth(quiet: quiet, t: t) +} + +fn observation_escalation_subject(quiet_subject: ObservationSubject, active_leaf: ObservationSubject, quiet: Millisecond, t: DwellThreshold) -> ObservationSubject { + if observation_subject_contains(ancestor: quiet_subject, descendant: active_leaf) { + ObservationSubject { + segments: active_leaf.segments.take(n: observation_escalation_reveal_depth(quiet_subject: quiet_subject, quiet: quiet, t: t)) + } + } else { + quiet_subject + } +} + +type WatchdogVerdict + = WatchdogQuiet + | WatchdogOrphanedBegin { subject: ObservationSubject, elapsed: Millisecond, budget: Millisecond } + +data observation_watchdog_note: String = "Law 1: every subject at entry grain and above emits a matched Begin and exactly one Outcome, so an orphaned Begin is a DETECTABLE DEFECT rather than a silence to be interpreted. The watchdog converts quiet-past-budget into a typed line, which also closes the batch-wall hang gap — a batch that never completes currently rides silently to the step cap. The exceeded-budget arithmetic is not restated here: it calls std.temporal_effect.stall_budget_verdict, the one carrier already answering elapsed-against-limit, so there is a single authority for what over-budget means and this module only adds the subject the verdict is about." + +fn observation_watchdog(subject: ObservationSubject, elapsed: Millisecond, budget: Millisecond) -> WatchdogVerdict { + match stall_budget_verdict(budget: EffectStallBudget { + step_id: observation_subject_render(subject: subject), + limit_ms: budget, + elapsed_ms: elapsed + }) { + StallWithinBudget => WatchdogQuiet + StallBudgetExceeded { step_id: _, elapsed_ms: e, limit_ms: l } => + WatchdogOrphanedBegin { subject: subject, elapsed: e, budget: l } + } +} + +data observation_empty_subject_note: String = "The empty-path arm NAMES the absence rather than inventing a subject, and it does not widen: unattributed is a label for a caller that built a subject with no identity, not a plausible substitute for one. It is not the absorbing fallback the house forbids — nothing is rerun, nothing is scanned wholesale, and the line is as loud as any other. It is nonetheless a CONSTRUCTION GAP, declared rather than left silent: the right shape is a segment list that cannot be empty, which the substrate cannot express today — a record-field where clause over a list is named future work in the where-clause lowering plan, and no list refinement exists in the corpus. Dissolve-on: when that lowering lands, ObservationSubject.segments takes the non-empty refinement, this arm becomes unreachable by construction, and it deletes." + +fn observation_subject_render(subject: ObservationSubject) -> NonEmptyStr { + match join(observation_subject_path(subject: subject), " ") { + "" => "run=unattributed" as NonEmptyStr + rendered => rendered as NonEmptyStr + } +} + +fn observation_watchdog_attention(v: WatchdogVerdict) -> AttentionLevel { + match v { + WatchdogQuiet => Ambient + WatchdogOrphanedBegin { subject: _, elapsed: _, budget: _ } => Anomaly + } +} + +type ObservationLaw { + id: NonEmptyStr + statement: NonEmptyStr + enforced_by: DeclarationRef +} + +data observation_laws_note: String = "The five laws land as DATA ROWS, operator-signed 2026-07-23. Each row names the declaration that enforces it, so a law with no enforcement is a detectable orphan rather than a sentence in a document nobody executes — the lockstep witness reads this module out of the live tree and reds when a named declaration is absent. Candidate convergence, flagged not assumed: the enforcement-intent lane models the operator's recurring standing directives as StandingIntent rows with a lens contract and a coverage receipt. These five are directives of exactly that shape, but StandingIntent's fields today are lens-enforcement machinery (scope roster, consumer requirement, narrowing reasons) that these rows would fill hollowly, so they stay plain rows and the consolidation is the operator's call when that lane generalizes." + +data observation_law_process_to_outcome: ObservationLaw = ObservationLaw { + id: "observation.process-to-outcome" as NonEmptyStr, + statement: "Every subject at entry grain and above emits a matched Begin and exactly one Outcome; quiet past budget becomes a typed line, never an interpreted silence." as NonEmptyStr, + enforced_by: DeclarationRef { + module_path: "std.observation", + decl_name: "observation_watchdog", + field: WholeDeclaration + } +} + +data observation_law_heartbeat_identity: ObservationLaw = ObservationLaw { + id: "observation.heartbeat-carries-identity" as NonEmptyStr, + statement: "The heartbeat carries identity, not just vitals: phase, entry, module and k of n. No phase exceeds T without a line naming its current subject." as NonEmptyStr, + enforced_by: DeclarationRef { + module_path: "std.observation", + decl_name: "observation_subject_render", + field: WholeDeclaration + } +} + +data observation_law_dwell_escalation: ObservationLaw = ObservationLaw { + id: "observation.attention-escalates-by-dwell" as NonEmptyStr, + statement: "Attention escalates by dwell time, recursively: quiet past T surfaces the child, past 2T the grandchild, down to the leaf. T is a data row with a measured basis." as NonEmptyStr, + enforced_by: DeclarationRef { + module_path: "std.observation", + decl_name: "observation_escalation_subject", + field: WholeDeclaration + } +} + +data observation_law_quiet_at_arms_length: ObservationLaw = ObservationLaw { + id: "observation.quiet-at-arms-length" as NonEmptyStr, + statement: "Sub-threshold work is silent and collapses to summaries; a red or a refusal expands fully and names itself. The asymmetry is the design, and expansion stays bounded." as NonEmptyStr, + enforced_by: DeclarationRef { + module_path: "std.observation", + decl_name: "observation_presentation", + field: WholeDeclaration + } +} + +data observation_law_every_response_true: ObservationLaw = ObservationLaw { + id: "observation.every-response-true" as NonEmptyStr, + statement: "Every rendered number is a projection of a receipt fact. No vibes strings, no per-site log levels, no fabricated value standing in for an unavailable measurement." as NonEmptyStr, + enforced_by: DeclarationRef { + module_path: "std.observation", + decl_name: "observation_attention", + field: WholeDeclaration + } +} + +data observation_laws: List = [ + observation_law_process_to_outcome, + observation_law_heartbeat_identity, + observation_law_dwell_escalation, + observation_law_quiet_at_arms_length, + observation_law_every_response_true +] + +data observation_outcome_predicate_contract: Disposition = Terminal { + reason: "Structural coproduct readers over ObservationOutcome and ObservationEventClass (observation_outcome_is_anomaly, observation_outcome_class, observation_event_class, observation_class_density and the density projections) — closed by the declared outcome and class variants, not interim scaffolds. Terminal because the sums are closed by construction and the class roster is censused: a new variant cannot be added without the totality witness naming it. Discriminating corpus: dag/test/claim/observation_model_witness_test.dag, which reds when a class loses its presentation, when Refused and Failed collapse onto one glyph, and when the density authority disagrees with what the presentation projects." +} + +data observation_subject_predicate_contract: Disposition = Terminal { + reason: "Structural readers over ObservationSegment, ObservationGrain and SchedulerHold (observation_grain_of_segment, observation_grain_depth, observation_subject_grain, observation_grain_at_or_above_entry, observation_hold_resource) — closed by declared variants. The hold reader is Terminal on the .dag side and additionally pinned to its realization: the lockstep witness reads the seed governor by execution, so a variant added on one side without the other reds rather than drifting." +} + +data observation_selection_predicate_contract: Disposition = Terminal { + reason: "Structural coproduct readers over SelectionNoOp (observation_no_op_is_actually_a_widen, observation_no_op_attention, observation_no_op_symbol, observation_no_op_explanation) — closed by the declared no-op variants. Terminal, and load-bearing rather than decorative: the widen reader is what keeps a departed path from being rendered as a no-op, which is the distinction the operator's closed-sum ruling exists to preserve." +} + +data observation_renders_nothing_disposition: Disposition = Terminal { + reason: "P0 carries the model only. Renderers are P1 and later, and the CI renderer's wiring lands after the atomic CI PR because both touch the floor's emit sites and churning them twice is the migration class the operator ruled out. This module therefore holds no line formatting, no glyph text, and no ANSI: it holds the vocabulary those renderers project from." +} diff --git a/dag/std/symbols.dag b/dag/std/symbols.dag index fa0f189dcc3..e1bb1cc3a92 100644 --- a/dag/std/symbols.dag +++ b/dag/std/symbols.dag @@ -41,6 +41,17 @@ type SymbolId | StatusWarning | StatusInfo + | StatusRefused + | StatusBlocked + | StatusTimedOut + | StatusDwell + | StatusFinal + | StatusPulse + + | ChangeAdded + | ChangeModified + | ChangeRemoved + | DataList | DataMap | DataSecret diff --git a/dag/test/claim/no_fake_anomalies_witness_test.dag b/dag/test/claim/no_fake_anomalies_witness_test.dag new file mode 100644 index 00000000000..d622af0b41a --- /dev/null +++ b/dag/test/claim/no_fake_anomalies_witness_test.dag @@ -0,0 +1,59 @@ +module test.claim.no_fake_anomalies_witness + +import extdeps.git +import extdeps.gunbc +import gunbc.ci_layer_roots { compile_clean_source_roots } +import gunbc.output_policy { ExpectSuccess } +import std.logic { Bool } +import std.types { String } +import tools.host_prelude { ClaimRun, claim_batch_claims_argv, witness_bin_release_path } +import v2.std.live_tree { LiveTreeDisposition, ReadsLiveTree } + +data live_tree_disposition: LiveTreeDisposition = ReadsLiveTree + +data no_fake_anomalies_note: String = "THE WALL for the expectation backfill (operator acceptance 2026-07-25). Converting the ~24 fake anomalies one call site at a time is a bounded backfill of a missing semantic axis, but the fix is a CONVENTION — 'annotate your red controls' — and conventions have half-lives: a red control written next month without an expectation paints an anomaly glyph again and the class regrows. This witness is what makes the class extinct instead of merely cleaned. It runs a PASSING corpus at DEFAULT verbosity in a child process and asserts its stderr carries no anomaly glyph at all, so the day anyone lands an unannotated deliberate-failure probe, CI reds on the PR rather than the noise accumulating silently. Decidable, cheap, and it needs no new mechanism — the glyphs are the observable the operator reads anyway. Checks BOTH glyph spellings because the child chooses its own rendering: emoji under GitHub Actions, ASCII on a plain terminal (gunbc.observation_seed_render), and a wall that only knew one spelling would pass vacuously in whichever environment it was not written for." + +data anomaly_glyph_emoji: String = "❌" +data anomaly_glyph_ascii: String = "✗" + +fn claim_batch_path() -> String { + let root = git.Core.Toplevel() + witness_bin_release_path(root: root.path, bin_name: "claim_batch") +} + +fn run_corpus_capturing_stderr(entry: String, function: String) -> String { + let root = git.Core.Toplevel() + let result = gunbc.WitnessBin.Run( + workdir: root.path, + bin_path: claim_batch_path(), + args: claim_batch_claims_argv( + source_roots: compile_clean_source_roots(), + claims: [ClaimRun { entry: entry, function: function }] + ), + expect: ExpectSuccess + ) + result.stderr +} + +fn stderr_has_anomaly_glyph(stderr: String) -> Bool { + string_contains(s: stderr, pattern: anomaly_glyph_emoji) + || string_contains(s: stderr, pattern: anomaly_glyph_ascii) +} + +test fn passing_run_shows_no_fake_anomaly_holds() -> Bool { + let stderr = run_corpus_capturing_stderr( + entry: "dag/test/claim/roadmap_belt_actuate_witness_test.dag", + function: "roadmap_belt_actuate_witnesses" + ) + !stderr_has_anomaly_glyph(stderr: stderr) +} + +data no_fake_anomalies_red_control_note: String = "The RED control does not perturb anything: it points at an entry that STILL carries a deliberate anomaly — host_prelude's cargo.Build.BuildInheritEnv, left loud on purpose because a failing production build is a real fault, not an observation (witness_bin_ensure_built_probe_note). So the pair is genuine: an entry whose probes are all annotated shows no glyph, an entry with a real remaining one does. NOTE THE COUPLING, because it is load-bearing: if that cargo call is ever annotated, or the witness stops exercising it, THIS control goes vacuously green and the wall above loses its proof of discrimination. Re-point it at another entry carrying a deliberate anomaly at that moment — do not delete it, and do not let it pass by accident." + +test fn red_control_entry_with_real_anomaly_still_shows_glyph_holds() -> Bool { + let stderr = run_corpus_capturing_stderr( + entry: "dag/test/claim/typed_witness_invocation_test.dag", + function: "typed_run_witness_bin_unbuildable_red_control_holds" + ) + stderr_has_anomaly_glyph(stderr: stderr) +} diff --git a/dag/test/claim/observation_ci_render_witness_test.dag b/dag/test/claim/observation_ci_render_witness_test.dag new file mode 100644 index 00000000000..0c810411cc8 --- /dev/null +++ b/dag/test/claim/observation_ci_render_witness_test.dag @@ -0,0 +1,289 @@ +module test.claim.observation_ci_render_witness_test + +import v2.std.live_tree { LiveTreeDisposition, SubstrateInputsOnly } +import std.types { NonEmptyStr, String, Bool, List } +import std.nat { Nat } +import std.symbols { Tier, Emoji, Unicode, Ascii, StatusPulse, StatusDwell, StatusBlocked, StatusRefused, NodeFailed, NodeCompleted } +import std.measure { millisecond, byte_size, basis_point } +import std.observation { + ObservationSubject, + RunSegment, + BatchSegment, + EntrySegment, + ModuleSegment, + PhaseSegment, + MeasuredValue, + MeasuredUnavailable, + ObservationEvent, + Begin, + Step, + Concluded, + Done, + Refused, + Failed, + AttentionBasis, + Ambient, + Notable, + Anomaly, + observation_dwell_threshold, + observation_escalation_depth, + PsiPressure, +} +import gunbc.observation_ci_render { + RenderedLine, + LinePlacement, + InsideCollapsedGroup, + OutsideGroup, + ci_line_placement, + ci_render_line, + ci_event_line, + ci_heartbeat_line, + ci_escalation_line, + ci_escalation_blocked_line, + ci_human_bytes, + ci_human_duration, + ci_human_percent, + ci_collapsed_summary, + ci_run_summary_line, + ci_refusal_restate_line, + HeartbeatSample, + FloorRunSummary, + ci_group_open, + ci_group_close, + ci_escalation_emission_count, + ci_window_is_responsive, + ci_run_summary_glyph, +} + +data live_tree_disposition: LiveTreeDisposition = SubstrateInputsOnly + +data witness_note: String = "P1 flagship acceptance (docs/plans/progress-observation-design.md section 5): re-render the CAPTURED crawl window of run 30044816605 through the escalation law and produce named activity where there was silence. The fixture below is not invented — every number is read off that run's log: the heartbeat at t=33m carried current=16107200512 swap=34359738368 psi_some_avg10=9.01 and named no subject, and the module the process was actually inside for the whole window, v2.compiler.normalized_tree, was disclosed only at walk end as typecheck-attribution ms=606984. The renderer is a pure function of the stream, which is exactly what makes replaying a real captured run possible." + +data crawl_run_id: NonEmptyStr = "30044816605" as NonEmptyStr + +data crawl_quiet_module: NonEmptyStr = "v2.compiler.normalized_tree" as NonEmptyStr + +data crawl_module_wall_ms: Int = 606984 + +data crawl_heartbeat_rss_bytes: Int = 16107200512 + +data crawl_heartbeat_swap_bytes: Int = 34359738368 + +data crawl_heartbeat_pressure_bp: Int = 901 + +fn crawl_entry_subject() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: crawl_run_id }, + BatchSegment { index: 3, label: "witness discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/virtual_media_unification_witness_test.dag" as NonEmptyStr } + ] + } +} + +fn crawl_leaf_subject() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: crawl_run_id }, + BatchSegment { index: 3, label: "witness discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/virtual_media_unification_witness_test.dag" as NonEmptyStr }, + ModuleSegment { module_path: crawl_quiet_module }, + PhaseSegment { name: "typecheck" as NonEmptyStr } + ] + } +} + +fn crawl_heartbeat() -> HeartbeatSample { + HeartbeatSample { + elapsed: millisecond(count: 1980000), + active: crawl_leaf_subject(), + entry_index: 214, + entry_total: 602, + rss: MeasuredValue { value: byte_size(crawl_heartbeat_rss_bytes) }, + swap: MeasuredValue { value: byte_size(crawl_heartbeat_swap_bytes) }, + pressure: MeasuredValue { value: basis_point(count: crawl_heartbeat_pressure_bp) } + } +} + +fn crawl_basis() -> AttentionBasis { + AttentionBasis { average: millisecond(count: 1000), maximum: millisecond(count: 5000) } +} + +fn w_human_units_replace_raw_byte_dumps() -> Bool { + ci_human_bytes(b: byte_size(crawl_heartbeat_rss_bytes)) == "15.0 GiB" && + ci_human_bytes(b: byte_size(crawl_heartbeat_swap_bytes)) == "32.0 GiB" && + ci_human_percent(bp: basis_point(count: crawl_heartbeat_pressure_bp)) == "9.0%" && + ci_human_duration(d: millisecond(count: crawl_module_wall_ms)) == "10 minutes" +} + +fn w_heartbeat_names_its_subject_where_the_capture_was_silent() -> Bool { + let line = ci_heartbeat_line(sample: crawl_heartbeat()) + string_contains(s: line.text, pattern: crawl_quiet_module) && + string_contains(s: line.text, pattern: "entry 214 of 602") && + string_contains(s: line.text, pattern: "15.0 GiB") && + !string_contains(s: line.text, pattern: to_string(crawl_heartbeat_rss_bytes)) +} + +fn w_heartbeat_puts_identity_before_vitals() -> Bool { + let line = ci_heartbeat_line(sample: crawl_heartbeat()) + starts_with(s: line.text, prefix: "33 minutes in — still in witness discovery") && + string_contains(s: line.text, pattern: "memory 15.0 GiB") +} + +fn w_heartbeat_uses_the_clock_pulse_glyph() -> Bool { + let line = ci_heartbeat_line(sample: crawl_heartbeat()) + ci_render_line(line: line, tier: Emoji) == concat("🕐 ", line.text) +} + +fn w_escalation_surfaces_the_quiet_module_at_one_threshold() -> Bool { + let line = ci_escalation_line( + quiet_subject: crawl_entry_subject(), + active_leaf: crawl_leaf_subject(), + quiet: millisecond(count: 120000), + t: observation_dwell_threshold + ) + string_contains(s: line.text, pattern: crawl_quiet_module) && + string_contains(s: line.text, pattern: "quiet for 2 minutes") && + line.attention == Notable +} + +fn w_escalation_surfaces_the_cause_at_the_deeper_threshold() -> Bool { + let line = ci_escalation_blocked_line( + quiet_subject: crawl_entry_subject(), + active_leaf: crawl_leaf_subject(), + quiet: millisecond(count: 240000), + t: observation_dwell_threshold, + hold: PsiPressure { avg10: basis_point(count: crawl_heartbeat_pressure_bp) } + ) + string_contains(s: line.text, pattern: "blocked on memory reclaim") && + string_contains(s: line.text, pattern: "9.0%") && + line.attention == Anomaly +} + +fn w_escalation_is_recursive_over_the_captured_window() -> Bool { + observation_escalation_depth(quiet: millisecond(count: 600000), t: observation_dwell_threshold) == 10 && + observation_escalation_depth(quiet: millisecond(count: 120000), t: observation_dwell_threshold) == 2 +} + +fn w_outcome_lines_carry_durations() -> Bool { + let event = ObservationEvent { + subject: crawl_entry_subject(), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: 230) }, + rss: MeasuredUnavailable { cause: "not sampled per entry" as NonEmptyStr } + } + string_contains(s: ci_event_line(event: event, basis: crawl_basis()).text, pattern: "in 230ms") +} + +fn w_refusal_never_renders_inside_a_collapsed_group() -> Bool { + let refusal = ObservationEvent { + subject: crawl_entry_subject(), + transition: Concluded { outcome: Refused { diagnostic: "FLOOR-BATCH-OVER-BUDGET" as NonEmptyStr } }, + wall: MeasuredValue { value: millisecond(count: 2472000) }, + rss: MeasuredUnavailable { cause: "not sampled per entry" as NonEmptyStr } + } + let line = ci_event_line(event: refusal, basis: crawl_basis()) + line.attention == Anomaly && + ci_line_placement(attention: line.attention) == OutsideGroup +} + +fn w_routine_green_collapses_inside_a_group() -> Bool { + let green = ObservationEvent { + subject: crawl_entry_subject(), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: 230) }, + rss: MeasuredUnavailable { cause: "not sampled per entry" as NonEmptyStr } + } + let line = ci_event_line(event: green, basis: crawl_basis()) + line.attention == Ambient && + ci_line_placement(attention: line.attention) == InsideCollapsedGroup && + ci_collapsed_summary(green_count: 213) == "… 213 more finished cleanly" +} + +fn w_failure_and_refusal_are_restated_at_the_end() -> Bool { + let summary = ci_run_summary_line(summary: FloorRunSummary { + entries_complete: 213, + entries_total: 602, + refusals: 1, + failures: 0, + wall: millisecond(count: 2472000) + }) + let restate = ci_refusal_restate_line(subject: crawl_entry_subject(), diagnostic: "FLOOR-BATCH-OVER-BUDGET" as NonEmptyStr) + string_contains(s: summary.text, pattern: "213 of 602") && + string_contains(s: summary.text, pattern: "1 refused") && + summary.attention == Anomaly && + string_contains(s: restate.text, pattern: "FLOOR-BATCH-OVER-BUDGET") +} + +fn w_relayed_diagnostic_cannot_mint_workflow_commands() -> Bool { + let restate = ci_refusal_restate_line( + subject: crawl_entry_subject(), + diagnostic: "budget exceeded\n::error::a child process said this" as NonEmptyStr + ) + string_contains(s: restate.text, pattern: "\n| ::error::") && + !string_contains(s: restate.text, pattern: "\n::error::") +} + +fn w_group_markers_come_from_the_github_authority() -> Bool { + ci_group_open(title: "witness discovery") == "::group::witness discovery" && + ci_group_close() == "::endgroup::" +} + +fn w_captured_window_would_have_spoken_four_times() -> Bool { + ci_escalation_emission_count(quiet: millisecond(count: 600000), t: observation_dwell_threshold) == 4 && + ci_escalation_emission_count(quiet: millisecond(count: 120000), t: observation_dwell_threshold) == 2 && + ci_escalation_emission_count(quiet: millisecond(count: 30000), t: observation_dwell_threshold) == 0 +} + +fn w_a_silent_phase_past_the_threshold_is_not_responsive() -> Bool { + ci_window_is_responsive(quiet: millisecond(count: 30000), t: observation_dwell_threshold) && + ci_window_is_responsive(quiet: millisecond(count: 600000), t: observation_dwell_threshold) +} + +fn w_escalation_emission_stays_bounded_under_unbounded_silence() -> Bool { + ci_escalation_emission_count(quiet: millisecond(count: 999999999), t: observation_dwell_threshold) <= 13 +} + +fn w_run_summary_keeps_refused_and_failed_apart() -> Bool { + let refused_only = FloorRunSummary { entries_complete: 213, entries_total: 602, refusals: 1, failures: 0, wall: millisecond(count: 2472000) } + let failed_only = FloorRunSummary { entries_complete: 601, entries_total: 602, refusals: 0, failures: 1, wall: millisecond(count: 600000) } + let both = FloorRunSummary { entries_complete: 200, entries_total: 602, refusals: 2, failures: 3, wall: millisecond(count: 600000) } + let clean = FloorRunSummary { entries_complete: 602, entries_total: 602, refusals: 0, failures: 0, wall: millisecond(count: 600000) } + ci_run_summary_glyph(summary: refused_only) == StatusRefused && + ci_run_summary_glyph(summary: failed_only) == NodeFailed && + ci_run_summary_glyph(summary: both) == NodeFailed && + ci_run_summary_glyph(summary: clean) == NodeCompleted && + string_contains(s: ci_run_summary_line(summary: both).text, pattern: "2 refused and 3 failed") +} + +fn w_unmeasured_duration_names_its_cause_rather_than_vanishing() -> Bool { + let event = ObservationEvent { + subject: crawl_entry_subject(), + transition: Concluded { outcome: Done }, + wall: MeasuredUnavailable { cause: "clock unavailable in this leg" as NonEmptyStr }, + rss: MeasuredUnavailable { cause: "not sampled per entry" as NonEmptyStr } + } + let line = ci_event_line(event: event, basis: crawl_basis()) + string_contains(s: line.text, pattern: "duration unmeasured: clock unavailable in this leg") +} + +test fn observation_ci_render_witnesses() -> Bool { + w_run_summary_keeps_refused_and_failed_apart() && + w_unmeasured_duration_names_its_cause_rather_than_vanishing() && + w_captured_window_would_have_spoken_four_times() && + w_a_silent_phase_past_the_threshold_is_not_responsive() && + w_escalation_emission_stays_bounded_under_unbounded_silence() && + w_human_units_replace_raw_byte_dumps() && + w_heartbeat_names_its_subject_where_the_capture_was_silent() && + w_heartbeat_puts_identity_before_vitals() && + w_heartbeat_uses_the_clock_pulse_glyph() && + w_escalation_surfaces_the_quiet_module_at_one_threshold() && + w_escalation_surfaces_the_cause_at_the_deeper_threshold() && + w_escalation_is_recursive_over_the_captured_window() && + w_outcome_lines_carry_durations() && + w_refusal_never_renders_inside_a_collapsed_group() && + w_routine_green_collapses_inside_a_group() && + w_failure_and_refusal_are_restated_at_the_end() && + w_relayed_diagnostic_cannot_mint_workflow_commands() && + w_group_markers_come_from_the_github_authority() +} diff --git a/dag/test/claim/observation_crawl_replay_test.dag b/dag/test/claim/observation_crawl_replay_test.dag new file mode 100644 index 00000000000..c12fdd07f3f --- /dev/null +++ b/dag/test/claim/observation_crawl_replay_test.dag @@ -0,0 +1,133 @@ +module test.claim.observation_crawl_replay_test + +import v2.std.live_tree { LiveTreeDisposition, SubstrateInputsOnly } +import std.types { String, NonEmptyStr, Bool, List } +import std.measure { millisecond, byte_size, basis_point } +import std.symbols { Emoji } +import std.observation { + ObservationSubject, + RunSegment, + BatchSegment, + EntrySegment, + ModuleSegment, + PhaseSegment, + MeasuredValue, + MeasuredUnavailable, + ObservationEvent, + Concluded, + Done, + AttentionBasis, + observation_dwell_threshold, + PsiPressure, +} +import gunbc.observation_ci_render { + RenderedLine, + ci_render_line, + ci_event_line, + ci_heartbeat_line, + ci_escalation_line, + ci_escalation_blocked_line, + ci_refusal_restate_line, + ci_run_summary_line, + ci_group_open, + ci_group_close, + ci_collapsed_summary, + HeartbeatSample, + FloorRunSummary, +} + +data live_tree_disposition: LiveTreeDisposition = SubstrateInputsOnly + +data witness_note: String = "End-to-end replay of the captured crawl window of run 30044816605 through the P1 CI renderer, asserting the WHOLE assembled block by execution — the flagship's before/after made a green witness rather than a hand-typed sample. Every input number is read off that run's log: the t=33m heartbeat carried current=16107200512 swap=34359738368 psi_some_avg10=9.01 and named no subject, and the module the process sat inside for 606984ms, v2.compiler.normalized_tree, was disclosed only at walk end. This witness renders those same facts and asserts the block now NAMES the activity where the capture was silent. observation_crawl_after_block is also the function the PR body's after-sample is produced from, so the pasted sample is a projection of a green execution, not prose." + +fn crawl_entry() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: "30044816605" as NonEmptyStr }, + BatchSegment { index: 3, label: "witness discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/virtual_media_unification_witness_test.dag" as NonEmptyStr } + ] + } +} + +fn crawl_leaf() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: "30044816605" as NonEmptyStr }, + BatchSegment { index: 3, label: "witness discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/virtual_media_unification_witness_test.dag" as NonEmptyStr }, + ModuleSegment { module_path: "v2.compiler.normalized_tree" as NonEmptyStr }, + PhaseSegment { name: "typecheck" as NonEmptyStr } + ] + } +} + +fn crawl_basis() -> AttentionBasis { + AttentionBasis { average: millisecond(count: 1000), maximum: millisecond(count: 5000) } +} + +fn crawl_heartbeat_at(minute: Int, rss: Int, swap: Int, psi: Int) -> RenderedLine { + ci_heartbeat_line(sample: HeartbeatSample { + elapsed: millisecond(count: minute * 60000), + active: crawl_leaf(), + entry_index: 214, + entry_total: 602, + rss: MeasuredValue { value: byte_size(rss) }, + swap: MeasuredValue { value: byte_size(swap) }, + pressure: MeasuredValue { value: basis_point(count: psi) } + }) +} + +fn observation_crawl_after_block() -> String { + let lines = [ + ci_group_open(title: "witness discovery — batch 3 (602 entries)"), + ci_render_line(line: ci_event_line(event: ObservationEvent { + subject: crawl_entry(), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: 230) }, + rss: MeasuredUnavailable { cause: "not sampled per entry" as NonEmptyStr } + }, basis: crawl_basis()), tier: Emoji), + ci_collapsed_summary(green_count: 213), + ci_render_line(line: crawl_heartbeat_at(minute: 29, rss: 16106717184, swap: 34359738368, psi: 533), tier: Emoji), + ci_render_line(line: crawl_heartbeat_at(minute: 33, rss: 16107200512, swap: 34359738368, psi: 901), tier: Emoji), + ci_render_line(line: ci_escalation_line(quiet_subject: crawl_entry(), active_leaf: crawl_leaf(), quiet: millisecond(count: 120000), t: observation_dwell_threshold), tier: Emoji), + ci_render_line(line: ci_escalation_blocked_line(quiet_subject: crawl_entry(), active_leaf: crawl_leaf(), quiet: millisecond(count: 240000), t: observation_dwell_threshold, hold: PsiPressure { avg10: basis_point(count: 901) }), tier: Emoji), + ci_render_line(line: ci_refusal_restate_line(subject: crawl_entry(), diagnostic: "FLOOR-BATCH-OVER-BUDGET wall=41m12s budget=22m" as NonEmptyStr), tier: Emoji), + ci_group_close(), + ci_render_line(line: ci_run_summary_line(summary: FloorRunSummary { + entries_complete: 213, + entries_total: 602, + refusals: 1, + failures: 0, + wall: millisecond(count: 2472000) + }), tier: Emoji) + ] + join(lines, "\n") +} + +fn w_replay_names_the_module_where_the_capture_was_silent() -> Bool { + let block = observation_crawl_after_block() + string_contains(s: block, pattern: "still in witness discovery: entry 214 of 602, now typecheck v2.compiler.normalized_tree") && + string_contains(s: block, pattern: "quiet for 2 minutes — still working") && + string_contains(s: block, pattern: "quiet for 4 minutes — blocked on memory reclaim (pressure 9.0%)") +} + +fn w_replay_uses_human_units_not_raw_bytes() -> Bool { + let block = observation_crawl_after_block() + string_contains(s: block, pattern: "memory 15.0 GiB, swap 32.0 GiB, pressure 9.0%") && + !string_contains(s: block, pattern: "16107200512") && + !string_contains(s: block, pattern: "current=") +} + +fn w_replay_ends_in_a_refusal_summary_not_a_silent_timeout() -> Bool { + let block = observation_crawl_after_block() + string_contains(s: block, pattern: "213 of 602 entries finished") && + string_contains(s: block, pattern: "1 refused and 0 failed") && + string_contains(s: block, pattern: "FLOOR-BATCH-OVER-BUDGET") +} + +test fn observation_crawl_replay_witnesses() -> Bool { + w_replay_names_the_module_where_the_capture_was_silent() && + w_replay_uses_human_units_not_raw_bytes() && + w_replay_ends_in_a_refusal_summary_not_a_silent_timeout() +} diff --git a/dag/test/claim/observation_emit_census_witness_test.dag b/dag/test/claim/observation_emit_census_witness_test.dag new file mode 100644 index 00000000000..fff29c35804 --- /dev/null +++ b/dag/test/claim/observation_emit_census_witness_test.dag @@ -0,0 +1,209 @@ +module test.claim.observation_emit_census_witness_test + +import extdeps.filesystem.filesystem_io +import v2.std.live_tree { LiveTreeDisposition, ReadsLiveTree } +import std.types { String, NonEmptyStr, Bool, List } +import std.nat { Nat } +import gunbc.observation_emit_census { + CensusedEmitSite, + EmitSiteDisposition, + MigratedToObservation, + CountedFrontierSite, + observation_emit_roster, + observation_emit_frontier_count, + emit_site_is_frontier, + emit_site_dissolve_on, + floor_memory_site, + gantt_site, + governor_site, + typecheck_attribution_site, + measurement_site, + shell_site, + floor_drain_site, + gate_warm_cost_site, + receipt_site, + file_trace_site, + resolve_split_site, + assembly_split_site, +} + +data live_tree_disposition: LiveTreeDisposition = ReadsLiveTree + +data witness_note: String = "P3 census hygiene, executable against the live seed (docs/plans/progress-observation-design.md section 5). A census that is not checked against the code it censuses is coverage-by-illusion — an inert lens is itself a lie. So every rostered marker is held against the seed file it names: a marker that has vanished reds (the roster went stale, the site was renamed or deleted without re-census), and a frontier row must carry a real, non-empty dissolve-on so the debt stays prioritizable rather than open-ended. Migrated sites keep their marker strings so the roster cannot go stale, and each has a RED that the raw byte shape is gone from the seed." + +fn census_source(path: String) -> String { + let r = filesystem_read(path: path) + r.content +} + +fn census_marker_present(site: CensusedEmitSite) -> Bool { + string_contains(s: census_source(path: site.source_file), pattern: site.marker) +} + +fn w_every_rostered_marker_still_exists_in_the_seed() -> Bool { + fold(observation_emit_roster, init: true, f: (acc, site) => + acc && census_marker_present(site: site)) +} + +fn w_roster_staleness_check_discriminates() -> Bool { + let bogus = CensusedEmitSite { + marker: "[this-marker-does-not-exist-in-the-seed]" as NonEmptyStr, + source_file: "src/v1/stage0/src/bin/claim_executor.rs" as NonEmptyStr, + disposition: CountedFrontierSite { + reason: "control" as NonEmptyStr, + dissolve_on: "control" as NonEmptyStr + } + } + !census_marker_present(site: bogus) +} + +fn w_every_frontier_row_carries_a_real_dissolve_on() -> Bool { + fold(observation_emit_roster, init: true, f: (acc, site) => + if emit_site_is_frontier(site: site) { + acc && emit_site_dissolve_on(site: site) != "" + } else { + acc + }) +} + +fn w_the_named_negative_example_has_migrated() -> Bool { + census_marker_present(site: floor_memory_site) && + !emit_site_is_frontier(site: floor_memory_site) && + string_contains(s: floor_memory_site.marker, pattern: "floor-memory") && + string_contains(s: emit_site_dissolve_on(site: floor_memory_site), pattern: "ci_heartbeat_line") +} + +fn w_gantt_has_migrated() -> Bool { + census_marker_present(site: gantt_site) && + !emit_site_is_frontier(site: gantt_site) && + string_contains(s: gantt_site.marker, pattern: "gantt") && + string_contains(s: emit_site_dissolve_on(site: gantt_site), pattern: "ci_event_line") && + string_contains(s: emit_site_dissolve_on(site: gantt_site), pattern: "phase_begin_line") +} + +fn w_governor_has_migrated() -> Bool { + census_marker_present(site: governor_site) && + !emit_site_is_frontier(site: governor_site) && + string_contains(s: governor_site.marker, pattern: "governor") && + string_contains(s: emit_site_dissolve_on(site: governor_site), pattern: "ci_hold_cause_text") && + string_contains(s: emit_site_dissolve_on(site: governor_site), pattern: "render_governor_hold_line_mirror") +} + +fn w_typecheck_attribution_has_migrated() -> Bool { + census_marker_present(site: typecheck_attribution_site) && + !emit_site_is_frontier(site: typecheck_attribution_site) && + string_contains(s: typecheck_attribution_site.marker, pattern: "typecheck-attribution") && + string_contains(s: emit_site_dissolve_on(site: typecheck_attribution_site), pattern: "ci_event_line") && + string_contains(s: emit_site_dissolve_on(site: typecheck_attribution_site), pattern: "typecheck_concluded_line") +} + +fn w_measurement_has_migrated() -> Bool { + census_marker_present(site: measurement_site) && + !emit_site_is_frontier(site: measurement_site) && + string_contains(s: measurement_site.marker, pattern: "measurement") && + string_contains(s: emit_site_dissolve_on(site: measurement_site), pattern: "ci_measurement_rss_line") && + string_contains(s: emit_site_dissolve_on(site: measurement_site), pattern: "render_peak_rss_line_mirror") +} + +fn w_shell_has_migrated() -> Bool { + census_marker_present(site: shell_site) && + !emit_site_is_frontier(site: shell_site) && + string_contains(s: shell_site.marker, pattern: "shell") && + string_contains(s: emit_site_dissolve_on(site: shell_site), pattern: "shell_effect_failed_line") && + string_contains(s: emit_site_dissolve_on(site: shell_site), pattern: "render_shell_effect_") +} + +fn w_the_census_states_post_snapshot_frontier_count() -> Bool { + observation_emit_frontier_count() == 6 && + count(observation_emit_roster) == 12 +} + +data w_post_snapshot_live_tags_are_rostered_bidirectional_note: String = "Operator finding 3 (run 30142403230): hygiene is bidirectional — growth tags born after the initial census snapshot must appear on the roster, not only the reverse (rostered markers still exist). Extended for #7205 resolve/assembly split tags." + +fn w_post_snapshot_live_tags_are_rostered_bidirectional() -> Bool { + census_marker_present(site: floor_drain_site) && + emit_site_is_frontier(site: floor_drain_site) && + census_marker_present(site: gate_warm_cost_site) && + emit_site_is_frontier(site: gate_warm_cost_site) && + census_marker_present(site: receipt_site) && + emit_site_is_frontier(site: receipt_site) && + census_marker_present(site: file_trace_site) && + emit_site_is_frontier(site: file_trace_site) && + census_marker_present(site: resolve_split_site) && + emit_site_is_frontier(site: resolve_split_site) && + census_marker_present(site: assembly_split_site) && + emit_site_is_frontier(site: assembly_split_site) +} + +fn w_the_raw_byte_dump_shape_is_gone_from_the_seed() -> Bool { + let executor = census_source(path: "src/v1/stage0/src/bin/claim_executor.rs") + !string_contains(s: executor, pattern: "[floor-memory] t=") && + !string_contains(s: executor, pattern: "current={}") && + string_contains(s: executor, pattern: "render_heartbeat_line_mirror") +} + +fn w_the_raw_gantt_shape_is_gone_from_the_seed() -> Bool { + let rt = census_source(path: "src/v1/stage0/src/v1_rt.rs") + let executor = census_source(path: "src/v1/stage0/src/bin/claim_executor.rs") + !string_contains(s: rt, pattern: "t_ms=") && + !string_contains(s: rt, pattern: "rss_mib=") && + string_contains(s: rt, pattern: "render_phase_begin_line_mirror") && + string_contains(s: rt, pattern: "render_phase_concluded_line_mirror") && + !string_contains(s: executor, pattern: "[gantt] claim_executor wall:") && + string_contains(s: executor, pattern: "render_phase_concluded_line_mirror") +} + +fn w_the_raw_governor_shape_is_gone_from_the_seed() -> Bool { + let gov = census_source(path: "src/v1/stage0/src/memory_governor.rs") + !string_contains(s: gov, pattern: "[governor] hard back-off:") && + !string_contains(s: gov, pattern: "[governor] creep back-off:") && + !string_contains(s: gov, pattern: "[governor] receipt:") && + !string_contains(s: gov, pattern: "[governor] adaptive width") && + string_contains(s: gov, pattern: "render_governor_hold_line_mirror") && + string_contains(s: gov, pattern: "GOVERNOR_CENSUS_MARKER") +} + +fn w_the_raw_typecheck_attribution_shape_is_gone_from_the_seed() -> Bool { + let cli = census_source(path: "src/v1/stage0/src/cli_run.rs") + !string_contains(s: cli, pattern: "[typecheck-attribution] module=") && + string_contains(s: cli, pattern: "render_typecheck_concluded_line_mirror") && + string_contains(s: cli, pattern: "TYPECHECK_ATTRIBUTION_CENSUS_MARKER") +} + +fn w_the_raw_measurement_shape_is_gone_from_the_seed() -> Bool { + let executor = census_source(path: "src/v1/stage0/src/bin/claim_executor.rs") + let cli = census_source(path: "src/v1/stage0/src/cli_run.rs") + !string_contains(s: executor, pattern: "bytes (VmHWM)") && + !string_contains(s: executor, pattern: "[measurement] floor peak RSS:") && + !string_contains(s: executor, pattern: "[measurement] cgroup peak:") && + string_contains(s: executor, pattern: "render_peak_rss_line_mirror") && + string_contains(s: cli, pattern: "MEASUREMENT_CENSUS_MARKER") +} + +fn w_the_raw_shell_shape_is_gone_from_the_seed() -> Bool { + let interp = census_source(path: "src/v1/stage0/src/v1_interpreter.rs") + !string_contains(s: interp, pattern: "[shell] done exit=") && + !string_contains(s: interp, pattern: "[shell] stderr (exit=") && + string_contains(s: interp, pattern: "render_shell_effect_failed_line_mirror") && + string_contains(s: interp, pattern: "SHELL_CENSUS_MARKER") +} + +test fn observation_emit_census_witnesses() -> Bool { + w_every_rostered_marker_still_exists_in_the_seed() && + w_roster_staleness_check_discriminates() && + w_every_frontier_row_carries_a_real_dissolve_on() && + w_the_named_negative_example_has_migrated() && + w_gantt_has_migrated() && + w_governor_has_migrated() && + w_typecheck_attribution_has_migrated() && + w_measurement_has_migrated() && + w_shell_has_migrated() && + w_the_census_states_post_snapshot_frontier_count() && + w_post_snapshot_live_tags_are_rostered_bidirectional() && + w_the_raw_byte_dump_shape_is_gone_from_the_seed() && + w_the_raw_gantt_shape_is_gone_from_the_seed() && + w_the_raw_governor_shape_is_gone_from_the_seed() && + w_the_raw_typecheck_attribution_shape_is_gone_from_the_seed() && + w_the_raw_measurement_shape_is_gone_from_the_seed() && + w_the_raw_shell_shape_is_gone_from_the_seed() +} diff --git a/dag/test/claim/observation_lockstep_witness_test.dag b/dag/test/claim/observation_lockstep_witness_test.dag new file mode 100644 index 00000000000..0bd63f7f752 --- /dev/null +++ b/dag/test/claim/observation_lockstep_witness_test.dag @@ -0,0 +1,84 @@ +module test.claim.observation_lockstep_witness_test + +import extdeps.filesystem.filesystem_io +import v2.std.live_tree { LiveTreeDisposition, ReadsLiveTree } +import std.types { String, Bool, List } +import std.observation { ObservationLaw, observation_laws, observation_heartbeat_period } +import std.measure { millisecond_count } + +data live_tree_disposition: LiveTreeDisposition = ReadsLiveTree + +data witness_note: String = "Lockstep half of the P0 acceptance (docs/plans/progress-observation-design.md section 5): the laws are data rows and the substrate model is the authority, so both are held against the live tree rather than against a restatement. Three couplings are proven by execution here. One, no orphan law — every law row names a declaration that must actually exist in std.observation, so a law cannot be a sentence nobody executes. Two, the SchedulerHold coproduct and the seed governor's HoldReason must carry the same variants, so a hold reason added on one side reds instead of silently rendering as a missing cause. Three, the dwell threshold's derivation from the heartbeat period is checked against the period the seed actually sleeps, so the row cannot drift into taste. Each check carries a negative control in the same run: a name known to be absent must read as absent, which is what separates this from a grep that passes vacuously." + +data observation_module_path: String = "dag/std/observation.dag" +data governor_source_path: String = "src/v1/stage0/src/memory_governor.rs" +data executor_source_path: String = "src/v1/stage0/src/bin/claim_executor.rs" + +fn lockstep_source(path: String) -> String { + let r = filesystem_read(path: path) + r.content +} + +fn lockstep_has(path: String, needle: String) -> Bool { + string_contains(s: lockstep_source(path: path), pattern: needle) +} + +fn w_every_law_names_a_declaration_that_exists() -> Bool { + let src = lockstep_source(path: observation_module_path) + fold(observation_laws, init: true, f: (acc, l) => + acc && string_contains(s: src, pattern: concat("fn ", concat(l.enforced_by.decl_name, "("))) + ) +} + +fn w_orphan_law_check_discriminates() -> Bool { + let src = lockstep_source(path: observation_module_path) + !string_contains(s: src, pattern: "fn observation_law_with_no_enforcer(") +} + +data governor_hold_variants: List = [ + "WindowFull", + "CurrentHighWater", + "PsiPressure", + "SwapGrowth", + "AwaitFirstCost", + "InsufficientHeadroom", + "AdmissionCeiling" +] + +fn w_scheduler_hold_is_in_lockstep_with_the_seed_governor() -> Bool { + let governor = lockstep_source(path: governor_source_path) + let model = lockstep_source(path: observation_module_path) + fold(governor_hold_variants, init: true, f: (acc, v) => + acc && + string_contains(s: governor, pattern: concat("HoldReason::", v)) && + string_contains(s: model, pattern: v) + ) +} + +fn w_lockstep_check_discriminates() -> Bool { + let governor = lockstep_source(path: governor_source_path) + let model = lockstep_source(path: observation_module_path) + !string_contains(s: governor, pattern: "HoldReason::DiskPressure") && + !string_contains(s: model, pattern: "DiskPressure") +} + +fn w_heartbeat_period_matches_the_seed_cadence() -> Bool { + millisecond_count(m: observation_heartbeat_period) == 60000 && + lockstep_has(path: executor_source_path, needle: "from_secs(60)") && + lockstep_has(path: executor_source_path, needle: "floor-memory-heartbeat") +} + +fn w_heartbeat_source_still_refuses_to_fabricate() -> Bool { + lockstep_has(path: executor_source_path, needle: "refusing to fabricate") && + lockstep_has(path: executor_source_path, needle: "render_heartbeat_line_mirror") && + lockstep_has(path: executor_source_path, needle: "heartbeat_feed_snapshot") +} + +test fn observation_lockstep_witnesses() -> Bool { + w_every_law_names_a_declaration_that_exists() && + w_orphan_law_check_discriminates() && + w_scheduler_hold_is_in_lockstep_with_the_seed_governor() && + w_lockstep_check_discriminates() && + w_heartbeat_period_matches_the_seed_cadence() && + w_heartbeat_source_still_refuses_to_fabricate() +} diff --git a/dag/test/claim/observation_model_witness_test.dag b/dag/test/claim/observation_model_witness_test.dag new file mode 100644 index 00000000000..db270fe484b --- /dev/null +++ b/dag/test/claim/observation_model_witness_test.dag @@ -0,0 +1,561 @@ +module test.claim.observation_model_witness_test + +import v2.std.live_tree { LiveTreeDisposition, SubstrateInputsOnly } +import std.types { NonEmptyStr, String, Bool, List } +import std.symbols { + Tier, + Emoji, + Unicode, + Ascii, + SymbolId, + SemanticColor, + Success, + Warning, + Error, + StatusWarning, + StatusPulse, + ChangeAdded, + ChangeModified, + ChangeRemoved, +} +import std.change { KeyedDiffAddedHunk, KeyedDiffRemovedHunk, KeyedDiffModifiedHunk } +import std.measure { Millisecond, millisecond, ByteSize, byte_size, BasisPoint, basis_point, basis_point_count } +import std.perturbation { BackwardDiffInDiffOut, ForwardSameInSameOut, perturbation_consistent } +import extdeps.render.glyphs { resolve_symbol, symbol_color, reward_animal_at, reward_animals } +import std.observation { + ObservationSegment, + RunSegment, + BatchSegment, + EntrySegment, + ModuleSegment, + PhaseSegment, + ObservationSubject, + ObservationGrain, + RunGrain, + BatchGrain, + EntryGrain, + ModuleGrain, + PhaseGrain, + observation_grain_of_segment, + observation_grain_depth, + observation_subject_grain, + observation_grain_at_or_above_entry, + observation_subject_contains, + observation_subject_path, + observation_subject_render, + Measured, + MeasuredValue, + MeasuredUnavailable, + ObservationOutcome, + Done, + Refused, + Failed, + TimedOut, + Skipped, + Final, + ObservationTransition, + Begin, + Step, + Concluded, + ObservationEvent, + SchedulerHold, + WindowFull, + CurrentHighWater, + PsiPressure, + SwapGrowth, + AwaitFirstCost, + InsufficientHeadroom, + AdmissionCeiling, + ContendedResource, + MemoryBudget, + AdmissionWindow, + MaturationReserve, + ContentionRemaining, + RemainingUnknown, + RemainingBounded, + BlockedOn, + observation_hold_resource, + observation_blocked_on, + AttentionLevel, + Ambient, + Notable, + Anomaly, + AttentionBasis, + observation_attention, + observation_outcome_is_anomaly, + ObservationEventClass, + ClassBegin, + ClassStep, + ClassDone, + ClassRefused, + ClassFailed, + ClassTimedOut, + ClassSkipped, + ClassFinal, + ClassBlocked, + ClassDwellEscalation, + observation_event_classes, + observation_event_class, + observation_class_is_intrinsic_anomaly, + ObservationDensity, + RoutineCollapsible, + SummaryAlwaysShown, + AnomalyExpanded, + observation_class_density, + observation_density_collapsible, + observation_density_expands_fully, + observation_class_glyph, + ObservationPresentation, + observation_presentation, + DwellThreshold, + observation_dwell_threshold, + observation_escalation_depth, + observation_escalation_subject, + WatchdogVerdict, + WatchdogQuiet, + WatchdogOrphanedBegin, + observation_watchdog, + observation_watchdog_attention, + ObservationLaw, + observation_laws, + ChangePresentation, + observation_change_presentation, + SelectionNoOp, + DocsPolicy, + Uncovered, + NoDeclsTouched, + DepartedPath, + selection_noop_generated_artifact, + observation_no_op_is_actually_a_widen, + observation_no_op_attention, + observation_no_op_symbol, + observation_no_op_explanation, +} + +data live_tree_disposition: LiveTreeDisposition = SubstrateInputsOnly + +data witness_note: String = "P0 acceptance for the observation model (docs/plans/progress-observation-design.md section 5): presentation totality over the class roster, the derived attention assignment, the derived BlockedOn, the recursive dwell escalation, the orphaned-begin watchdog, and the single-authority glyph perturbation. Every check here runs against the model itself; no renderer exists in this phase. Discrimination is built in rather than asserted: the Refused-versus-Failed check reds if the two ever collapse onto one glyph, the perturbation check reds if a presentation glyph is hardcoded instead of read from the glyph table, and the attention checks red if a threshold is smuggled into this module instead of taken from the supplied basis." + +fn obs_run_subject() -> ObservationSubject { + ObservationSubject { segments: [RunSegment { id: "30044816605" as NonEmptyStr }] } +} + +fn obs_entry_subject() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: "30044816605" as NonEmptyStr }, + BatchSegment { index: 3, label: "discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/effect_reach_test.dag" as NonEmptyStr } + ] + } +} + +fn obs_leaf_subject() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: "30044816605" as NonEmptyStr }, + BatchSegment { index: 3, label: "discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/effect_reach_test.dag" as NonEmptyStr }, + ModuleSegment { module_path: "v2.compiler.normalized_tree" as NonEmptyStr }, + PhaseSegment { name: "typecheck" as NonEmptyStr } + ] + } +} + +fn obs_event(transition: ObservationTransition, wall_ms: Int) -> ObservationEvent { + ObservationEvent { + subject: obs_entry_subject(), + transition: transition, + wall: MeasuredValue { value: millisecond(count: wall_ms) }, + rss: MeasuredUnavailable { cause: "cgroup memory.current unreadable in this fixture" as NonEmptyStr } + } +} + +fn obs_basis() -> AttentionBasis { + AttentionBasis { average: millisecond(count: 1000), maximum: millisecond(count: 5000) } +} + +fn obs_glyph_resolves_all_tiers(c: ObservationEventClass) -> Bool { + let p = observation_presentation(c: c) + match resolve_symbol(id: p.glyph, tier: Emoji) { + Present { value: e } => + match resolve_symbol(id: p.glyph, tier: Unicode) { + Present { value: u } => + match resolve_symbol(id: p.glyph, tier: Ascii) { + Present { value: a } => e != "" && u != "" && a != "" + Absent => false + } + Absent => false + } + Absent => false + } +} + +fn w_presentation_is_total_over_class_roster() -> Bool { + fold(observation_event_classes, init: true, f: (acc, c) => + acc && obs_glyph_resolves_all_tiers(c: c) + ) +} + +fn w_class_roster_covers_every_constructed_class() -> Bool { + count(observation_event_classes) == 10 && + observation_event_class(event: obs_event(transition: Begin, wall_ms: 1)) == ClassBegin && + observation_event_class(event: obs_event(transition: Step { k: 34, n: 61 }, wall_ms: 1)) == ClassStep && + observation_event_class(event: obs_event(transition: Concluded { outcome: Done }, wall_ms: 1)) == ClassDone && + observation_event_class(event: obs_event(transition: Concluded { outcome: Refused { diagnostic: "FLOOR-BATCH-OVER-BUDGET" as NonEmptyStr } }, wall_ms: 1)) == ClassRefused && + observation_event_class(event: obs_event(transition: Concluded { outcome: Failed { error: "panic" as NonEmptyStr, output: "" } }, wall_ms: 1)) == ClassFailed && + observation_event_class(event: obs_event(transition: Concluded { outcome: Skipped { reason: "unaffected" as NonEmptyStr } }, wall_ms: 1)) == ClassSkipped && + observation_event_class(event: obs_event(transition: Concluded { outcome: Final { shown_failures: 0 } }, wall_ms: 1)) == ClassFinal +} + +fn w_refused_is_distinct_from_failed() -> Bool { + let refused = observation_presentation(c: ClassRefused) + let failed = observation_presentation(c: ClassFailed) + let timed_out = observation_presentation(c: ClassTimedOut) + match resolve_symbol(id: refused.glyph, tier: Unicode) { + Present { value: r } => + match resolve_symbol(id: failed.glyph, tier: Unicode) { + Present { value: f } => + match resolve_symbol(id: timed_out.glyph, tier: Unicode) { + Present { value: t } => r != f && r != t && f != t + Absent => false + } + Absent => false + } + Absent => false + } +} + +fn w_anomaly_classes_never_collapse() -> Bool { + fold(observation_event_classes, init: true, f: (acc, c) => { + let p = observation_presentation(c: c) + if observation_class_is_intrinsic_anomaly(c: c) { + acc && !p.collapsible && p.expands_fully + } else { + acc && !p.expands_fully + } + }) +} + +fn w_density_is_the_single_display_authority() -> Bool { + observation_class_density(c: ClassRefused) == AnomalyExpanded && + observation_class_density(c: ClassFailed) == AnomalyExpanded && + observation_class_density(c: ClassTimedOut) == AnomalyExpanded && + observation_class_density(c: ClassBlocked) == AnomalyExpanded && + observation_class_density(c: ClassDwellEscalation) == AnomalyExpanded && + observation_class_density(c: ClassFinal) == SummaryAlwaysShown && + observation_class_density(c: ClassBegin) == RoutineCollapsible && + observation_class_density(c: ClassStep) == RoutineCollapsible && + observation_class_density(c: ClassDone) == RoutineCollapsible && + observation_class_density(c: ClassSkipped) == RoutineCollapsible +} + +fn w_presentation_projects_density_rather_than_restating_it() -> Bool { + fold(observation_event_classes, init: true, f: (acc, c) => { + let p = observation_presentation(c: c) + let d = observation_class_density(c: c) + acc && + p.collapsible == observation_density_collapsible(d: d) && + p.expands_fully == observation_density_expands_fully(d: d) && + observation_class_is_intrinsic_anomaly(c: c) == observation_density_expands_fully(d: d) + }) +} + +fn w_routine_progress_collapses_but_the_summary_always_shows() -> Bool { + observation_presentation(c: ClassBegin).collapsible && + observation_presentation(c: ClassStep).collapsible && + observation_presentation(c: ClassDone).collapsible && + observation_presentation(c: ClassSkipped).collapsible && + !observation_presentation(c: ClassFinal).collapsible && + !observation_class_is_intrinsic_anomaly(c: ClassFinal) +} + +fn w_refusal_is_anomaly_on_any_basis() -> Bool { + let refused = obs_event(transition: Concluded { outcome: Refused { diagnostic: "budget" as NonEmptyStr } }, wall_ms: 1) + let generous = AttentionBasis { average: millisecond(count: 999999), maximum: millisecond(count: 999999) } + observation_attention(event: refused, basis: generous) == Anomaly && + observation_attention(event: refused, basis: obs_basis()) == Anomaly +} + +fn w_attention_grounds_on_supplied_basis() -> Bool { + let quick = obs_event(transition: Concluded { outcome: Done }, wall_ms: 500) + let over_average = obs_event(transition: Concluded { outcome: Done }, wall_ms: 2000) + let over_maximum = obs_event(transition: Concluded { outcome: Done }, wall_ms: 9000) + observation_attention(event: quick, basis: obs_basis()) == Ambient && + observation_attention(event: over_average, basis: obs_basis()) == Notable && + observation_attention(event: over_maximum, basis: obs_basis()) == Anomaly +} + +fn w_attention_moves_with_the_basis_not_a_local_constant() -> Bool { + let event = obs_event(transition: Concluded { outcome: Done }, wall_ms: 2000) + let strict = AttentionBasis { average: millisecond(count: 100), maximum: millisecond(count: 200) } + let loose = AttentionBasis { average: millisecond(count: 10000), maximum: millisecond(count: 20000) } + perturbation_consistent( + reading: BackwardDiffInDiffOut, + output_responded: observation_attention(event: event, basis: strict) != observation_attention(event: event, basis: loose) + ) +} + +fn w_unavailable_measurement_carries_a_cause() -> Bool { + let event = obs_event(transition: Concluded { outcome: Done }, wall_ms: 1) + match event.rss { + MeasuredValue { value: _ } => false + MeasuredUnavailable { cause: c } => c != "" + } +} + +fn w_blocked_on_resource_is_derived_for_every_hold() -> Bool { + observation_hold_resource(hold: WindowFull { active: 4, target: 4 }) == AdmissionWindow && + observation_hold_resource(hold: CurrentHighWater { current: byte_size(1), high_water: byte_size(2) }) == MemoryBudget && + observation_hold_resource(hold: PsiPressure { avg10: basis_point(count: 900) }) == MemoryBudget && + observation_hold_resource(hold: SwapGrowth { delta: byte_size(1) }) == MemoryBudget && + observation_hold_resource(hold: AwaitFirstCost { undigested: 2 }) == MaturationReserve && + observation_hold_resource(hold: InsufficientHeadroom { current: byte_size(1), share: byte_size(1), high_water: byte_size(2) }) == MemoryBudget && + observation_hold_resource(hold: AdmissionCeiling { current: byte_size(1), ceiling: byte_size(2) }) == MaturationReserve +} + +fn w_blocked_on_carries_holders_and_named_remaining() -> Bool { + let b = observation_blocked_on( + hold: PsiPressure { avg10: basis_point(count: 900) }, + holders: [obs_leaf_subject()], + remaining: RemainingUnknown { cause: "governor exposes no bound for reclaim" as NonEmptyStr } + ) + count(b.holders) == 1 && + b.resource == MemoryBudget && + match b.remaining { + RemainingUnknown { cause: c } => c != "" + RemainingBounded { by: _ } => false + } +} + +fn w_escalation_depth_is_recursive_not_one_level() -> Bool { + let t = observation_dwell_threshold + observation_escalation_depth(quiet: millisecond(count: 0), t: t) == 0 && + observation_escalation_depth(quiet: millisecond(count: 60000), t: t) == 1 && + observation_escalation_depth(quiet: millisecond(count: 120000), t: t) == 2 && + observation_escalation_depth(quiet: millisecond(count: 240000), t: t) == 4 +} + +fn w_escalation_surfaces_deeper_subject_over_time() -> Bool { + let t = observation_dwell_threshold + let quiet = obs_entry_subject() + let leaf = obs_leaf_subject() + let at_t = observation_escalation_subject(quiet_subject: quiet, active_leaf: leaf, quiet: millisecond(count: 60000), t: t) + let at_2t = observation_escalation_subject(quiet_subject: quiet, active_leaf: leaf, quiet: millisecond(count: 120000), t: t) + count(at_t.segments) == 4 && + count(at_2t.segments) == 5 && + observation_subject_contains(ancestor: at_t, descendant: at_2t) +} + +fn w_escalation_refuses_to_surface_an_unrelated_subject() -> Bool { + let t = observation_dwell_threshold + let unrelated = ObservationSubject { segments: [RunSegment { id: "other-run" as NonEmptyStr }] } + let surfaced = observation_escalation_subject( + quiet_subject: obs_entry_subject(), + active_leaf: unrelated, + quiet: millisecond(count: 240000), + t: t + ) + count(surfaced.segments) == 3 +} + +fn w_watchdog_converts_quiet_past_budget_into_a_typed_line() -> Bool { + let within = observation_watchdog(subject: obs_entry_subject(), elapsed: millisecond(count: 1000), budget: millisecond(count: 5000)) + let past = observation_watchdog(subject: obs_entry_subject(), elapsed: millisecond(count: 607000), budget: millisecond(count: 60000)) + observation_watchdog_attention(v: within) == Ambient && + observation_watchdog_attention(v: past) == Anomaly && + match past { + WatchdogQuiet => false + WatchdogOrphanedBegin { subject: s, elapsed: _, budget: _ } => count(s.segments) == 3 + } +} + +fn w_subject_identity_is_rendered_not_anonymous() -> Bool { + let rendered = observation_subject_render(subject: obs_leaf_subject()) + string_contains(s: rendered, pattern: "module=v2.compiler.normalized_tree") && + string_contains(s: rendered, pattern: "phase=typecheck") && + string_contains(s: rendered, pattern: "entry=") && + string_contains(s: rendered, pattern: "batch=") +} + +fn w_containment_holds_along_the_path() -> Bool { + observation_subject_contains(ancestor: obs_run_subject(), descendant: obs_leaf_subject()) && + observation_subject_contains(ancestor: obs_entry_subject(), descendant: obs_leaf_subject()) && + !observation_subject_contains(ancestor: obs_leaf_subject(), descendant: obs_run_subject()) +} + +fn w_grain_orders_the_containment_tree() -> Bool { + observation_subject_grain(subject: obs_run_subject()) == RunGrain && + observation_subject_grain(subject: obs_entry_subject()) == EntryGrain && + observation_subject_grain(subject: obs_leaf_subject()) == PhaseGrain && + observation_grain_at_or_above_entry(g: RunGrain) && + observation_grain_at_or_above_entry(g: EntryGrain) && + !observation_grain_at_or_above_entry(g: PhaseGrain) +} + +fn w_presentation_glyph_reads_the_table_not_a_hardcoded_string() -> Bool { + let p = observation_presentation(c: ClassRefused) + match resolve_symbol(id: p.glyph, tier: Unicode) { + Present { value: u } => + match resolve_symbol(id: p.glyph, tier: Ascii) { + Present { value: a } => + perturbation_consistent(reading: BackwardDiffInDiffOut, output_responded: u != a) && + a == "REFUSED" + Absent => false + } + Absent => false + } +} + +fn w_presentation_colour_comes_from_the_glyph_authority() -> Bool { + fold(observation_event_classes, init: true, f: (acc, c) => { + let p = observation_presentation(c: c) + acc && match symbol_color(id: p.glyph) { + Present { value: _ } => true + Absent => false + } + }) +} + +fn w_reward_animal_selection_is_deterministic_and_wraps() -> Bool { + let n = count(reward_animals) + n > 0 && + match reward_animal_at(index: 3, tier: Ascii) { + Present { value: first } => + match reward_animal_at(index: 3, tier: Ascii) { + Present { value: again } => + match reward_animal_at(index: 3 + n, tier: Ascii) { + Present { value: wrapped } => first == again && first == wrapped + Absent => false + } + Absent => false + } + Absent => false + } +} + +fn w_every_law_is_stated_and_bound_to_an_enforcer() -> Bool { + count(observation_laws) == 5 && + fold(observation_laws, init: true, f: (acc, l) => + acc && l.id != "" && l.statement != "" && l.enforced_by.module_path == "std.observation" && l.enforced_by.decl_name != "" + ) +} + +fn w_change_kind_projects_from_the_one_diff_carrier() -> Bool { + let added = observation_change_presentation(hunk: KeyedDiffAddedHunk { hunk_key: "a", to: 1 }) + let removed = observation_change_presentation(hunk: KeyedDiffRemovedHunk { hunk_key: "a", from: 1 }) + let modified = observation_change_presentation(hunk: KeyedDiffModifiedHunk { hunk_key: "a", from: 1, to: 2 }) + added.tag == "new" && + removed.tag == "removed" && + modified.tag == "edited" && + added.symbol == ChangeAdded && + removed.symbol == ChangeRemoved && + modified.symbol == ChangeModified +} + +fn w_change_kind_colour_follows_git_diff_convention() -> Bool { + match symbol_color(id: ChangeAdded) { + Present { value: a } => + match symbol_color(id: ChangeModified) { + Present { value: m } => + match symbol_color(id: ChangeRemoved) { + Present { value: r } => a == Success && m == Warning && r == Error + Absent => false + } + Absent => false + } + Absent => false + } +} + +fn w_change_kind_degrades_losslessly_without_colour() -> Bool { + let added = observation_change_presentation(hunk: KeyedDiffAddedHunk { hunk_key: "a", to: 1 }) + let removed = observation_change_presentation(hunk: KeyedDiffRemovedHunk { hunk_key: "a", from: 1 }) + let modified = observation_change_presentation(hunk: KeyedDiffModifiedHunk { hunk_key: "a", from: 1, to: 2 }) + added.tag != "" && removed.tag != "" && modified.tag != "" && + added.tag != removed.tag && added.tag != modified.tag && removed.tag != modified.tag +} + +fn w_no_op_kinds_each_explain_themselves() -> Bool { + let kinds = [ + DocsPolicy, + Uncovered { declaration: "std.observation.observation_attention" as NonEmptyStr }, + NoDeclsTouched, + selection_noop_generated_artifact(drift_gate_authority: "gunbc.ci_workflow" as NonEmptyStr), + DepartedPath { cause: "src/v1/stage0/src/gone.rs" as NonEmptyStr } + ] + count(kinds) == 5 && + fold(kinds, init: true, f: (acc, n) => { + acc && + observation_no_op_explanation(n: n) != "" && + match symbol_color(id: observation_no_op_symbol(n: n)) { + Present { value: _ } => true + Absent => false + } + }) +} + +fn w_uncovered_is_a_nudge_not_a_comfort() -> Bool { + observation_no_op_attention(n: Uncovered { declaration: "std.observation.observation_watchdog" as NonEmptyStr }) == Notable && + observation_no_op_attention(n: DocsPolicy) == Ambient && + observation_no_op_symbol(n: Uncovered { declaration: "x" as NonEmptyStr }) == StatusWarning +} + +fn w_departed_path_is_not_a_no_op() -> Bool { + observation_no_op_is_actually_a_widen(n: DepartedPath { cause: "deleted module" as NonEmptyStr }) && + !observation_no_op_is_actually_a_widen(n: DocsPolicy) && + !observation_no_op_is_actually_a_widen(n: NoDeclsTouched) && + !observation_no_op_is_actually_a_widen(n: selection_noop_generated_artifact(drift_gate_authority: "x" as NonEmptyStr)) && + !observation_no_op_is_actually_a_widen(n: Uncovered { declaration: "x" as NonEmptyStr }) && + observation_no_op_attention(n: DepartedPath { cause: "deleted module" as NonEmptyStr }) == Notable +} + +fn w_psi_pressure_carries_a_scaled_magnitude_not_a_scalar() -> Bool { + let hold = PsiPressure { avg10: basis_point(count: 3750) } + match hold { + PsiPressure { avg10: r } => basis_point_count(bp: r) == 3750 + _ => false + } +} + +fn w_status_pulse_glyph_is_available_to_the_heartbeat() -> Bool { + match resolve_symbol(id: StatusPulse, tier: Emoji) { + Present { value: e } => e == "🕐" + Absent => false + } +} + +test fn observation_model_witnesses() -> Bool { + w_presentation_is_total_over_class_roster() && + w_class_roster_covers_every_constructed_class() && + w_refused_is_distinct_from_failed() && + w_anomaly_classes_never_collapse() && + w_density_is_the_single_display_authority() && + w_presentation_projects_density_rather_than_restating_it() && + w_routine_progress_collapses_but_the_summary_always_shows() && + w_refusal_is_anomaly_on_any_basis() && + w_attention_grounds_on_supplied_basis() && + w_attention_moves_with_the_basis_not_a_local_constant() && + w_unavailable_measurement_carries_a_cause() && + w_blocked_on_resource_is_derived_for_every_hold() && + w_blocked_on_carries_holders_and_named_remaining() && + w_escalation_depth_is_recursive_not_one_level() && + w_escalation_surfaces_deeper_subject_over_time() && + w_escalation_refuses_to_surface_an_unrelated_subject() && + w_watchdog_converts_quiet_past_budget_into_a_typed_line() && + w_subject_identity_is_rendered_not_anonymous() && + w_containment_holds_along_the_path() && + w_grain_orders_the_containment_tree() && + w_presentation_glyph_reads_the_table_not_a_hardcoded_string() && + w_presentation_colour_comes_from_the_glyph_authority() && + w_reward_animal_selection_is_deterministic_and_wraps() && + w_every_law_is_stated_and_bound_to_an_enforcer() && + w_change_kind_projects_from_the_one_diff_carrier() && + w_change_kind_colour_follows_git_diff_convention() && + w_change_kind_degrades_losslessly_without_colour() && + w_no_op_kinds_each_explain_themselves() && + w_uncovered_is_a_nudge_not_a_comfort() && + w_departed_path_is_not_a_no_op() && + w_psi_pressure_carries_a_scaled_magnitude_not_a_scalar() && + w_status_pulse_glyph_is_available_to_the_heartbeat() +} diff --git a/dag/test/claim/observation_tty_render_witness_test.dag b/dag/test/claim/observation_tty_render_witness_test.dag new file mode 100644 index 00000000000..ff99d37909f --- /dev/null +++ b/dag/test/claim/observation_tty_render_witness_test.dag @@ -0,0 +1,174 @@ +module test.claim.observation_tty_render_witness_test + +import v2.std.live_tree { LiveTreeDisposition, SubstrateInputsOnly } +import std.types { NonEmptyStr, String, Bool, List } +import std.nat { Nat } +import std.symbols { Tier, Emoji, Ascii, Spinner0, StatusBlocked, StatusFinal } +import std.measure { millisecond, byte_size, basis_point } +import std.render { CursorAction, Overwrite, Append, RenderCapability } +import std.observation { + ObservationSubject, + RunSegment, + BatchSegment, + EntrySegment, + ModuleSegment, + PhaseSegment, + MeasuredValue, + MeasuredUnavailable, + ObservationEvent, + Concluded, + Done, + Refused, + Failed, + AttentionBasis, + Ambient, + Notable, + Anomaly, + BlockedOn, + MemoryBudget, + RemainingBounded, + RemainingUnknown, +} +import gunbc.observation_tty_render { + PreambleFields, + tty_preamble_line, + tty_cursor_action, + tty_group_line, + tty_blocked_line, + tty_final_line, + tty_event_emission, + tty_render_emission, + TtyEmission, +} + +data live_tree_disposition: LiveTreeDisposition = SubstrateInputsOnly + +data witness_note: String = "P2 acceptance (docs/plans/progress-observation-design.md section 3b): the TTY renderer is a SIBLING of the CI renderer over the same std.observation carriers, differing only where the medium differs. The witnesses prove the sibling property by execution — the same event drives both surfaces and moves together — plus the TTY-specific facts: a required preamble, in-place repaint derived from attention, inline BlockedOn with named remaining, and a deterministic reward animal on Final. The three upgrades over the reference (durations, Refused vs Failed, recursive escalation) are inherited from the shared carriers, not re-earned." + +fn tty_cap() -> RenderCapability { + RenderCapability { color: true, tier: Emoji, cursor_addressable: true } +} + +fn pipe_cap() -> RenderCapability { + RenderCapability { color: false, tier: Ascii, cursor_addressable: false } +} + +fn tty_entry_subject() -> ObservationSubject { + ObservationSubject { + segments: [ + RunSegment { id: "30044816605" as NonEmptyStr }, + BatchSegment { index: 3, label: "witness discovery" as NonEmptyStr }, + EntrySegment { source_path: "dag/test/claim/effect_reach_test.dag" as NonEmptyStr }, + ModuleSegment { module_path: "v2.compiler.normalized_tree" as NonEmptyStr }, + PhaseSegment { name: "typecheck" as NonEmptyStr } + ] + } +} + +fn tty_basis() -> AttentionBasis { + AttentionBasis { average: millisecond(count: 1000), maximum: millisecond(count: 5000) } +} + +fn w_preamble_names_what_why_and_how_much() -> Bool { + let line = tty_preamble_line(fields: PreambleFields { + run_id: "30044816605" as NonEmptyStr, + trigger: "pull_request" as NonEmptyStr, + diff_summary: "3 files touched" as NonEmptyStr, + entry_total: 602 + }) + string_contains(s: line.text, pattern: "30044816605") && + string_contains(s: line.text, pattern: "pull_request") && + string_contains(s: line.text, pattern: "3 files touched") && + string_contains(s: line.text, pattern: "602 entries") +} + +fn w_routine_repaints_in_place_but_anomaly_scrolls() -> Bool { + tty_cursor_action(attention: Ambient, cap: tty_cap()) == Overwrite && + tty_cursor_action(attention: Notable, cap: tty_cap()) == Append && + tty_cursor_action(attention: Anomaly, cap: tty_cap()) == Append +} + +fn w_non_addressable_terminal_always_appends() -> Bool { + tty_cursor_action(attention: Ambient, cap: pipe_cap()) == Append && + tty_cursor_action(attention: Anomaly, cap: pipe_cap()) == Append +} + +fn w_refusal_never_repaints_away() -> Bool { + let refusal = ObservationEvent { + subject: tty_entry_subject(), + transition: Concluded { outcome: Refused { diagnostic: "FLOOR-BATCH-OVER-BUDGET" as NonEmptyStr } }, + wall: MeasuredValue { value: millisecond(count: 2472000) }, + rss: MeasuredUnavailable { cause: "not sampled" as NonEmptyStr } + } + let emission = tty_event_emission(event: refusal, basis: tty_basis(), cap: tty_cap()) + emission.line.attention == Anomaly && + emission.cursor == Append +} + +fn w_same_event_drives_both_surfaces() -> Bool { + let event = ObservationEvent { + subject: tty_entry_subject(), + transition: Concluded { outcome: Done }, + wall: MeasuredValue { value: millisecond(count: 230) }, + rss: MeasuredUnavailable { cause: "not sampled" as NonEmptyStr } + } + let emission = tty_event_emission(event: event, basis: tty_basis(), cap: tty_cap()) + string_contains(s: tty_render_emission(emission: emission, tier: Emoji), pattern: "in 230ms") +} + +fn w_group_line_names_the_current_entry() -> Bool { + let line = tty_group_line(active: tty_entry_subject(), k: 214, n: 602, spinner: Spinner0) + string_contains(s: line.text, pattern: "214 of 602") && + string_contains(s: line.text, pattern: "v2.compiler.normalized_tree") && + line.attention == Ambient +} + +fn w_blocked_line_carries_holder_and_bounded_remaining() -> Bool { + let bounded = tty_blocked_line(blocked: BlockedOn { + resource: MemoryBudget, + holders: [tty_entry_subject()], + remaining: RemainingBounded { by: millisecond(count: 80000) } + }) + string_contains(s: bounded.text, pattern: "1 holder(s)") && + string_contains(s: bounded.text, pattern: "80 seconds remaining") && + bounded.attention == Anomaly +} + +fn w_blocked_line_names_why_remaining_is_unknown() -> Bool { + let unknown = tty_blocked_line(blocked: BlockedOn { + resource: MemoryBudget, + holders: [tty_entry_subject()], + remaining: RemainingUnknown { cause: "governor exposes no reclaim bound" as NonEmptyStr } + }) + string_contains(s: unknown.text, pattern: "no estimate (governor exposes no reclaim bound)") && + !string_contains(s: unknown.text, pattern: "remaining") +} + +fn w_reward_animal_is_deterministic_and_degrades_to_a_word() -> Bool { + let emoji_clean = tty_final_line(shown_failures: 0, run_index: 3, tier: Emoji) + let emoji_again = tty_final_line(shown_failures: 0, run_index: 3, tier: Emoji) + let ascii_clean = tty_final_line(shown_failures: 0, run_index: 3, tier: Ascii) + emoji_clean.text == emoji_again.text && + string_contains(s: emoji_clean.text, pattern: "all clean") && + emoji_clean.attention == Ambient && + !string_contains(s: ascii_clean.text, pattern: "🦦") +} + +fn w_final_with_failures_names_them_and_is_anomaly() -> Bool { + let line = tty_final_line(shown_failures: 3, run_index: 3, tier: Emoji) + string_contains(s: line.text, pattern: "3 problems shown") && + line.attention == Anomaly +} + +test fn observation_tty_render_witnesses() -> Bool { + w_preamble_names_what_why_and_how_much() && + w_routine_repaints_in_place_but_anomaly_scrolls() && + w_non_addressable_terminal_always_appends() && + w_refusal_never_repaints_away() && + w_same_event_drives_both_surfaces() && + w_group_line_names_the_current_entry() && + w_blocked_line_carries_holder_and_bounded_remaining() && + w_blocked_line_names_why_remaining_is_unknown() && + w_reward_animal_is_deterministic_and_degrades_to_a_word() && + w_final_with_failures_names_them_and_is_anomaly() +} diff --git a/dag/test/claim/output_policy_witness_test.dag b/dag/test/claim/output_policy_witness_test.dag index f0cd03f0d72..9be94b8ad37 100644 --- a/dag/test/claim/output_policy_witness_test.dag +++ b/dag/test/claim/output_policy_witness_test.dag @@ -35,8 +35,8 @@ fn w_instrumentation_is_debug_only() -> Bool { dec_is(d: channel_decision(channel: Instrumentation, verbosity: Verbose), suppressed: false, condensed: false, full: true) } -fn w_shell_trace_condenses_at_normal() -> Bool { - dec_is(d: channel_decision(channel: ShellTrace, verbosity: Normal), suppressed: false, condensed: true, full: false) && +fn w_shell_trace_is_debug_only_at_normal() -> Bool { + dec_is(d: channel_decision(channel: ShellTrace, verbosity: Normal), suppressed: true, condensed: false, full: false) && dec_is(d: channel_decision(channel: ShellTrace, verbosity: Quiet), suppressed: true, condensed: false, full: false) && dec_is(d: channel_decision(channel: ShellTrace, verbosity: Verbose), suppressed: false, condensed: false, full: true) } @@ -71,7 +71,7 @@ fn w_resolve_channel_policy_normal_bundles_decisions() -> Bool { dec_is(d: p.diagnostic, suppressed: false, condensed: false, full: true) && dec_is(d: p.claim_result, suppressed: false, condensed: false, full: true) && dec_is(d: p.progress, suppressed: false, condensed: true, full: false) && - dec_is(d: p.shell_trace, suppressed: false, condensed: true, full: false) && + dec_is(d: p.shell_trace, suppressed: true, condensed: false, full: false) && dec_is(d: p.instrumentation, suppressed: true, condensed: false, full: false) } @@ -138,11 +138,15 @@ fn w_neutralization_is_the_only_route_to_surfaced_text() -> Bool { subject_text_line_guard == "| " } -fn w_quiet_suppresses_and_verbose_surfaces_regardless_of_outcome() -> Bool { +data w_quiet_ambient_silent_but_divergence_still_surfaces_note: String = "Observation dissolve: Quiet suppresses Ambient ShellTrace scaffolding, but stream disposition follows divergence — Anomaly is never silenced by channel Suppressed (DESIGN section 5). Verbose still surfaces every stream (Full)." + +fn w_quiet_ambient_silent_but_divergence_still_surfaces() -> Bool { disp_is(d: shell_stream(verbosity: Quiet, expected: ExpectFailure, observed: ObservedSuccess), - surface: false, counts: false, suppressed: true) && + surface: true, counts: false, suppressed: false) && disp_is(d: shell_stream(verbosity: Quiet, expected: ExpectSuccess, observed: observed_outcome_from_exit(exit: 1)), - surface: false, counts: false, suppressed: true) && + surface: true, counts: false, suppressed: false) && + disp_is(d: shell_stream(verbosity: Quiet, expected: ExpectSuccess, observed: ObservedSuccess), + surface: false, counts: true, suppressed: false) && disp_is(d: shell_stream(verbosity: Verbose, expected: ExpectSuccess, observed: ObservedSuccess), surface: true, counts: false, suppressed: false) && disp_is(d: shell_stream(verbosity: Verbose, expected: ExpectFailure, observed: observed_outcome_from_exit(exit: 1)), @@ -169,25 +173,42 @@ fn w_shell_trace_stream_policy_projects_the_four_corners() -> Bool { disp_is(d: p.expect_success_observed_failure, surface: true, counts: false, suppressed: false) && disp_is(d: p.expect_failure_observed_success, surface: true, counts: false, suppressed: false) && disp_is(d: p.expect_failure_observed_failure, surface: false, counts: true, suppressed: false) && + disp_is(d: p.outcome_is_data_observed_success, surface: false, counts: true, suppressed: false) && + disp_is(d: p.outcome_is_data_observed_failure, surface: false, counts: true, suppressed: false) && p.subject_line_guard == "| " } +data outcome_is_data_discrimination_note: String = "The two OutcomeIsData corners above both assert counts-not-surface, which on its own is the shape a rubber stamp would also have — so it is NOT the discriminating claim. The claim that discriminates is w_outcome_is_data_is_exit_invariant below: under ExpectSuccess and ExpectFailure the disposition FLIPS between the two observed poles, and under OutcomeIsData it must NOT, because the exit is an answer rather than a verdict. An implementation that quietly treated OutcomeIsData as an alias for ExpectSuccess would pass the corner assertions and fail this one." + +fn disp_surfaces(d: StreamDisposition) -> Bool { + disp_is(d: d, surface: true, counts: false, suppressed: false) +} + +fn w_outcome_is_data_is_exit_invariant() -> Bool { + let p = resolve_shell_trace_stream_policy(verbose: false, quiet: false) + let es_flips = disp_surfaces(d: p.expect_success_observed_failure) && !disp_surfaces(d: p.expect_success_observed_success) + let ef_flips = disp_surfaces(d: p.expect_failure_observed_success) && !disp_surfaces(d: p.expect_failure_observed_failure) + let oid_still = disp_surfaces(d: p.outcome_is_data_observed_success) == disp_surfaces(d: p.outcome_is_data_observed_failure) + es_flips && ef_flips && oid_still +} + test fn output_policy_effect_stream_witnesses() -> Bool { w_divergence_surfaces_agreement_counts_at_normal() && w_red_control_failing_as_declared_does_not_replay_content() && w_red_control_that_unexpectedly_succeeds_surfaces() && w_surfaced_subject_text_cannot_mint_annotations() && w_neutralization_is_the_only_route_to_surfaced_text() && - w_quiet_suppresses_and_verbose_surfaces_regardless_of_outcome() && + w_quiet_ambient_silent_but_divergence_still_surfaces() && w_effect_stream_disposition_is_exit_invariant() && w_expectation_axis_is_load_bearing_red_control() && - w_shell_trace_stream_policy_projects_the_four_corners() + w_shell_trace_stream_policy_projects_the_four_corners() && + w_outcome_is_data_is_exit_invariant() } test fn output_policy_witnesses() -> Bool { w_diagnostic_never_hidden() && w_instrumentation_is_debug_only() && - w_shell_trace_condenses_at_normal() && + w_shell_trace_is_debug_only_at_normal() && w_claim_result_shown_at_normal_hidden_at_quiet() && w_progress_condenses_at_normal() && w_verbose_wins_over_quiet() && diff --git a/dag/test/claim/render_glyphs_ansi_witness_test.dag b/dag/test/claim/render_glyphs_ansi_witness_test.dag index e0b01326f05..efbdb87c2d2 100644 --- a/dag/test/claim/render_glyphs_ansi_witness_test.dag +++ b/dag/test/claim/render_glyphs_ansi_witness_test.dag @@ -50,10 +50,28 @@ fn w_success_ansi_is_256_green() -> Bool { } } +data ansi_escape_is_a_real_byte_note: String = "w_success_ansi_is_256_green above compares one \\x1b literal against another, so it agreed with itself for as long as the tokenizer left BOTH undecoded — it passed throughout the period when every colour this repo authored rendered as the literal text backslash-x-1-b (seen in CI run 30167957464's TOP SLOWEST WITNESSES table). Equality against a same-source literal can never discriminate a decode defect. The two witnesses below do: they read the CODE POINT of the first character (27 = ESC, never 92 = backslash) and pin the decoded length, both of which are properties of the decode rather than of the spelling." + +fn w_ansi_first_char_is_escape_byte() -> Bool { + match ansi_code(c: Success) { + Present { value: code } => code_point(char_at(s: code, pos: 0)) == 27 + Absent => false + } +} + +fn w_ansi_code_has_no_literal_backslash() -> Bool { + match ansi_code(c: Success) { + Present { value: code } => !string_contains(s: code, pattern: "\\") + Absent => false + } +} + test fn render_glyphs_ansi_witnesses() -> Bool { w_completed_ascii() && w_completed_unicode_differs_from_ascii() && w_glyph_at_selects_matching_tier_text() && w_failed_color_is_error() && - w_success_ansi_is_256_green() + w_success_ansi_is_256_green() && + w_ansi_first_char_is_escape_byte() && + w_ansi_code_has_no_literal_backslash() } diff --git a/dag/test/claim/typed_witness_invocation_test.dag b/dag/test/claim/typed_witness_invocation_test.dag index e983288cbcf..1ec79957b1f 100644 --- a/dag/test/claim/typed_witness_invocation_test.dag +++ b/dag/test/claim/typed_witness_invocation_test.dag @@ -1,5 +1,6 @@ module test.claim.typed_witness_invocation +import gunbc.output_policy { ExpectFailure } import extdeps.shell import extdeps.git import extdeps.gunbc @@ -20,7 +21,7 @@ test fn typed_is_executable_positive_holds() -> Bool { } test fn typed_is_executable_negative_red_control_holds() -> Bool { - let result = shell.Test.IsExecutable(path: "/nonexistent/no/such/binary") + let result = shell.Test.IsExecutable(path: "/nonexistent/no/such/binary", expect: ExpectFailure) result.executable == false } @@ -30,7 +31,7 @@ test fn typed_witness_bin_run_argv_holds() -> Bool { } test fn typed_witness_bin_run_nonzero_red_control_holds() -> Bool { - let result = gunbc.WitnessBin.Run(workdir: "/", bin_path: "/bin/false") + let result = gunbc.WitnessBin.Run(workdir: "/", bin_path: "/bin/false", expect: ExpectFailure) result.success == false } @@ -60,7 +61,7 @@ test fn typed_is_nonempty_positive_holds() -> Bool { } test fn typed_is_nonempty_zero_byte_red_control_holds() -> Bool { - let result = shell.Test.IsNonEmpty(path: "/dev/null") + let result = shell.Test.IsNonEmpty(path: "/dev/null", expect: ExpectFailure) result.nonempty == false } @@ -93,7 +94,7 @@ test fn typed_claim_batch_pooled_argv_shape_holds() -> Bool { } test fn typed_claim_batch_pooled_run_red_control_holds() -> Bool { - let result = gunbc.WitnessBin.Run(workdir: "/", bin_path: "/bin/false", args: ["--entry", "x.dag"]) + let result = gunbc.WitnessBin.Run(workdir: "/", bin_path: "/bin/false", args: ["--entry", "x.dag"], expect: ExpectFailure) result.success == false } @@ -101,7 +102,8 @@ test fn typed_claim_executor_verify_build_artifacts_red_control_holds() -> Bool let result = claim_executor.Executor.VerifyBuildArtifacts( workdir: "/", bin_path: "/bin/false", - artifact_paths: ["/bin/sh"] + artifact_paths: ["/bin/sh"], + expect: ExpectFailure ) result.success == false } diff --git a/dag/tools/dag_compile_clean_transport.dag b/dag/tools/dag_compile_clean_transport.dag index 429ec0bf609..5e2b52aced6 100644 --- a/dag/tools/dag_compile_clean_transport.dag +++ b/dag/tools/dag_compile_clean_transport.dag @@ -5,6 +5,7 @@ import extdeps.filesystem.filesystem_io import extdeps.git import extdeps.gunbc import gunbc.ci_layer_roots { compile_clean_source_roots } +import gunbc.output_policy { ExpectFailure, ExpectSuccess } import std.logic { Bool } import tools.host_prelude { witness_bin, witness_bin_ready, witness_bin_release_path } @@ -105,7 +106,8 @@ fn dcc_perturb_compile_with_args( let compile = gunbc.WitnessBin.Run( workdir: root.path, bin_path: dcc_gunbc_path(root: root.path), - args: compile_args + args: compile_args, + expect: if expect_success { ExpectSuccess } else { ExpectFailure } ) let cleaned = shell.Remove.RecursiveForce(path: tmp_path) wrote.success && cleaned.success && (compile.success == expect_success) diff --git a/dag/tools/host_prelude.dag b/dag/tools/host_prelude.dag index fa1e4718042..4b6b2895634 100644 --- a/dag/tools/host_prelude.dag +++ b/dag/tools/host_prelude.dag @@ -7,6 +7,7 @@ import extdeps.git import extdeps.gunbc import gunbc.cli_services import gunbc.cli_invoke { source_root_transport_argv } +import gunbc.output_policy { OutcomeIsData } import std.logic { Bool } data witness_invocation_doc: String = "Shared witness-binary invocation. A dag/tools transport states WHICH binaries must exist (WitnessBin: prereqs + the one to invoke) and its argv Strings; this module realizes that intent as TYPED service operations (shell.Test.IsExecutable, cargo.Build.Build, gunbc.WitnessBin.Run at the git.Core.Toplevel root) - no bash is serialized or executed on the run path, per the shell-emission-model rule that runtime-present sites are argv invocations wearing bash syntax and need no shell. The former ShellProgram builders (witness_bin_program, gunbc_claims_program) and ensure_bin_built_and_verified shell-stmt chain dissolved with the typed WitnessBin path (Phase D, 2026-07-14); toolchain_provision_shell_exec is the only surviving consumer of this module's shell path." @@ -54,12 +55,14 @@ type WitnessBinReadiness = WitnessBinReady { path: String } | WitnessBinRefused { bin_name: String, path: String, reason: WitnessBinRefusalReason } +data witness_bin_readiness_outcome_is_data_note: String = "The three probes below are annotated OutcomeIsData (operator ruling 2026-07-25). A non-zero exit from `test -f`/`-x`/`-s` here means SUBJECT ABSENT — an answer, not a fault — and it is ambient in BOTH of this helper's contexts: the production readiness path (witness_bin_run_if_ready, below) responds by building, and the build-artifact red controls respond by asserting the refusal arm. Because the arm is correct for every caller, it is annotatable HERE, once, with no expectation threaded through this production signature and no ambient frame left open. ADMISSIBILITY (the §5 guard in gunbc.output_policy outcome_is_data_note): this fn returns WitnessBinReadiness — a typed verdict whose arms ARE the adjudication — never unit, so the exit provably feeds a typed consumer. What stays anomalous is dispatch failure: if the probe itself cannot run, that is not an exit code and does not travel this arm." + fn witness_bin_artifact_readiness(bin_name: String, path: String) -> WitnessBinReadiness { - let is_file = shell.Test.IsFile(path: path) + let is_file = shell.Test.IsFile(path: path, expect: OutcomeIsData) if is_file.is_file { - let exec = shell.Test.IsExecutable(path: path) + let exec = shell.Test.IsExecutable(path: path, expect: OutcomeIsData) if exec.executable { - let nonempty = shell.Test.IsNonEmpty(path: path) + let nonempty = shell.Test.IsNonEmpty(path: path, expect: OutcomeIsData) if nonempty.nonempty { WitnessBinReady { path: path } } else { @@ -80,9 +83,11 @@ fn witness_bin_readiness_is_ready(readiness: WitnessBinReadiness) -> Bool { } } +data witness_bin_ensure_built_probe_note: String = "The presence probe below is OutcomeIsData for the same reason as witness_bin_artifact_readiness: a non-zero `test -x` means NOT BUILT YET, which is the question this fn asks, and the production arm answers it by building while the red control answers it by asserting refusal. The cargo.Build.BuildInheritEnv call further down is DELIBERATELY NOT annotated, and the distinction is worth stating because both effects sit in the same typed fn. The §5 guard's LETTER admits both — this fn returns WitnessBinReadiness and the build's exit is folded into WitnessBinRefused. Its SPIRIT does not: a failing `cargo build` in the production path is a genuine fault an engineer wants shouted, not an observation about the subject's state. Silencing it would trade a fake anomaly (which the guard exists to remove) for a hidden real one (which is what the whole arc is about). So it stays loud pending an operator call; the residual ❌ from the unbuildable red control is that decision showing, not an oversight." + fn witness_bin_ensure_built_typed(root: String, bin: WitnessBin) -> WitnessBinReadiness { let path = witness_bin_release_path(root: root, bin_name: bin.bin_name) - let present = shell.Test.IsExecutable(path: path) + let present = shell.Test.IsExecutable(path: path, expect: OutcomeIsData) if present.executable { witness_bin_artifact_readiness(bin_name: bin.bin_name, path: path) } else { diff --git a/docs/plans/shell-to-dag-residual-census-and-arc-completion.md b/docs/plans/shell-to-dag-residual-census-and-arc-completion.md index 8d1c08be6aa..bf5eab16402 100644 --- a/docs/plans/shell-to-dag-residual-census-and-arc-completion.md +++ b/docs/plans/shell-to-dag-residual-census-and-arc-completion.md @@ -13,6 +13,19 @@ So srv1/srv2 subsumption and srv3 bringup are the **same** work seen twice: gene --- +## 0b. Finding — modeled ops whose ONLY realization is a shell escape where a NATIVE handler is correct (transport-decomposition lane; filed 2026-07-24, do-not-fix-here) + +A distinct axis from §1's *emission* census: these are ops whose **interface shape is right** but whose **single hardwired transport is wrong** — the verbatim §3(b) "single hardwired transport is the N×M-adapter trap" tell. The shape belongs to the dependency; the transport is a Realization *handler, one of N* (§2). Each of these has exactly one handler — `shell` — where a **native in-process handler** is the correct realization when locality is `OnTarget`, and `shell`/`ssh` is the handler only when the target is another process on another host. + +| op (extdeps) | modeled transport | native handler that's missing | receipt | +| --- | --- | --- | --- | +| `shell.Env.Get` (`extdeps/shell/shell.dag:42`) | `shell { argv: ["printenv", "{name}"] }` | `std::env::var` — reading an env var **the process already holds in its own environment**. Realized today as `wet_env_var` spawning `printenv` (`v1_interpreter.rs:5096`). Reading your own environment isn't a host effect at all. | floor diff-observation spawns 5 `printenv` children per pass (compile-clean scope + discovery selection), repeated per floor pass — pure log clutter; ~ms each, **not a floor-time lever** | +| `shell.Which.Check` (`extdeps/shell/shell.dag:57`) | `command -v` (and the ssh `command -v ` ×2 at §3's `host_effect_realize`) | native path-search when `OnTarget`; ssh `command -v` only for a remote host | sibling flagged in the transport review (2026-07-24); same root | + +**One root, three lanes** (operator, 2026-07-24): these two, the transport review's `test -x`/`command -v` hand-strings for ssh, and the wall worker's fight are all *modeled operations whose only realization is a shell escape*. The dissolution is the transport-decomposition Realization: **same operation, two handlers, native chosen when locality is `OnTarget`** — `Env.Get`'s native read is the lane's **cheapest first consumer** (a pure in-process read, no `host_effect_apply` even). Not scheduled here; recorded so the deficit is counted and prioritizable (§6), never absorbed into "it's only a few ms." + +--- + ## 1. Residual-shell census (current tree) Five categories. "Genuine emitter" = emits bash that actually runs; "oracle/scaffold" = `serialize_bash` retained only for a test. diff --git a/src/v1/01_tokenize.dag b/src/v1/01_tokenize.dag index c3cc529b80c..a8e4cf18cad 100644 --- a/src/v1/01_tokenize.dag +++ b/src/v1/01_tokenize.dag @@ -350,6 +350,37 @@ fn process_escapes(raw: String) -> String { process_escapes_loop(source: raw, pos: 0, acc: []) } +data hex_escape_note: String = "\\xNN decodes to the byte NN names. Before this arm every \\x fell through to the unknown-escape branch below, which preserves the backslash literally — so every ANSI code authored in .dag (extdeps.render.ansi.csi_esc, extdeps.render.terminal's reset, gunbc.ci_render's red/reset) emitted the six characters backslash-x-1-b instead of ESC, and no colour this repo authored had ever rendered. The witness that was supposed to catch it compared one undecoded literal against another, so it agreed with itself and could not fail (DESIGN §5 — a check satisfied by editing the declaration while the realization lies)." + +data unknown_escape_passthrough_frontier: String = "🟡 dissolve-on: tokenizer_unknown_escape_strict_close (opened 2026-07-25) — the else-arm below resolves an UNRECOGNIZED escape to concat(backslash, next), so \\s silently means backslash-s and is indistinguishable from a typo'd escape. That is a closed-vocabulary violation (DESIGN §4: the vocabulary is closed; §5: a failure arm refuses, never widens) and it is knowingly RETAINED, not accepted: refusing today would red 142 occurrences across 38 files (textual census 2026-07-25 — grep over .dag, so an upper bound, and DESIGN's own caution about grep census applies), overwhelmingly regex fragments (\\' 28, \\. 19, \\| 13, \\/ 12, \\B 10) that intend the backslash to survive. NOT the four sites a \\x-only count suggests. DISSOLVES WHEN a raw-string form lands to carry regex fragments, the 142 occurrences migrate onto it, and this arm becomes a typed located parse refusal (the Rust rule: unknown escape is an error, a literal backslash is spelled \\\\). Because that is a PARSE-SEMANTICS change over a tokenizer every lane's probes compile through, the strict close lands with a corpus-wide compile receipt, never the site migration alone." + +fn hex_digit_value(ch: String) -> Int? { + let cp = code_point(ch) + if cp >= 48 && cp <= 57 { + Present { value: cp - 48 } + } else { + if cp >= 97 && cp <= 102 { + Present { value: cp - 87 } + } else { + if cp >= 65 && cp <= 70 { + Present { value: cp - 55 } + } else { + none + } + } + } +} + +fn hex_escape_char(hi: String, lo: String) -> String? { + match hex_digit_value(ch: hi) { + Present { value: h } => match hex_digit_value(ch: lo) { + Present { value: l } => Present { value: from_code_point(h * 16 + l) } + Absent => none + } + Absent => none + } +} + fn process_escapes_loop(source: String, pos: Int, acc: List) -> String { if pos >= string_length(s: source) { join(acc, separator: "") @@ -357,14 +388,23 @@ fn process_escapes_loop(source: String, pos: Int, acc: List) -> String { let ch = char_at(s: source, pos: pos) if ch == "\\" && pos + 1 < string_length(s: source) { let next = char_at(s: source, pos: pos + 1) - let resolved = if next == "\"" { "\"" } - else if next == "\\" { "\\" } - else if next == "n" { "\n" } - else if next == "t" { "\t" } - else if next == "{" { "{" } - else if next == "}" { "}" } - else { concat("\\", next) } - process_escapes_loop(source: source, pos: pos + 2, acc: list_push(acc, resolved)) + if next == "x" && pos + 3 < string_length(s: source) { + match hex_escape_char(hi: char_at(s: source, pos: pos + 2), lo: char_at(s: source, pos: pos + 3)) { + Present { value: decoded } => + process_escapes_loop(source: source, pos: pos + 4, acc: list_push(acc, decoded)) + Absent => + process_escapes_loop(source: source, pos: pos + 2, acc: list_push(acc, concat("\\", next))) + } + } else { + let resolved = if next == "\"" { "\"" } + else if next == "\\" { "\\" } + else if next == "n" { "\n" } + else if next == "t" { "\t" } + else if next == "{" { "{" } + else if next == "}" { "}" } + else { concat("\\", next) } + process_escapes_loop(source: source, pos: pos + 2, acc: list_push(acc, resolved)) + } } else { process_escapes_loop(source: source, pos: pos + 1, acc: list_push(acc, ch)) } diff --git a/src/v1/05_emit_rust.dag b/src/v1/05_emit_rust.dag index 31590bb0187..880ecc1466f 100644 --- a/src/v1/05_emit_rust.dag +++ b/src/v1/05_emit_rust.dag @@ -10390,6 +10390,10 @@ fn emit_subcommand_enum(workflow_funcs: List, has_pipeline: Bool) make_indent(level: depth + 2), "/// TestClaim / witness run: Bool false → exit 1; requires --entry\n", make_indent(level: depth + 2), "#[arg(long)]\n", make_indent(level: depth + 2), "claim_run: bool,\n", + make_indent(level: depth + 2), "/// Named argument for the entry function, repeatable: `--arg name=value`.\n", + make_indent(level: depth + 2), "/// Values enter as String; a missing `=` refuses rather than guessing.\n", + make_indent(level: depth + 2), "#[arg(long = \"arg\")]\n", + make_indent(level: depth + 2), "args: Vec,\n", make_indent(level: depth + 1), "},"] let run_variant = if has_pipeline { run_variant_lines |> join(separator: "") } else { "" } let all_variants = if has_pipeline { concat(variants, [compile_variant, run_variant]) } else { variants } @@ -10601,8 +10605,8 @@ fn emit_compile_match_arm(crate_name: String) -> String { fn emit_run_match_arm(crate_name: String) -> String { concat( - "\n Commands::Run { source_roots, function, entry, claim_run } => {\n", - " cli_run::handle_run_with_options(source_roots, function, entry, cli.dry_run, claim_run);\n", + "\n Commands::Run { source_roots, function, entry, claim_run, args } => {\n", + " cli_run::handle_run_with_options(source_roots, function, entry, cli.dry_run, claim_run, args);\n", " },") } diff --git a/src/v1/runtime_rust.dag b/src/v1/runtime_rust.dag index 03826074a7d..f5a7c5bf402 100644 --- a/src/v1/runtime_rust.dag +++ b/src/v1/runtime_rust.dag @@ -443,28 +443,78 @@ fn rt_filesystem() -> String { fn rt_hash_ops() -> String { concat( "const FNV1A64_OFFSET: u64 = 0xcbf29ce484222325;\n", - "const FNV1A64_PRIME: u64 = 0x100000001b3;\n\n", - "fn fnv1a64(bytes: &[u8]) -> u64 {\n", + "const FNV1A64_PRIME: u64 = 0x100000001b3;\n", + "\n", + "fn fnv1a64(bytes: &[u8]) -> u64 \{\n", " let mut hash = FNV1A64_OFFSET;\n", - " for b in bytes {\n", + " for b in bytes \{\n", " hash ^= *b as u64;\n", " hash = hash.wrapping_mul(FNV1A64_PRIME);\n", - " }\n", + " \}\n", " hash\n", - "}\n\n", - "pub type Hash = String;\n\n", - "const HASH_DIGEST_LEN: usize = 16;\n\n", - "pub fn is_hash_digest(s: &str) -> bool {\n", + "\}\n", + "\n", + "pub type Hash = String;\n", + "\n", + "const HASH_DIGEST_LEN: usize = 16;\n", + "\n", + "pub fn is_hash_digest(s: &str) -> bool \{\n", " s.len() == HASH_DIGEST_LEN && s.bytes().all(|b| b.is_ascii_hexdigit())\n", - "}\n\n", - "fn expect_hash_digest(s: &str, arg: &str) {\n", - " if !is_hash_digest(s) {\n", - " panic!(\"{} must be a 16-char hex Hash digest\", arg);\n", - " }\n", - "}\n\n", + "\}\n", + "\n", + "fn expect_hash_digest(s: &str, arg: &str) \{\n", + " if !is_hash_digest(s) \{\n", + " panic!(\"\{\} must be a 16-char hex Hash digest\", arg);\n", + " \}\n", + "\}\n", + "\n", + "/// Kept so `gunbc.observation_emit_census` roster hygiene cannot go stale after the\n", + "/// raw gantt key=value shape (process-absolute millis and rss-mib fields) dissolves into the observation projection.\n", + "#[allow(dead_code)]\n", + "pub const GANTT_CENSUS_MARKER: &str = \"[gantt]\";\n", + "\n", + "/// Mirror of `gunbc.observation_ci_render.ci_minute_switch_seconds`.\n", + "const OBS_MINUTE_SWITCH_SECONDS: u64 = 90;\n", + "\n", + "fn obs_human_duration(ms: u64) -> String \{\n", + " if ms < 1_000 \{\n", + " format!(\"\{ms\}ms\")\n", + " \} else if ms < OBS_MINUTE_SWITCH_SECONDS * 1_000 \{\n", + " format!(\"\{\} seconds\", ms / 1_000)\n", + " \} else \{\n", + " format!(\"\{\} minutes\", ms / 60_000)\n", + " \}\n", + "\}\n", + "\n", + "fn obs_phase_emoji() -> bool \{\n", + " std::env::var(\"GITHUB_ACTIONS\").as_deref() == Ok(\"true\")\n", + "\}\n", + "\n", + "/// Pure Rust mirror of `gunbc.observation_seed_render.phase_begin_line` —\n", + "/// `ci_event_line(Begin) ∘ ci_render_line`. Justified divergence from the\n", + "/// interpreter seed boundary used by floor prelude marks: `trace_mark` runs\n", + "/// inside compile itself (hand-synced path and interpreter intrinsic), and\n", + "/// calling back into the interpreter to render would recurse. Proven\n", + "/// byte-equal to the `.dag` oracle by the seed RED.\n", + "pub fn render_phase_begin_line_mirror(phase: &str, emoji: bool) -> String \{\n", + " let glyph = if emoji \{ \"🔄\" \} else \{ \"◐\" \};\n", + " format!(\"\{glyph\} started \{phase\}\")\n", + "\}\n", + "\n", + "/// Pure Rust mirror of `gunbc.observation_seed_render.phase_concluded_line` —\n", + "/// `ci_event_line(Concluded\{Done\}) ∘ ci_render_line`. Same justified-divergence\n", + "/// note as `render_phase_begin_line_mirror`.\n", + "pub fn render_phase_concluded_line_mirror(phase: &str, elapsed_ms: u64, emoji: bool) -> String \{\n", + " let glyph = if emoji \{ \"✅\" \} else \{ \"✓\" \};\n", + " format!(\"\{glyph\} \{phase\} done in \{\}\", obs_human_duration(elapsed_ms))\n", + "\}\n", + "\n", "/// Stage-boundary trace mark — the v1-seed interim realization of the v2 per-RealizedStep\n", - "/// CostAccount (std.realization_measurement). One stderr line per mark; consecutive marks\n", - "/// define the segments of a natural Gantt read directly off any run log.\n", + "/// CostAccount (std.realization_measurement). Wired (gantt flip): projects Begin/Concluded\n", + "/// on a PhaseSegment via the observation mirrors of `ci_event_line`. Segment wall on\n", + "/// Concluded (matched `.begin`→`.done`, else delta since last mark) — never the old\n", + "/// process-absolute millis field. RSS stays on heartbeat/measurement (`ci_event_line` does\n", + "/// not project `event.rss` — explicit divergence from the deleted rss-mib field).\n", "///\n", "/// **Dissolution trigger (DESIGN §6):** delete this fn, the `trace_mark` registry row in\n", "/// `04_method.dag` (+ hand-synced twin `v1_compiler_infer_method.rs`), the nine\n", @@ -475,48 +525,80 @@ fn rt_hash_ops() -> String { "/// as `phase_profile.rs` / `GUNBC_FLOOR_GANTT`, per `docs/plans/ci-floor-fractal-gantt.md`\n", "/// § dissolution). Receipt = that witness green with these marks deleted and stage walls\n", "/// still attributable from the model path.\n", - "pub fn trace_mark(label: String) {\n", - " use std::sync::OnceLock;\n", + "pub fn trace_mark(label: String) \{\n", + " use std::sync::\{Mutex, OnceLock\};\n", " use std::time::Instant;\n", " static TRACE_T0: OnceLock = OnceLock::new();\n", - " let ms = TRACE_T0.get_or_init(Instant::now).elapsed().as_millis();\n", - " let mut rss_mib = String::from(\"absent\");\n", - " if let Ok(status) = std::fs::read_to_string(\"/proc/self/status\") {\n", - " for line in status.lines() {\n", - " if let Some(rest) = line.strip_prefix(\"VmRSS:\") {\n", - " if let Some(kib) = rest\n", - " .split_whitespace()\n", - " .next()\n", - " .and_then(|k| k.parse::().ok())\n", - " {\n", - " rss_mib = (kib / 1024).to_string();\n", - " }\n", - " }\n", - " }\n", - " }\n", - " eprintln!(\"[gantt] {} t_ms={} rss_mib={}\", label, ms, rss_mib);\n", - "}\n\n", + " static LAST_MARK: OnceLock> = OnceLock::new();\n", + " static OPENS: OnceLock>> = OnceLock::new();\n", + " let t0 = TRACE_T0.get_or_init(Instant::now);\n", + " let last = LAST_MARK.get_or_init(|| Mutex::new(*t0));\n", + " let opens = OPENS.get_or_init(|| Mutex::new(std::collections::HashMap::new()));\n", + " let emoji = obs_phase_emoji();\n", + " let now = Instant::now();\n", + " let _ = GANTT_CENSUS_MARKER;\n", + " if let Some(phase) = label.strip_suffix(\".begin\") \{\n", + " if let Ok(mut map) = opens.lock() \{\n", + " map.insert(phase.to_string(), now);\n", + " \}\n", + " if let Ok(mut l) = last.lock() \{\n", + " *l = now;\n", + " \}\n", + " eprintln!(\"\{\}\", render_phase_begin_line_mirror(phase, emoji));\n", + " \} else if let Some(phase) = label.strip_suffix(\".done\") \{\n", + " let elapsed_ms = \{\n", + " let started = opens.lock().ok().and_then(|mut m| m.remove(phase));\n", + " match started \{\n", + " Some(t) => now.saturating_duration_since(t).as_millis() as u64,\n", + " None => \{\n", + " let prev = last.lock().map(|l| *l).unwrap_or(*t0);\n", + " now.saturating_duration_since(prev).as_millis() as u64\n", + " \}\n", + " \}\n", + " \};\n", + " if let Ok(mut l) = last.lock() \{\n", + " *l = now;\n", + " \}\n", + " eprintln!(\n", + " \"\{\}\",\n", + " render_phase_concluded_line_mirror(phase, elapsed_ms, emoji)\n", + " );\n", + " \} else \{\n", + " let prev = last.lock().map(|l| *l).unwrap_or(*t0);\n", + " let elapsed_ms = now.saturating_duration_since(prev).as_millis() as u64;\n", + " if let Ok(mut l) = last.lock() \{\n", + " *l = now;\n", + " \}\n", + " eprintln!(\n", + " \"\{\}\",\n", + " render_phase_concluded_line_mirror(&label, elapsed_ms, emoji)\n", + " );\n", + " \}\n", + "\}\n", + "\n", "/// Content hash over raw bytes — the byte-level single authority. `atom_identity_hash`\n", "/// is the `String` projection of this. Use this directly for arbitrary binary content\n", "/// (e.g. an executable or serialized payload): routing bytes through `String`/\n", "/// `from_utf8_lossy` first collapses every invalid UTF-8 sequence to U+FFFD, so distinct\n", "/// byte sequences would hash equal — a §5 silent-collision fail-open for content-addressing.\n", - "pub fn bytes_identity_hash(bytes: &[u8]) -> Hash {\n", - " format!(\"{:016x}\", fnv1a64(bytes))\n", - "}\n\n", - "pub fn atom_identity_hash(s: String) -> Hash {\n", + "pub fn bytes_identity_hash(bytes: &[u8]) -> Hash \{\n", + " format!(\"\{:016x\}\", fnv1a64(bytes))\n", + "\}\n", + "\n", + "pub fn atom_identity_hash(s: String) -> Hash \{\n", " bytes_identity_hash(s.as_bytes())\n", - "}\n\n", - "pub fn hash_combine(a: Hash, b: Hash) -> Hash {\n", + "\}\n", + "\n", + "pub fn hash_combine(a: Hash, b: Hash) -> Hash \{\n", " expect_hash_digest(&a, \"a\");\n", " expect_hash_digest(&b, \"b\");\n", " let mut bytes = a.into_bytes();\n", " bytes.push(0);\n", " bytes.extend_from_slice(b.as_bytes());\n", - " format!(\"{:016x}\", fnv1a64(&bytes))\n", - "}\n\n") + " format!(\"\{:016x\}\", fnv1a64(&bytes))\n", + "\}\n", + "\n") } - fn rt_resolution_silent_pick_telemetry() -> String { concat( "/// Read-only silent-pick telemetry for resolution divergence census slice 2.\n", diff --git a/src/v1/stage0/src/bin/claim_batch.rs b/src/v1/stage0/src/bin/claim_batch.rs index 441fb49477e..08836d4bfb5 100644 --- a/src/v1/stage0/src/bin/claim_batch.rs +++ b/src/v1/stage0/src/bin/claim_batch.rs @@ -77,10 +77,11 @@ fn children_max_rss_bytes() -> Option { } fn emit_rss_measurement(label: &str) { - match peak_rss_bytes() { - Some(bytes) => eprintln!("[measurement] {label}: {bytes} bytes (VmHWM)"), - None => eprintln!("[measurement] {label}: unavailable (no /proc/self/status)"), - } + let emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); + eprintln!( + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror(label, peak_rss_bytes(), emoji) + ); } fn print_interp_stats(ctx: &InterpContext, flatten_baseline: (u64, u64)) { @@ -578,7 +579,15 @@ fn run() -> Result { Ok(keys) => { emit_rss_measurement("post-mock-precompute-rss"); if let Some(bytes) = children_max_rss_bytes() { - eprintln!("[measurement] post-mock-precompute-children-max-rss: {bytes} bytes (getrusage RUSAGE_CHILDREN)"); + let emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); + eprintln!( + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror( + "post-mock-precompute-children-max-rss", + Some(bytes), + emoji, + ) + ); } if keys.is_empty() { eprintln!( @@ -720,7 +729,15 @@ fn run() -> Result { emit_rss_measurement("per-shard-peak-rss"); if let Some(bytes) = children_max_rss_bytes() { - eprintln!("[measurement] children-max-rss: {bytes} bytes (getrusage RUSAGE_CHILDREN)"); + let emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); + eprintln!( + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror( + "children-max-rss", + Some(bytes), + emoji, + ) + ); } if any_failed { diff --git a/src/v1/stage0/src/bin/claim_executor.rs b/src/v1/stage0/src/bin/claim_executor.rs index 533c15d958a..db8bb0d678c 100644 --- a/src/v1/stage0/src/bin/claim_executor.rs +++ b/src/v1/stage0/src/bin/claim_executor.rs @@ -12,6 +12,7 @@ use std::time::Instant; use v1_compiler::cli_run::workspace_root; use v1_compiler::cli_run::{ compute_histogram_data, compute_witness_timing_rows, enable_floor_compile_clean_lazy_install, + heartbeat_feed_enter_batch, heartbeat_feed_entry_completed, heartbeat_feed_snapshot, install_floor_compile_clean_receipt, make_eval_context, resolve_entry_graph, resolve_entry_graph_shared, run_claim, run_discovery_corpus_with_options, run_value, set_phase, top_n_slowest_witnesses, ClaimOutcome, DiscoveryCorpusOptions, DiscoverySummary, @@ -20,8 +21,8 @@ use v1_compiler::cli_run::{ }; use v1_compiler::memory_governor::{ binding_cap_cgroup_dir, binding_high_cgroup_dir, floor_budget_below_minimum_footprint, - leaf_cgroup_dir, mem_total_bytes, memory_events_field, memory_pressure_some_avg10, - read_cgroup_raw, read_cgroup_u64, AdmittedSlot, MemoryGovernor, + leaf_cgroup_dir, mem_total_bytes, memory_pressure_some_avg10, read_cgroup_raw, read_cgroup_u64, + AdmittedSlot, MemoryGovernor, }; use v1_compiler::v1_interpreter::{ color_enabled, paint, run_in_context_with_args, sgr, ExecutionMode, InterpContext, Value, @@ -62,66 +63,104 @@ fn read_positive_budget_ms( } } -/// THE COST WALL (CI floor endgame D5 — authority `gunbc.ci_spec.gunbc_ci_floor_batch_wall_budget_seconds` -/// + `gunbc_ci_floor_batch_wall_budget_note`): per-batch wall budgets for the floor plan, read -/// fail-closed at arm time from the plan ctx (the fast-lane-budget pattern). Budgets are -/// admission/scheduling facts at the walk grain — witness verdicts never carry a wall-clock -/// term (the ruling split reconciled in the carrier note). `GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS` -/// is the RED-control fault injection: it can only LOWER budgets (min), so it can force the -/// refusal for a control run but can never open the gate — not an escape hatch by construction. -fn read_floor_batch_wall_budgets_ms( +/// THE COST WALL (Piece 3 derived clamp — authority `gunbc.ci_spec.gunbc_ci_floor_batch_clamp_params` +/// + `gunbc_ci_floor_batch_clamp_note`): the per-batch clamp is `overhead_seconds*1000 + +/// runtime_unit_count * per_unit_ms`. This reads the two index-aligned param lists fail-closed at +/// arm time (the fast-lane-budget pattern); the clamp itself is computed at enforcement, because the +/// affected-set-selected unit count is a runtime datum the schedule does not hold. Clamps are +/// admission/scheduling facts at the walk grain — witness verdicts never carry a wall-clock term +/// (the ruling split reconciled in the carrier note). +fn read_floor_batch_clamp_params( plan_ctx: &InterpContext, batch_count: usize, -) -> Result, String> { - let items = match run_value(plan_ctx, "gunbc_ci_floor_batch_wall_budgets_seconds") { +) -> Result, String> { + let overhead_items = match run_value(plan_ctx, "gunbc_ci_floor_batch_clamp_overhead_seconds") { Ok(Value::List(items)) => items, Ok(other) => { return Err(format!( - "claim_executor: gunbc_ci_floor_batch_wall_budgets_seconds must be a List, got {other:?} (fail-closed)" + "claim_executor: gunbc_ci_floor_batch_clamp_overhead_seconds must be a List, got {other:?} (fail-closed)" )); } Err(msg) => { return Err(format!( - "claim_executor: floor plan schedules batches but gunbc_ci_floor_batch_wall_budgets_seconds is unavailable (fail-closed): {msg}" + "claim_executor: floor plan schedules batches but gunbc_ci_floor_batch_clamp_overhead_seconds is unavailable (fail-closed): {msg}" )); } }; - let mut budgets_ms: Vec = Vec::new(); - for item in items.iter() { + let mut overheads_ms: Vec = Vec::new(); + for item in overhead_items.iter() { match item { - Value::Int(n) if *n > 0 => budgets_ms.push(*n as u128 * 1000), + Value::Int(n) if *n > 0 => overheads_ms.push(*n as u128 * 1000), other => { return Err(format!( - "claim_executor: gunbc_ci_floor_batch_wall_budgets_seconds rows must be positive Ints, got {other:?} (fail-closed)" + "claim_executor: gunbc_ci_floor_batch_clamp_overhead_seconds rows must be positive Ints, got {other:?} (fail-closed)" )); } } } - if budgets_ms.len() != batch_count { - return Err(format!( - "claim_executor: gunbc_ci_floor_batch_wall_budgets_seconds has {} row(s) but the plan schedules {} batch(es) — the budget list must cover the schedule exactly (fail-closed; update gunbc.ci_spec beside the schedule change)", - budgets_ms.len(), - batch_count - )); - } - if let Ok(tighten) = std::env::var("GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS") { - match tighten.parse::() { - Ok(t) => { - for b in budgets_ms.iter_mut() { - *b = (*b).min(t); - } - eprintln!( - "claim_executor: GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS={t} — budgets tightened (RED-control injection; tighten-only, can never widen a budget)" - ); - } - Err(_) => { + let rate_items = match run_value(plan_ctx, "gunbc_ci_floor_batch_clamp_per_unit_ms") { + Ok(Value::List(items)) => items, + Ok(other) => { + return Err(format!( + "claim_executor: gunbc_ci_floor_batch_clamp_per_unit_ms must be a List, got {other:?} (fail-closed)" + )); + } + Err(msg) => { + return Err(format!( + "claim_executor: floor plan schedules batches but gunbc_ci_floor_batch_clamp_per_unit_ms is unavailable (fail-closed): {msg}" + )); + } + }; + let mut rates_ms: Vec = Vec::new(); + for item in rate_items.iter() { + match item { + Value::Int(n) if *n >= 0 => rates_ms.push(*n as u128), + other => { return Err(format!( - "claim_executor: GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS must parse as milliseconds, got {tighten:?} (fail-closed)" + "claim_executor: gunbc_ci_floor_batch_clamp_per_unit_ms rows must be non-negative Ints, got {other:?} (fail-closed)" )); } } } - Ok(budgets_ms) + if overheads_ms.len() != batch_count || rates_ms.len() != batch_count { + return Err(format!( + "claim_executor: floor batch clamp params (overhead {} row(s), rate {} row(s)) must each cover the {} scheduled batch(es) exactly (fail-closed; update gunbc.ci_spec beside the schedule change)", + overheads_ms.len(), + rates_ms.len(), + batch_count + )); + } + Ok(overheads_ms.into_iter().zip(rates_ms).collect()) +} + +/// The RED-control fault injection (`GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS`): lowers the COMPUTED +/// per-batch clamp (min) at enforcement, so it can force a FLOOR-BATCH-OVER-BUDGET refusal for a +/// control run but can never open the gate — tighten-only by construction, never an escape hatch. +fn read_floor_batch_budget_tighten_ms() -> Result, String> { + match std::env::var("GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS") { + Ok(t) => match t.parse::() { + Ok(v) => Ok(Some(v)), + Err(_) => Err(format!( + "claim_executor: GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS must parse as milliseconds, got {t:?} (fail-closed)" + )), + }, + Err(_) => Ok(None), + } +} + +/// Runtime per-batch unit count for the derived clamp: a discovery aggregate result contributes +/// its post-selection witness count (`corpus_witnesses`); every single-claim gate row contributes 1. +fn batch_runtime_unit_count(results: &[ClaimResult]) -> u128 { + results + .iter() + .map(|r| { + if r.corpus_witnesses > 0 { + r.corpus_witnesses as u128 + } else { + 1u128 + } + }) + .sum() } /// SCAFFOLD (§7 seed-retained HAND-RUST — authority: @@ -1395,6 +1434,59 @@ struct WalkOutcome { failure_details: Vec, } +/// Render a floor phase-completion line through the single-authority observation +/// renderer (`gunbc.observation_ci_render`, via the seed boundary +/// `gunbc.observation_seed_render`) instead of a raw `[t+…]` byte string in the seed +/// — the format lives in `.dag`, this only transports primitives across the boundary, +/// exactly as `cli_run::install_output_policy` calls `output_policy.resolve_channel_policy`. +/// The seed occurrence "a floor phase concluded in `elapsed_ms`" is modelled as a +/// `Concluded` event on a `PhaseSegment` subject and projected by +/// `ci_event_line ∘ ci_render_line`. Resolve is memoized in the process resolve store, +/// so the render module is resolved once and every later mark is a cache hit + a cheap +/// eval. Returns `None` only if the renderer cannot be resolved/evaluated; the caller +/// degrades loudly and never reproduces the old marker (§5: a failure arm refuses, it +/// does not widen). The entry is located under whichever source root holds it as an +/// absolute path, so resolution does not depend on the process CWD (production runs +/// from the repo root; `cargo test` runs from the crate dir). +fn render_phase_concluded_line( + source_roots: &[String], + phase: &str, + elapsed_ms: u64, + overhead_ms: u64, + emoji: bool, +) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "phase_concluded_line", + &[ + (Some("phase".to_string()), Value::Str(phase.to_string())), + ( + Some("elapsed_ms".to_string()), + Value::Int(elapsed_ms as i64), + ), + ( + Some("overhead_ms".to_string()), + Value::Int(overhead_ms as i64), + ), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } +} + fn peak_rss_bytes() -> Option { let status = std::fs::read_to_string("/proc/self/status").ok()?; let line = status.lines().find(|l| l.starts_with("VmHWM"))?; @@ -1402,9 +1494,101 @@ fn peak_rss_bytes() -> Option { Some(kb.saturating_mul(1024)) } -fn heartbeat_field(v: Option) -> String { - v.map(|b| b.to_string()) - .unwrap_or_else(|| "unreadable".into()) +/// Mirror of `gunbc.observation_ci_render.ci_minute_switch_seconds` — where the +/// sentence form switches from seconds to minutes (display policy, not a unit). +const CI_MINUTE_SWITCH_SECONDS: u64 = 90; + +/// Mirror of `gunbc.observation_seed_render.seed_heartbeat_unreadable_cause`. +const SEED_HEARTBEAT_UNREADABLE_CAUSE: &str = "cgroup field unreadable"; + +/// Pure Rust mirror of `gunbc.observation_seed_render.seed_heartbeat_line` — +/// `ci_heartbeat_line ∘ ci_render_line` over the seed's real input space. The +/// heartbeat thread cannot call the interpreter (would build a duplicate module +/// index under the memory envelope it watches — DESIGN §2); this mirror is proven +/// byte-equal to the `.dag` oracle by `render_heartbeat_line_mirror_matches_seed_oracle`. +/// Subject is batch-grain only (parallel entries → no fabricated per-module detail). +fn render_heartbeat_line_mirror( + elapsed_ms: u64, + batch_label: &str, + entry_index: u64, + entry_total: u64, + rss_bytes: Option, + swap_bytes: Option, + pressure_bp: Option, + emoji: bool, +) -> String { + let glyph = if emoji { "🕐" } else { "◷" }; + let duration = mirror_ci_human_duration(elapsed_ms); + let rss = mirror_ci_measured_bytes(rss_bytes); + let swap = mirror_ci_measured_bytes(swap_bytes); + let pressure = mirror_ci_measured_percent(pressure_bp); + format!( + "{glyph} {duration} in — still in {batch_label}: entry {entry_index} of {entry_total}. memory {rss}, swap {swap}, pressure {pressure}" + ) +} + +fn mirror_ci_human_duration(ms: u64) -> String { + if ms < 1_000 { + format!("{ms}ms") + } else if ms < CI_MINUTE_SWITCH_SECONDS * 1_000 { + format!("{} seconds", ms / 1_000) + } else { + format!("{} minutes", ms / 60_000) + } +} + +fn mirror_ci_tenths_text(tenths: u64) -> String { + format!("{}.{}", tenths / 10, tenths % 10) +} + +fn mirror_ci_human_bytes(bytes: u64) -> String { + // Mirror of ci_gibibyte_tenths: (bytes * 10) / gibibyte_scale_factor_bytes (2^30). + let tenths = (bytes.saturating_mul(10)) / 1_073_741_824; + format!("{} GiB", mirror_ci_tenths_text(tenths)) +} + +fn mirror_ci_human_percent(bp: u64) -> String { + // Mirror of ci_human_percent: tenths = bp / 10 → "9.0%". + format!("{}%", mirror_ci_tenths_text(bp / 10)) +} + +fn mirror_ci_measured_bytes(v: Option) -> String { + match v { + Some(b) => mirror_ci_human_bytes(b), + None => format!("unreadable ({SEED_HEARTBEAT_UNREADABLE_CAUSE})"), + } +} + +fn mirror_ci_measured_percent(v: Option) -> String { + match v { + Some(bp) => mirror_ci_human_percent(bp), + None => format!("unreadable ({SEED_HEARTBEAT_UNREADABLE_CAUSE})"), + } +} + +/// PSI `avg10` is a percent with one decimal (e.g. `"9.01"`). One basis point is +/// 0.01 percentage points, so percent × 100 = bp (9.01 → 901) — the same scale +/// `std.observation` carries for `PsiPressure.avg10`. +fn psi_avg10_to_basis_points(avg10: &str) -> Option { + let pct: f64 = avg10.parse().ok()?; + if !pct.is_finite() || pct < 0.0 { + return None; + } + Some((pct * 100.0).round() as u64) +} + +fn batch_heartbeat_label(batch: &[Runnable]) -> String { + if batch + .iter() + .any(|r| matches!(r, Runnable::DiscoveryBatch { .. })) + { + // Canonical crawl-window subject — matches the seed oracle's batch label. + "witness discovery".to_string() + } else if let Some(Runnable::SingleClaim { function, .. }) = batch.first() { + function.clone() + } else { + "batch".to_string() + } } /// Floor memory heartbeat — FIDELITY ONLY (reads state, changes no behavior; §5 stopped-line @@ -1412,13 +1596,13 @@ fn heartbeat_field(v: Option) -> String { /// invisible precisely here: `memory.high` throttles instead of killing, so the only log /// evidence was a post-hoc `memory.peak` pinned at `high + <1MiB` after a 30-49min silent /// tail. One synchronous regime-disclosure line at floor start (which limits bind, where), -/// then one line per minute from a detached thread (dies with the process): `memory.current`, -/// `memory.swap.current`, `memory.events` high-throttle count, PSI `some avg10`. The wedge -/// signature becomes: current pinned at high, swap climbing, high-events exploding, PSI avg10 -/// double digits — attributable to a 60s window instead of a forensic reconstruction. -/// Sampling denominator = the binding-high dir when set (the slot slice that throttles), else -/// the binding-cap dir, else the leaf (whole-machine regimes); absence of all three refuses -/// loudly and the floor proceeds unmonitored — never a fabricated zero. +/// then one line per minute from a detached thread (dies with the process), projected +/// through `render_heartbeat_line_mirror` — identity-first, human units, subject from the +/// HeartbeatFeed. The raw floor-memory byte dump (minute counter + raw current/swap +/// integers) is deleted (census negative example). Sampling denominator = the binding-high +/// dir when set (the slot slice that throttles), else the binding-cap dir, else the leaf +/// (whole-machine regimes); absence of all three refuses loudly and the floor proceeds +/// unmonitored — never a fabricated zero. fn spawn_floor_memory_heartbeat() { let high_dir = binding_high_cgroup_dir(); let cap_dir = binding_cap_cgroup_dir(); @@ -1442,24 +1626,36 @@ fn spawn_floor_memory_heartbeat() { ); return; }; + let emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); let spawned = std::thread::Builder::new() .name("floor-memory-heartbeat".into()) .spawn(move || { - let mut minute: u64 = 0; + let started = Instant::now(); loop { std::thread::sleep(std::time::Duration::from_secs(60)); - minute += 1; - let psi = read_cgroup_raw(&dir, "memory.pressure") + // Skip until a batch has armed the feed with a real entry_total — + // never a fabricated 0-of-0 during prelude / roster assembly. + let Some(feed) = heartbeat_feed_snapshot() else { + continue; + }; + let rss = read_cgroup_u64(&dir, "memory.current"); + let swap = read_cgroup_u64(&dir, "memory.swap.current"); + let pressure = read_cgroup_raw(&dir, "memory.pressure") .and_then(|c| memory_pressure_some_avg10(&c)) - .unwrap_or_else(|| "unreadable".into()); - let high_events = read_cgroup_raw(&dir, "memory.events") - .and_then(|c| memory_events_field(&c, "high")); - eprintln!( - "[floor-memory] t={minute}m current={} swap={} high_events={} psi_some_avg10={psi}", - heartbeat_field(read_cgroup_u64(&dir, "memory.current")), - heartbeat_field(read_cgroup_u64(&dir, "memory.swap.current")), - heartbeat_field(high_events), + .as_deref() + .and_then(psi_avg10_to_basis_points); + let elapsed_ms = started.elapsed().as_millis() as u64; + let line = render_heartbeat_line_mirror( + elapsed_ms, + &feed.batch_label, + feed.entry_done, + feed.entry_total, + rss, + swap, + pressure, + emoji, ); + eprintln!("{line}"); } }); if let Err(e) = spawned { @@ -1595,8 +1791,17 @@ fn verify_build_artifacts(paths: &[String]) -> Result { /// standalone `--measure-cgroup-peak` mode so the `ci` and `rust_tests` jobs report an /// identically-shaped line. `context` distinguishes the call site. fn emit_cgroup_measurement(context: &str) { + let emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); match cgroup_job_measurement() { Some(m) => { + let label = format!("cgroup peak @ {} ({context})", m.leaf_rel); + eprintln!( + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror(&label, Some(m.leaf_peak), emoji) + ); + // Diagnostic companions (cap/pids/sccache) stay as Ambient detail beside the + // Measured peak — not the old raw-byte `[measurement]` dump. Placement still + // reads the typed `cgroup_job_measurement` fact, not this prose. let cap = match m.cap_bytes { Some(b) => format!("{b} bytes"), None => "uncapped(RAM-bound)".to_string(), @@ -1611,16 +1816,23 @@ fn emit_cgroup_measurement(context: &str) { (None, _) => "not-found (treat as fixed host overhead)".to_string(), }; eprintln!( - "[measurement] cgroup peak: {peak} bytes (memory.peak @ {rel}) memory.max={cap} host_ram={host_ram} pids.current={pc} pids.max={pm} sccache-server-cgroup={sccache} context={context}", - peak = m.leaf_peak, - rel = m.leaf_rel, + " memory.max={cap} host_ram={host_ram} pids_current={pc} pids_max={pm} sccache-server-cgroup={sccache}", pc = m.pids_current, pm = m.pids_max ); } - None => eprintln!( - "[measurement] cgroup peak: unavailable (no leaf cgroup or memory.peak unreadable; kernel < 5.19?) context={context}" - ), + None => { + let label = format!("cgroup peak ({context})"); + eprintln!( + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror_with_cause( + &label, + None, + "no leaf cgroup or memory.peak unreadable; kernel < 5.19?", + emoji, + ) + ); + } } } @@ -1660,6 +1872,11 @@ fn sccache_server_cgroup_rel() -> Option { struct BatchRecord { batch_index: usize, wall_nanos: u128, + /// The derived clamp computed for this batch (overhead + unit_count*rate, tightened), or None + /// for budget-less plans (falsifier/regen). Recorded so the receipt never recomputes it. + clamp_ms: Option, + /// The runtime unit count the clamp used (discovery witnesses + gate rows). + unit_count: u128, /// Flattened results from all units in this batch (order: unit by unit). results: Vec, } @@ -1676,42 +1893,32 @@ fn write_resolve_receipt(batch_records: &[BatchRecord]) -> bool { write_resolve_receipt_at(std::path::Path::new("target"), batch_records) } -/// Per-batch wall receipt (THE COST WALL, D5): typed rows — one wall/budget/verdict triple -/// per batch — so the floor's time story is a receipt, not a log archaeology exercise. -/// `OverBudget` rows correspond one-to-one with the FLOOR-BATCH-OVER-BUDGET refusals the -/// walk printed; a budget-less run (falsifier/regen plans) records walls with +/// Per-batch wall receipt (THE COST WALL, Piece 3 derived clamp): typed rows — one +/// wall/units/clamp/verdict group per batch — so the floor's time story is a receipt, not a log +/// archaeology exercise. `OverBudget` rows correspond one-to-one with the FLOOR-BATCH-OVER-BUDGET +/// refusals the walk printed; a clamp-less run (falsifier/regen plans) records walls with /// `verdict=Unbudgeted`. Returns false on a write error — the walk fails closed here. -fn write_batch_wall_receipt( - batch_records: &[BatchRecord], - batch_wall_budgets_ms: Option<&[u128]>, -) -> bool { - write_batch_wall_receipt_at( - std::path::Path::new("target"), - batch_records, - batch_wall_budgets_ms, - ) +fn write_batch_wall_receipt(batch_records: &[BatchRecord]) -> bool { + write_batch_wall_receipt_at(std::path::Path::new("target"), batch_records) } -fn write_batch_wall_receipt_at( - base: &std::path::Path, - batch_records: &[BatchRecord], - batch_wall_budgets_ms: Option<&[u128]>, -) -> bool { +fn write_batch_wall_receipt_at(base: &std::path::Path, batch_records: &[BatchRecord]) -> bool { let mut body = String::new(); let mut over_budget = 0usize; for rec in batch_records { let n = rec.batch_index + 1; let wall_ms = rec.wall_nanos / 1_000_000; body.push_str(&format!("batch_{n}_wall_ms={wall_ms}\n")); - match batch_wall_budgets_ms.and_then(|b| b.get(rec.batch_index)) { - Some(budget_ms) => { - let verdict = if wall_ms > *budget_ms { + match rec.clamp_ms { + Some(clamp_ms) => { + let verdict = if wall_ms > clamp_ms { over_budget += 1; "OverBudget" } else { "WithinBudget" }; - body.push_str(&format!("batch_{n}_budget_ms={budget_ms}\n")); + body.push_str(&format!("batch_{n}_units={}\n", rec.unit_count)); + body.push_str(&format!("batch_{n}_clamp_ms={clamp_ms}\n")); body.push_str(&format!("batch_{n}_verdict={verdict}\n")); } None => { @@ -1737,6 +1944,71 @@ fn write_batch_wall_receipt_at( true } +fn write_gate_warm_cost_receipt(batch_records: &[BatchRecord]) -> bool { + write_gate_warm_cost_receipt_at(std::path::Path::new("target"), batch_records) +} + +/// Per-gate warm-cost TSV (D2 placement probe, ci-two-tier-placement-redesign §9.1): one row per +/// gate/claim carrying its warm eval wall, resolve time, and combined warm cost — the PLACEMENT +/// ROSTER's measurement basis. A gate rides PrTier only if its measured warm cost is within the +/// 5s fast-lane budget (else fail-closed to Gauntlet — v2.workflow.ci_placement). Denominated PER +/// GATE (placement is per-gate), reusing the ClaimResult timings the floor already records +/// (operator ruling 2026-07-24: instrument the existing floor, do not add a throwaway probe +/// workflow). For a single-claim gate `wall_nanos` IS the eval wall (== thread-CPU on its one +/// thread); the discovery row carries the per-witness rate (serial-sum eval over the witness +/// count — the parallel batch wall is the batch-wall receipt's, not a per-witness figure). The +/// probe reads a WARM run's rows; run cold-then-warm on >=2 hosts and the roster records value + +/// host basis. Fail-closed on a write error (shares target/ with the gated receipts, so a write +/// failure here is the same disk fault that fails them); never a verdict term. +fn write_gate_warm_cost_receipt_at(base: &std::path::Path, batch_records: &[BatchRecord]) -> bool { + let mut body = + String::from("gate\tbatch\teval_ms\tresolve_ms\twarm_ms\twitnesses\ts_per_witness_us\n"); + for rec in batch_records { + let n = rec.batch_index + 1; + for result in &rec.results { + if result.corpus_witnesses > 0 { + let eval_ms = result.corpus_eval_nanos / 1_000_000; + let resolve_ms = result.corpus_resolve_nanos / 1_000_000; + let warm_ms = (result.corpus_eval_nanos + result.corpus_resolve_nanos) / 1_000_000; + let per_witness_us = + result.corpus_eval_nanos / (result.corpus_witnesses as u128) / 1_000; + body.push_str(&format!( + "discovery\t{n}\t{eval_ms}\t{resolve_ms}\t{warm_ms}\t{}\t{per_witness_us}\n", + result.corpus_witnesses + )); + } else { + let eval_ms = result.wall_nanos / 1_000_000; + let resolve_ms = result.resolve_nanos / 1_000_000; + let warm_ms = (result.wall_nanos + result.resolve_nanos) / 1_000_000; + body.push_str(&format!( + "{}\t{n}\t{eval_ms}\t{resolve_ms}\t{warm_ms}\t0\t0\n", + result.function + )); + } + } + } + // Mirror the TSV into the log (prefixed, grep-collectable) so the placement probe can lift + // it from get_job_logs on a fleet run without an artifact-upload step — the file stays for + // future .dag consumers (Piece 1 roster fill). + for line in body.lines() { + eprintln!("[gate-warm-cost] {line}"); + } + let path = base.join("floor-gate-warm-cost-receipt.tsv"); + if let Err(e) = std::fs::create_dir_all(base).and_then(|_| std::fs::write(&path, &body)) { + eprintln!( + "claim_executor: failed to write gate warm-cost receipt {}: {e} — walk fails closed here", + path.display() + ); + return false; + } + eprintln!( + "[receipt] floor gate warm-cost: {} batch(es) (TSV receipt: {})", + batch_records.len(), + path.display() + ); + true +} + fn write_resolve_receipt_at(base: &std::path::Path, batch_records: &[BatchRecord]) -> bool { let mut resolves_total: u64 = 0; let mut resolve_ms_total: u128 = 0; @@ -1824,7 +2096,9 @@ fn write_materialization_receipt() -> bool { true } -/// Emit a fractal Gantt tree to stderr when GUNBC_FLOOR_GANTT=1. +/// Emit a fractal post-walk tree to stderr when GUNBC_FLOOR_GANTT=1. +/// Wired (gantt flip): each row is a PhaseSegment Concluded projection via the +/// observation mirror — the raw `[gantt] … wall: {}ms` key=value tree is gone. fn emit_gantt(batch_records: &[BatchRecord], total_wall_nanos: u128) { let gantt_enabled = std::env::var("GUNBC_FLOOR_GANTT") .map(|v| v == "1") @@ -1832,64 +2106,72 @@ fn emit_gantt(batch_records: &[BatchRecord], total_wall_nanos: u128) { if !gantt_enabled { return; } - let total_ms = total_wall_nanos / 1_000_000; - eprintln!("[gantt] claim_executor wall: {}ms", total_ms); + let emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); + let total_ms = (total_wall_nanos / 1_000_000) as u64; + eprintln!( + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror("claim_executor", total_ms, emoji) + ); for rec in batch_records { - let batch_ms = rec.wall_nanos / 1_000_000; - let pct = if total_ms == 0 { - 0.0 - } else { - 100.0 * batch_ms as f64 / total_ms as f64 - }; + let batch_ms = (rec.wall_nanos / 1_000_000) as u64; + let batch_label = format!("batch {}", rec.batch_index + 1); eprintln!( - "[gantt] batch {} wall: {}ms ({:.1}%)", - rec.batch_index + 1, - batch_ms, - pct, + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror(&batch_label, batch_ms, emoji) ); for result in &rec.results { - let batch_pct = |ns: u128| -> f64 { - if rec.wall_nanos == 0 { - 0.0 - } else { - 100.0 * ns as f64 / rec.wall_nanos as f64 - } - }; if result.corpus_witnesses > 0 { - // Discovery batch: show serial-sum breakdown. - let corpus_resolve_ms = result.corpus_resolve_nanos / 1_000_000; - let corpus_eval_ms = result.corpus_eval_nanos / 1_000_000; - eprintln!( - "[gantt] {} ({} witnesses)", + let corpus_resolve_ms = (result.corpus_resolve_nanos / 1_000_000) as u64; + let corpus_eval_ms = (result.corpus_eval_nanos / 1_000_000) as u64; + let name = format!( + "{} ({} witnesses)", result.function, result.corpus_witnesses ); eprintln!( - "[gantt] resolve (serial sum): {}ms ({:.1}% of batch wall)", - corpus_resolve_ms, - batch_pct(result.corpus_resolve_nanos), + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror( + &name, + corpus_resolve_ms + corpus_eval_ms, + emoji + ) ); eprintln!( - "[gantt] eval (serial sum): {}ms ({:.1}% of batch wall)", - corpus_eval_ms, - batch_pct(result.corpus_eval_nanos), + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror( + &format!("{}.resolve", result.function), + corpus_resolve_ms, + emoji + ) + ); + eprintln!( + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror( + &format!("{}.eval", result.function), + corpus_eval_ms, + emoji + ) ); } else { - // Single claim: show resolve (if charged) + eval. if result.resolve_nanos > 0 { - let resolve_ms = result.resolve_nanos / 1_000_000; + let resolve_ms = (result.resolve_nanos / 1_000_000) as u64; eprintln!( - "[gantt] resolve (entry): {}ms ({:.1}% of batch wall)", - resolve_ms, - batch_pct(result.resolve_nanos), + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror( + "resolve (entry)", + resolve_ms, + emoji + ) ); } - let wall_ms = result.wall_nanos / 1_000_000; - let ok = if result.ok { "PASS" } else { "FAIL" }; + let wall_ms = (result.wall_nanos / 1_000_000) as u64; + let label = if result.ok { + result.function.clone() + } else { + format!("{} [FAIL]", result.function) + }; eprintln!( - "[gantt] {}: {}ms [{ok}] ({:.1}% of batch wall)", - result.function, - wall_ms, - batch_pct(result.wall_nanos), + "{}", + v1_compiler::v1_rt::render_phase_concluded_line_mirror(&label, wall_ms, emoji) ); } } @@ -1969,7 +2251,8 @@ fn run_walk( fast_lane_eval_budget_ms: Option, falsifier_self_host_wet_budgets: FalsifierSelfHostWetBudgets, stop_policy: FloorBatchStopPolicy, - batch_wall_budgets_ms: Option<&[u128]>, + batch_clamp_params: Option<&[(u128, u128)]>, + budget_tighten_ms: Option, ) -> WalkOutcome { let mut any_failed = false; let mut batches_run = 0usize; @@ -1990,6 +2273,21 @@ fn run_walk( for (bi, batch) in batches.iter().enumerate() { batches_run = bi + 1; let units = group_batch_units(batch); + // Arm the observation heartbeat feed at batch-enter: discovery leaves + // entry_total pending (filled when the roster's entry-group count is known); + // SingleClaim arms immediately with the claim count. Never a fabricated 0-of-0. + let label = batch_heartbeat_label(batch); + let entry_total = if batch + .iter() + .any(|r| matches!(r, Runnable::DiscoveryBatch { .. })) + { + None + } else if batch.is_empty() { + None + } else { + Some(batch.len() as u64) + }; + heartbeat_feed_enter_batch(bi as u64, &label, entry_total); eprintln!( "claim_executor: batch {} — {} node(s) in {} resolve-group(s), governor target_width={}", bi + 1, @@ -2088,6 +2386,15 @@ fn run_walk( v1_compiler::v1_interpreter::group_end(); } for result in &batch_results { + // SingleClaim path: discovery advances the feed via + // `index_schedule_entry_completed`; gate batches have no schedule + // retention, so each claim result is the per-entry completed tick. + if !batch + .iter() + .any(|r| matches!(r, Runnable::DiscoveryBatch { .. })) + { + heartbeat_feed_entry_completed(); + } if result.ok { println!( "{}", @@ -2130,40 +2437,48 @@ fn run_walk( any_failed = true; } let batch_wall_nanos = batch_start.elapsed().as_nanos(); - // THE COST WALL (D5): over-budget is a typed, located refusal — batch id, measured - // wall, budget row — that reds the walk. It never widens (no rerun, no scope change, - // no cap raise); witness verdicts inside the batch stand as evaluated (the budget is - // an admission/scheduling fact, not a verdict term — carrier note has the ruling split). - if let Some(budgets) = batch_wall_budgets_ms { - if let Some(budget_ms) = budgets.get(bi) { - let wall_ms = batch_wall_nanos / 1_000_000; - if wall_ms > *budget_ms { - println!( - "{}", - paint( - &format!( - "✗ FLOOR-BATCH-OVER-BUDGET batch={} wall_ms={} budget_ms={} (budget row: gunbc.ci_spec gunbc_ci_floor_batch_wall_budget_seconds[{}]; raising it requires an appended receipt note per gunbc_ci_floor_batch_wall_budget_note — a refusal, never a widen)", - bi + 1, - wall_ms, - budget_ms, - bi - ), - sgr::ERROR - ) - ); - failure_details.push(format!( - "batch={} BudgetExceeded{{wall_ms={},budget_ms={}}}", - bi + 1, - wall_ms, - budget_ms - )); - any_failed = true; - } + // THE COST WALL (Piece 3 derived clamp): the per-batch clamp is overhead + runtime unit + // count * rate, computed HERE where the affected-set-selected count is known (the schedule + // holds one opaque discovery runnable; the count is runtime). Over-clamp is a typed, located + // refusal that reds the walk; it never widens (no rerun, no scope change, no cap raise). + // Witness verdicts inside the batch stand as evaluated (the clamp is an admission/scheduling + // fact, not a verdict term — carrier note has the ruling split). + let batch_unit_count = batch_runtime_unit_count(&batch_results); + let batch_clamp_ms: Option = + batch_clamp_params + .and_then(|p| p.get(bi)) + .map(|&(overhead_ms, rate_ms)| { + let mut clamp = overhead_ms + batch_unit_count * rate_ms; + if let Some(t) = budget_tighten_ms { + clamp = clamp.min(t); + } + clamp + }); + if let Some(clamp_ms) = batch_clamp_ms { + let wall_ms = batch_wall_nanos / 1_000_000; + if wall_ms > clamp_ms { + println!( + "{}", + paint( + &format!( + "✗ FLOOR-BATCH-OVER-BUDGET batch={} wall_ms={} clamp_ms={} units={} (clamp = overhead + units*rate; authority gunbc.ci_spec gunbc_ci_floor_batch_clamp_params[{}]; raising an overhead or rate requires an operator-signed line per gunbc_ci_floor_batch_clamp_note — a refusal, never a widen)", + bi + 1, + wall_ms, + clamp_ms, + batch_unit_count, + bi + ), + sgr::ERROR + ) + ); + any_failed = true; } } batch_records.push(BatchRecord { batch_index: bi, wall_nanos: batch_wall_nanos, + clamp_ms: batch_clamp_ms, + unit_count: batch_unit_count, results: batch_results, }); if any_failed { @@ -2187,7 +2502,8 @@ fn run_walk( let total_wall_nanos = walk_start.elapsed().as_nanos(); emit_gantt(&batch_records, total_wall_nanos); let resolve_receipt_ok = write_resolve_receipt(&batch_records); - let batch_wall_receipt_ok = write_batch_wall_receipt(&batch_records, batch_wall_budgets_ms); + let batch_wall_receipt_ok = write_batch_wall_receipt(&batch_records); + let gate_warm_cost_receipt_ok = write_gate_warm_cost_receipt(&batch_records); // Memo contexts absorb their ledger totals into the process accumulator on // Drop, so they must die before the materialization receipt is written. drop(walk_memo); @@ -2196,6 +2512,7 @@ fn run_walk( any_failed: any_failed || !resolve_receipt_ok || !batch_wall_receipt_ok + || !gate_warm_cost_receipt_ok || !materialization_receipt_ok, batches_run, failure_details, @@ -2400,6 +2717,7 @@ fn run_perturb_check( FalsifierSelfHostWetBudgets::default(), FloorBatchStopPolicy::StopBeforeDependents, None, + None, ); let _ = fs::remove_dir_all(&tmp); @@ -2505,18 +2823,62 @@ fn run() -> Result { } }; - // Coarse wall-clock phase marks: the pre-walk phases (hygiene walk, policy - // install, plan resolve, plan eval, width eval) are interpreter-heavy and used - // to be SILENT — a 30-minute prelude looked identical to a hang. Every phase - // now stamps a line so the floor log itemizes its own time. + // Coarse phase marks for the pre-walk prelude (hygiene walk, policy install, + // plan resolve/eval, governor arm) — interpreter-heavy phases that used to be + // SILENT, so a 30-minute prelude looked identical to a hang. These now render + // through the single-authority observation renderer (`gunbc.observation_ci_render`, + // via the `gunbc.observation_seed_render` boundary) as `✅ done in ` instead of a raw `[t+…]` byte string the seed would fork the format + // into. Each mark carries its OWN wall (delta since the last mark), not a running + // `t+`, so the log itemizes which prelude phase is slow — the step toward the + // per-phase receipt keys the ci_spec prelude-coverage-hole follow-up calls for. let floor_started = Instant::now(); + let phase_last = std::cell::Cell::new(floor_started); + // Emoji glyphs under GitHub Actions, Unicode on a plain terminal (the same medium + // split `install_group_syntax` makes for `::group::` markers). + let phase_glyph_emoji = std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"); + // Placement basis only (see `gunbc.observation_seed_render`): the clamp overhead + // `gunbc.ci_spec` budgets for non-per-unit floor time, so a prelude phase completing + // within it reads Ambient. Coarse + placement-only — `ci_render_line` reads only the + // glyph and text — and dissolves when the stream driver reads the basis from ci_spec. + const PHASE_BASIS_OVERHEAD_MS: u64 = 300_000; + let phase_mark_roots = source_roots.clone(); let phase_mark = |label: &str| { - eprintln!( - "claim_executor: [t+{:.1}s] {label}", - floor_started.elapsed().as_secs_f64() - ); + let now = Instant::now(); + let delta_ms = now.saturating_duration_since(phase_last.get()).as_millis() as u64; + phase_last.set(now); + match render_phase_concluded_line( + &phase_mark_roots, + label, + delta_ms, + PHASE_BASIS_OVERHEAD_MS, + phase_glyph_emoji, + ) { + Some(line) => eprintln!("{line}"), + // §5: refuse to fabricate the pretty format when the renderer is unreachable, + // and never reproduce the deleted `[t+…]` marker — name the degradation loudly. + None => eprintln!( + "claim_executor: phase {label} (+{:.1}s) [observation renderer unavailable]", + (delta_ms as f64) / 1000.0 + ), + } }; + // Install the host-effect trace policy from the .dag authority FIRST — before the + // naming-hygiene walk below and every subsequent corpus read — so `[file] read` / + // `[rest]` / `[hermetic:mock]` etc. are funnelled per `gunbc.output_policy` + // (Instrumentation is Suppressed at Normal, the CI default) instead of flooding the + // floor log. Installing AFTER the walk (the prior order) left the walk's whole-tree + // read at the `Full` default — ~2.3k `[file] read` lines, the firehose the + // observation-emit census (`gunbc.observation_emit_census`) targets. The walk still + // runs before plan evaluation, so a naming violation stays the cheapest failure. + v1_compiler::cli_run::install_output_policy(&source_roots); + // Install the per-target group-marker syntax (GitHub Actions `::group::` vs a + // plain-terminal header) from the .dag authority, so the parallel walk folds each + // batch's host-effect traces into a collapsible group. + v1_compiler::cli_run::install_group_syntax(&source_roots); + phase_mark("output-policy + group-syntax install"); + // Under the opt-in inversion the plan's DiscoveryBatches carry explicit entries // only (or are absent entirely on an empty roster), and the explicit-only path // skips the tree-walk naming hygiene (`test fn` outside `*_test.dag`, `__` @@ -2534,17 +2896,7 @@ fn run() -> Result { return Err(ExitCode::from(1)); } } - phase_mark("naming-hygiene walk complete"); - - // Install the host-effect trace policy from the .dag authority once, before - // discovery threads spawn, so `[file] read` / `[rest]` / `[hermetic:mock]` etc. - // are funnelled per `gunbc.output_policy` instead of flooding the floor log. - v1_compiler::cli_run::install_output_policy(&source_roots); - // Install the per-target group-marker syntax (GitHub Actions `::group::` vs a - // plain-terminal header) from the .dag authority, so the parallel walk folds each - // batch's host-effect traces into a collapsible group. - v1_compiler::cli_run::install_group_syntax(&source_roots); - phase_mark("output policy + group syntax installed"); + phase_mark("naming-hygiene walk"); if perturb_check { return run_perturb_check(&source_roots, &plan_entry, &plan_function); @@ -2560,7 +2912,7 @@ fn run() -> Result { return Err(ExitCode::from(1)); } }; - phase_mark("plan entry resolved"); + phase_mark("plan resolve"); let plan_ctx = make_eval_context(&plan_graph, plan_indices.clone(), ExecutionMode::Hermetic); let batches = match eval_plan_in_ctx(&plan_ctx, &plan_entry, &plan_function) { @@ -2673,12 +3025,13 @@ fn run() -> Result { FalsifierSelfHostWetBudgets::default() }; let batch_stop_policy = resolve_floor_batch_stop_policy(&plan_ctx, &plan_function); - // THE COST WALL (D5): the floor plan's per-batch wall budgets, read fail-closed at arm - // time (the fast-lane-budget pattern). Scoped to the full floor plan only: the - // plan-artifact shortcut runs a single batch of the same schedule and the falsifier - // carries its own receipt budgets, so neither reads this list. - let batch_wall_budgets_ms: Option> = if plan_function == "gunbc_ci_floor_batches" { - match read_floor_batch_wall_budgets_ms(&plan_ctx, batches.len()) { + // THE COST WALL (Piece 3 derived clamp): the floor plan's per-batch clamp params, read + // fail-closed at arm time (the fast-lane-budget pattern). Scoped to the full floor plan only: + // the plan-artifact shortcut runs a single batch of the same schedule and the falsifier + // carries its own receipt budgets, so neither reads these lists. + let batch_clamp_params: Option> = if plan_function == "gunbc_ci_floor_batches" + { + match read_floor_batch_clamp_params(&plan_ctx, batches.len()) { Ok(v) => Some(v), Err(msg) => { eprintln!("{msg}"); @@ -2688,8 +3041,15 @@ fn run() -> Result { } else { None }; + let budget_tighten_ms: Option = match read_floor_batch_budget_tighten_ms() { + Ok(v) => v, + Err(msg) => { + eprintln!("{msg}"); + return Err(ExitCode::from(1)); + } + }; drop(plan_ctx); - phase_mark("plan evaluated"); + phase_mark("plan eval"); eprintln!( "claim_executor: [{}] executor plan = {} batch(es) from {}::{}", @@ -2719,7 +3079,7 @@ fn run() -> Result { return Err(ExitCode::from(1)); } } - phase_mark("memory governor armed; starting batch walk"); + phase_mark("memory-governor arm"); spawn_floor_memory_heartbeat(); // Plans whose schedule carries the compile-clean gate node: the gate only CONSUMES the @@ -2750,24 +3110,41 @@ fn run() -> Result { fast_lane_eval_budget_ms, falsifier_self_host_wet_budgets, batch_stop_policy, - batch_wall_budgets_ms.as_deref(), + batch_clamp_params.as_deref(), + budget_tighten_ms, ); + // Floor receipts block — data, not outcomes. One named group; pulse glyphs only + // (operator live-log 2026-07-25: outcome glyphs for outcomes only). + v1_compiler::v1_interpreter::group_begin("floor receipts"); match peak_rss_bytes() { Some(bytes) => { - eprintln!("[measurement] floor peak RSS: {bytes} bytes (VmHWM) (adaptive width)"); + eprintln!( + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror( + "floor peak RSS (adaptive width)", + Some(bytes), + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"), + ) + ); } None => eprintln!( - "[measurement] floor peak RSS: unavailable (no /proc/self/status) (adaptive width)" + "{}", + v1_compiler::cli_run::render_peak_rss_line_mirror( + "floor peak RSS (adaptive width)", + None, + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true"), + ) ), } // The governor receipt is the §5-counted degradation story for the run: every graceful // hold, hard back-off, and forced-serial admission, beside the width actually reached. eprintln!("{}", governor.receipt_line()); - // [measurement] WHOLE-TREE cgroup peak — the SOUND placement divisor input (SELF-RSS above omits + // WHOLE-TREE cgroup peak — the SOUND placement divisor input (SELF-RSS above omits // child rustc/sccache PIDs; cgroup-v2 `memory.peak` at the leaf job cgroup is hierarchical and // captures them). Single authority `emit_cgroup_measurement` so the `ci` and `rust_tests` jobs // report an identically-shaped line. Runtime-harmless read-only. emit_cgroup_measurement("floor adaptive-width"); + v1_compiler::v1_interpreter::group_end(); if outcome.any_failed { emit_falsifier_failure_class(&outcome.failure_details); } @@ -3023,7 +3400,7 @@ mod tests { let _ = fs::remove_file(&base); fs::write(&base, b"a file where the receipt dir should be").unwrap(); assert!(!write_resolve_receipt_at(&base, &[])); - assert!(!write_batch_wall_receipt_at(&base, &[], None)); + assert!(!write_batch_wall_receipt_at(&base, &[])); let _ = fs::remove_file(&base); } @@ -3035,30 +3412,40 @@ mod tests { let base = std::env::temp_dir().join(format!("claim-executor-batch-wall-{}", std::process::id())); let _ = fs::remove_dir_all(&base); - let records = vec![ + // Clamped: batch 0 wall 5s > clamp 2s (OverBudget); batch 1 wall 1s < clamp 2s (WithinBudget). + let clamped_records = vec![ BatchRecord { batch_index: 0, wall_nanos: 5_000_000_000, // 5s + clamp_ms: Some(2_000), // 2s + unit_count: 3, results: Vec::new(), }, BatchRecord { batch_index: 1, wall_nanos: 1_000_000_000, // 1s + clamp_ms: Some(2_000), // 2s + unit_count: 0, results: Vec::new(), }, ]; - let budgets_ms: Vec = vec![2_000, 2_000]; // 2s each: batch 1 over, batch 2 within - assert!(write_batch_wall_receipt_at( - &base, - &records, - Some(&budgets_ms) - )); + assert!(write_batch_wall_receipt_at(&base, &clamped_records)); let body = fs::read_to_string(base.join("floor-batch-wall-receipt.txt")).unwrap(); assert!(body.contains("batch_1_wall_ms=5000")); + assert!(body.contains("batch_1_units=3")); + assert!(body.contains("batch_1_clamp_ms=2000")); assert!(body.contains("batch_1_verdict=OverBudget")); assert!(body.contains("batch_2_verdict=WithinBudget")); assert!(body.contains("over_budget_batches=1")); - assert!(write_batch_wall_receipt_at(&base, &records, None)); + // Clamp-less (falsifier/regen plans): records carry clamp_ms None -> Unbudgeted. + let unbudgeted_records = vec![BatchRecord { + batch_index: 0, + wall_nanos: 5_000_000_000, + clamp_ms: None, + unit_count: 0, + results: Vec::new(), + }]; + assert!(write_batch_wall_receipt_at(&base, &unbudgeted_records)); let body = fs::read_to_string(base.join("floor-batch-wall-receipt.txt")).unwrap(); assert!(body.contains("batch_1_verdict=Unbudgeted")); assert!(body.contains("over_budget_batches=0")); @@ -3171,6 +3558,654 @@ mod tests { ); } + // The seed→.dag render boundary by execution: `render_phase_concluded_line` + // resolves `gunbc.observation_seed_render` and projects a floor phase mark through + // the single-authority renderer, so the seed speaks the observation vocabulary + // rather than a raw `[t+…]` byte string. Discriminating RED: a helper that fell + // back to the raw marker, dropped the phase, or forked the duration format fails + // one of the three asserts below (human units, completed glyph, no old marker). + #[test] + fn phase_mark_renders_through_the_observation_render_authority() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + // Emoji tier (the CI target): 48s concludes as `✅ naming-hygiene walk done in + // 48 seconds`. overhead_ms is the placement basis only (the 300s clamp overhead). + let line = + render_phase_concluded_line(&roots, "naming-hygiene walk", 48_000, 300_000, true) + .expect("observation_seed_render must resolve and render a phase line"); + assert!( + line.contains("naming-hygiene walk") && line.contains("done in 48 seconds"), + "phase line must name the phase in human units: {line:?}" + ); + assert!( + line.starts_with('✅'), + "a Done outcome concludes with the completed glyph at the Emoji tier: {line:?}" + ); + assert!( + !line.contains("[t+"), + "the projection must not carry the deleted raw phase marker: {line:?}" + ); + } + + fn run_seed_phase_begin_line( + source_roots: &[String], + phase: &str, + overhead_ms: u64, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "phase_begin_line", + &[ + (Some("phase".to_string()), Value::Str(phase.to_string())), + ( + Some("overhead_ms".to_string()), + Value::Int(overhead_ms as i64), + ), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + // Gantt flip byte-oracle: compile-path mirrors of phase_begin_line / + // phase_concluded_line must stay byte-equal to the .dag seed (justified + // divergence — interpreter render from inside compile would recurse). + #[test] + fn gantt_phase_mirrors_match_seed_oracle() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + let begin_oracle = run_seed_phase_begin_line(&roots, "compile.frontend", 300_000, true) + .expect("phase_begin_line must resolve and render"); + let begin_mirror = + v1_compiler::v1_rt::render_phase_begin_line_mirror("compile.frontend", true); + assert_eq!( + begin_oracle, begin_mirror, + "begin mirror must be byte-equal to seed oracle" + ); + assert!( + begin_oracle.starts_with('🔄') && begin_oracle.contains("started compile.frontend"), + "begin line shape: {begin_oracle:?}" + ); + + let done_oracle = + render_phase_concluded_line(&roots, "compile.frontend", 12_000, 300_000, true) + .expect("phase_concluded_line must resolve and render"); + let done_mirror = v1_compiler::v1_rt::render_phase_concluded_line_mirror( + "compile.frontend", + 12_000, + true, + ); + assert_eq!( + done_oracle, done_mirror, + "concluded mirror must be byte-equal to seed oracle" + ); + assert!( + done_oracle.contains("compile.frontend done in 12 seconds"), + "concluded line shape: {done_oracle:?}" + ); + } + + fn run_seed_psi_hold_line( + source_roots: &[String], + avg10_bp: u64, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "seed_psi_hold_line", + &[ + (Some("avg10_bp".to_string()), Value::Int(avg10_bp as i64)), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + fn run_seed_high_water_hold_line( + source_roots: &[String], + current_bytes: u64, + high_water_bytes: u64, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "seed_high_water_hold_line", + &[ + ( + Some("current_bytes".to_string()), + Value::Int(current_bytes as i64), + ), + ( + Some("high_water_bytes".to_string()), + Value::Int(high_water_bytes as i64), + ), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + // Governor flip byte-oracle: HoldReason → ci_hold_cause_text mirrors must stay + // byte-equal to seed_psi_hold_line / seed_high_water_hold_line. + #[test] + fn governor_hold_mirrors_match_seed_oracle() { + use v1_compiler::memory_governor::{render_governor_hold_line_mirror, HoldReason}; + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + // 37.5% → 3750 basis points + let psi_oracle = run_seed_psi_hold_line(&roots, 3750, true) + .expect("seed_psi_hold_line must resolve and render"); + let psi_mirror = + render_governor_hold_line_mirror(&HoldReason::PsiPressure { avg10: 37.5 }, true); + assert_eq!( + psi_oracle, psi_mirror, + "psi hold mirror must be byte-equal to seed oracle" + ); + assert!( + psi_oracle.starts_with('⏳') && psi_oracle.contains("blocked on memory reclaim"), + "psi hold shape: {psi_oracle:?}" + ); + + let hw_oracle = run_seed_high_water_hold_line(&roots, 8_589_934_592, 10_737_418_240, true) + .expect("seed_high_water_hold_line must resolve and render"); + let hw_mirror = render_governor_hold_line_mirror( + &HoldReason::CurrentHighWater { + current: 8_589_934_592, + high_water: 10_737_418_240, + }, + true, + ); + assert_eq!( + hw_oracle, hw_mirror, + "high-water hold mirror must be byte-equal to seed oracle" + ); + assert!( + hw_oracle.contains("blocked on the memory high-water line"), + "high-water hold shape: {hw_oracle:?}" + ); + } + + fn run_seed_typecheck_concluded_line( + source_roots: &[String], + module_path: &str, + elapsed_ms: u64, + overhead_ms: u64, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "typecheck_concluded_line", + &[ + ( + Some("module_path".to_string()), + Value::Str(module_path.to_string()), + ), + ( + Some("elapsed_ms".to_string()), + Value::Int(elapsed_ms as i64), + ), + ( + Some("overhead_ms".to_string()), + Value::Int(overhead_ms as i64), + ), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + #[test] + fn typecheck_attribution_mirrors_match_seed_oracle() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + let oracle = run_seed_typecheck_concluded_line( + &roots, + "v2.compiler.normalized_tree", + 606_984, + 300_000, + true, + ) + .expect("typecheck_concluded_line must resolve and render"); + let mirror = v1_compiler::cli_run::render_typecheck_concluded_line_mirror( + "v2.compiler.normalized_tree", + 606_984, + true, + ); + assert_eq!( + oracle, mirror, + "typecheck concluded mirror must be byte-equal to seed oracle" + ); + assert!( + oracle.starts_with('✅') + && oracle.contains("typecheck v2.compiler.normalized_tree done in 10 minutes"), + "typecheck concluded shape: {oracle:?}" + ); + let begin_mirror = v1_compiler::cli_run::render_typecheck_begin_line_mirror( + "v2.compiler.normalized_tree", + true, + ); + assert_eq!( + begin_mirror, + "🔄 started typecheck v2.compiler.normalized_tree" + ); + } + + fn run_seed_peak_rss_line( + source_roots: &[String], + label: &str, + rss_bytes: u64, + rss_available: bool, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "seed_peak_rss_line", + &[ + (Some("label".to_string()), Value::Str(label.to_string())), + (Some("rss_bytes".to_string()), Value::Int(rss_bytes as i64)), + ( + Some("rss_available".to_string()), + Value::Bool(rss_available), + ), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + #[test] + fn peak_rss_mirror_matches_seed_oracle() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + // 1 GiB exact → "1.0 GiB" + let oracle = run_seed_peak_rss_line(&roots, "floor peak RSS", 1_073_741_824, true, true) + .expect("seed_peak_rss_line must resolve and render"); + let mirror = v1_compiler::cli_run::render_peak_rss_line_mirror( + "floor peak RSS", + Some(1_073_741_824), + true, + ); + assert_eq!(oracle, mirror, "peak RSS mirror must be byte-equal to seed"); + assert_eq!(oracle, "🕐 floor peak RSS — 1.0 GiB"); + + // 1536 MiB → "1.5 GiB" + let oracle15 = run_seed_peak_rss_line(&roots, "cgroup peak", 1_610_612_736, true, false) + .expect("seed"); + let mirror15 = v1_compiler::cli_run::render_peak_rss_line_mirror( + "cgroup peak", + Some(1_610_612_736), + false, + ); + assert_eq!(oracle15, mirror15); + assert_eq!(oracle15, "◷ cgroup peak — 1.5 GiB"); + + let unread = run_seed_peak_rss_line(&roots, "floor peak RSS", 0, false, true) + .expect("unreadable seed"); + let unread_mirror = + v1_compiler::cli_run::render_peak_rss_line_mirror("floor peak RSS", None, true); + assert_eq!(unread, unread_mirror); + assert!(unread.contains("unreadable (no /proc/self/status)")); + } + + fn run_seed_shell_effect_failed_line( + source_roots: &[String], + intent: &str, + argv_collapsed: &str, + exit_code: u64, + elapsed_ms: u64, + overhead_ms: u64, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "shell_effect_failed_line", + &[ + (Some("intent".to_string()), Value::Str(intent.to_string())), + ( + Some("argv_collapsed".to_string()), + Value::Str(argv_collapsed.to_string()), + ), + (Some("exit_code".to_string()), Value::Int(exit_code as i64)), + ( + Some("elapsed_ms".to_string()), + Value::Int(elapsed_ms as i64), + ), + ( + Some("overhead_ms".to_string()), + Value::Int(overhead_ms as i64), + ), + (Some("emoji".to_string()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + #[test] + fn shell_effect_failed_mirror_matches_seed_oracle() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + let oracle = run_seed_shell_effect_failed_line( + &roots, + "shell.Exec.Run", + "echo hi", + 1, + 2000, + 300_000, + true, + ) + .expect("shell_effect_failed_line must resolve and render"); + let mirror = v1_compiler::v1_interpreter::render_shell_effect_failed_line_mirror( + "shell.Exec.Run", + "echo hi", + 1, + 2000, + true, + ); + assert_eq!( + oracle, mirror, + "shell Failed mirror must be byte-equal to seed oracle" + ); + assert_eq!( + oracle, + "❌ shell.Exec.Run failed: $ echo hi (exit=1) in 2 seconds" + ); + } + + #[allow(clippy::too_many_arguments)] + fn run_seed_heartbeat_line( + source_roots: &[String], + elapsed_ms: u64, + batch_label: &str, + entry_index: u64, + entry_total: u64, + rss: Option, + swap: Option, + pressure: Option, + emoji: bool, + ) -> Option { + let entry = source_roots + .iter() + .map(|r| Path::new(r).join("gunbc/observation_seed_render.dag")) + .find(|p| p.exists())? + .to_string_lossy() + .into_owned(); + let (graph, indices) = resolve_entry_graph_shared(source_roots, &entry).ok()?; + let ctx = make_eval_context(&graph, indices, ExecutionMode::Hermetic); + let out = run_in_context_with_args( + &ctx, + "seed_heartbeat_line", + &[ + (Some("elapsed_ms".into()), Value::Int(elapsed_ms as i64)), + (Some("batch_index".into()), Value::Int(0)), + ( + Some("batch_label".into()), + Value::Str(batch_label.to_string()), + ), + (Some("entry_index".into()), Value::Int(entry_index as i64)), + (Some("entry_total".into()), Value::Int(entry_total as i64)), + ( + Some("rss_bytes".into()), + Value::Int(rss.unwrap_or(0) as i64), + ), + (Some("rss_available".into()), Value::Bool(rss.is_some())), + ( + Some("swap_bytes".into()), + Value::Int(swap.unwrap_or(0) as i64), + ), + (Some("swap_available".into()), Value::Bool(swap.is_some())), + ( + Some("pressure_bp".into()), + Value::Int(pressure.unwrap_or(0) as i64), + ), + ( + Some("pressure_available".into()), + Value::Bool(pressure.is_some()), + ), + (Some("emoji".into()), Value::Bool(emoji)), + ], + false, + ) + .ok()?; + match out { + Value::Str(s) => Some(s), + _ => None, + } + } + + // The floor-memory heartbeat's seed→.dag boundary, proven by execution: + // seed_heartbeat_line takes the primitives the heartbeat thread has (elapsed, + // batch label, entry position, memory vitals) and projects them through the one + // renderer — identity first, human units, no raw byte dump. These golden strings + // are the oracle the Rust mirror is proven byte-equal to + // (`render_heartbeat_line_mirror_matches_seed_oracle`); the subject is batch-grain + // (parallel entries → no fabricated per-module detail). + #[test] + fn seed_heartbeat_line_renders_identity_first_in_human_units() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + // The captured crawl window's memory beat: identity first, human units, the + // raw byte value absent. + let line = run_seed_heartbeat_line( + &roots, + 1_980_000, + "witness discovery", + 214, + 602, + Some(16_107_200_512), + Some(34_359_738_368), + Some(901), + true, + ) + .expect("seed_heartbeat_line must resolve and render"); + assert_eq!( + line, + "🕐 33 minutes in — still in witness discovery: entry 214 of 602. memory 15.0 GiB, swap 32.0 GiB, pressure 9.0%" + ); + assert!( + !line.contains("16107200512"), + "raw bytes must not appear: {line}" + ); + + // An unreadable cgroup field names its cause, never a fabricated zero. + let unreadable = run_seed_heartbeat_line( + &roots, + 500, + "self-host fixed-point", + 0, + 2, + None, + Some(0), + None, + true, + ) + .expect("resolve"); + assert_eq!( + unreadable, + "🕐 500ms in — still in self-host fixed-point: entry 0 of 2. memory unreadable (cgroup field unreadable), swap 0.0 GiB, pressure unreadable (cgroup field unreadable)" + ); + } + + // Wiring flip 4b: the Rust mirror is proven byte-equal to the 4a seed oracle. + // The heartbeat thread cannot call the interpreter (duplicate module index under + // the memory envelope it watches); this pin is what keeps the mirror honest. + #[test] + fn render_heartbeat_line_mirror_matches_seed_oracle() { + let root = workspace_root(); + let roots = vec![ + root.join("src/v2").to_string_lossy().into_owned(), + root.join("dag").to_string_lossy().into_owned(), + ]; + let crawl = run_seed_heartbeat_line( + &roots, + 1_980_000, + "witness discovery", + 214, + 602, + Some(16_107_200_512), + Some(34_359_738_368), + Some(901), + true, + ) + .expect("seed oracle"); + let mirror = render_heartbeat_line_mirror( + 1_980_000, + "witness discovery", + 214, + 602, + Some(16_107_200_512), + Some(34_359_738_368), + Some(901), + true, + ); + assert_eq!( + mirror, crawl, + "mirror must be byte-equal to the seed oracle" + ); + assert_eq!( + mirror, + "🕐 33 minutes in — still in witness discovery: entry 214 of 602. memory 15.0 GiB, swap 32.0 GiB, pressure 9.0%" + ); + + let unreadable_oracle = run_seed_heartbeat_line( + &roots, + 500, + "self-host fixed-point", + 0, + 2, + None, + Some(0), + None, + true, + ) + .expect("seed oracle"); + let unreadable_mirror = render_heartbeat_line_mirror( + 500, + "self-host fixed-point", + 0, + 2, + None, + Some(0), + None, + true, + ); + assert_eq!( + unreadable_mirror, unreadable_oracle, + "unreadable fields must stay byte-equal" + ); + assert!( + !mirror.contains("16107200512") && !unreadable_mirror.contains("[floor-memory]"), + "mirror must not carry the deleted byte-dump shape" + ); + } + + #[test] + fn psi_avg10_converts_to_basis_points_at_observation_scale() { + assert_eq!(psi_avg10_to_basis_points("9.01"), Some(901)); + assert_eq!(psi_avg10_to_basis_points("37.5"), Some(3750)); + assert_eq!(psi_avg10_to_basis_points("0.0"), Some(0)); + assert_eq!(psi_avg10_to_basis_points("garbage"), None); + } + // The materialization-receipt chain by execution: a real entry resolves, a // claim evaluates on its InterpContext, and the ctx Drop absorbs ledger // totals into the process accumulator. The env latch is process-global and diff --git a/src/v1/stage0/src/cli_run.rs b/src/v1/stage0/src/cli_run.rs index 569f3882d90..2d5ed57eb18 100644 --- a/src/v1/stage0/src/cli_run.rs +++ b/src/v1/stage0/src/cli_run.rs @@ -3,7 +3,7 @@ use std::cell::{Cell, RefCell}; use std::collections::{BTreeMap, BTreeSet, HashSet, VecDeque}; use std::path::{Path, PathBuf}; use std::rc::Rc; -use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering}; use std::sync::{Arc, Mutex, OnceLock, RwLock}; use crate::coproduct_reflection::{decl_facts_corpus_walk, DeclFactRaw}; @@ -757,6 +757,65 @@ mod process_workspace_root_tests { /// Pins HAND-RUST `repo_relative_path` against `gunbc.cli_run_repo_grant` on the same /// fixture spellings. Witness: `dag/test/claim/cli_run_repo_grant_hand_rust_equivalence_witness_test.dag`. +#[cfg(test)] +mod cli_run_arg_channel_tests { + use super::parse_run_args; + use crate::v1_interpreter::Value; + + fn spec(items: &[&str]) -> Vec { + items.iter().map(|s| s.to_string()).collect() + } + + #[test] + fn named_arg_parses_to_a_named_string_value() { + let got = parse_run_args(&spec(&["node_id=roadmap-7"])).expect("well-formed --arg"); + assert_eq!(got.len(), 1); + assert_eq!(got[0].0.as_deref(), Some("node_id")); + assert!(matches!(&got[0].1, Value::Str(s) if s == "roadmap-7")); + } + + #[test] + fn value_may_contain_further_equals_signs() { + let got = parse_run_args(&spec(&["diff=a=b=c"])).expect("split on the first `=` only"); + assert!(matches!(&got[0].1, Value::Str(s) if s == "a=b=c")); + } + + #[test] + fn empty_value_is_admitted_and_distinct_from_absent() { + let got = parse_run_args(&spec(&["flag="])).expect("empty value is a value"); + assert!(matches!(&got[0].1, Value::Str(s) if s.is_empty())); + } + + // RED controls: the refusals are the point of the channel (§5). A bare + // token has no defensible position to occupy, so it must not be guessed + // into one. + #[test] + fn bare_token_without_equals_refuses() { + let err = parse_run_args(&spec(&["node_id"])).expect_err("no `=` must refuse"); + assert!( + err.contains("name=value"), + "diagnostic names the form: {err}" + ); + } + + #[test] + fn empty_parameter_name_refuses() { + let err = parse_run_args(&spec(&["=orphan"])).expect_err("empty name must refuse"); + assert!( + err.contains("empty parameter name"), + "diagnostic locates the fault: {err}" + ); + } + + #[test] + fn one_malformed_spec_refuses_the_whole_list() { + assert!( + parse_run_args(&spec(&["ok=1", "broken", "also_ok=2"])).is_err(), + "a partial parse would silently drop a caller's argument" + ); + } +} + #[cfg(test)] mod cli_run_repo_grant_equivalence_tests { use super::{process_workspace_root, repo_relative_path}; @@ -2975,6 +3034,9 @@ fn floor_compile_clean_emit_ok_via_index( sources, ResolveTypecheckGate::Strict, "floor-compile-clean-gate", + // Ephemeral: the whole-tree aggregate graph must NOT join the process share tier + // (D0.1) — it would pin every TypedModule in the tree for the process lifetime. + ResolvedGraphMemoShare::Ephemeral, ) { Ok(resolved) => resolved, Err(msg) => { @@ -4182,19 +4244,6 @@ pub fn import_closure_live_paths_with_facts( import_closure_from_adjacency(entry_path, &facts.adjacency) } -/// Like `import_closure_live_paths_with_facts` but over the WIDER `selection_adjacency` -/// (import + strict reference edges) — so a module reached cross-entry only through a -/// bare-reference edge is still refcounted by schedule-derived retention. Over-retaining -/// (the safe direction: never premature-evict a reference-reached module into a -/// recompute-on-miss) and equally cheap: a BFS over prebuilt adjacency, no per-entry -/// source loading, no #6848 both-closure fixpoint. -pub(crate) fn selection_closure_live_paths_with_facts( - entry_path: &str, - facts: &ModuleGraphFactsLive, -) -> Vec { - import_closure_from_adjacency(entry_path, &facts.selection_adjacency) -} - impl ModuleGraphFactsLive { /// Repo-relative paths of every declared module in the facts scan — the existence /// set for refuse-vs-answer decisions (a module can be absent from `adjacency` @@ -5958,6 +6007,30 @@ pub fn entry_closure_sources_len_for_test(index: &MultiEntryIndex) -> usize { index.entry_closure_sources.borrow().len() } +/// Drop every assembled per-closure graph from the in-process share memo, leaving the +/// per-module typed cache warm. Reproduces the production state after the compile-clean +/// gate warms the shared index Ephemerally (D0.1): modules cached, no per-entry graph +/// pin — so the next per-entry resolve misses the memo and takes reconcile's ALL-HITS +/// probe, the path D0.2's schedule-key registration lives on. +#[cfg(any(test, feature = "interp_test_witness"))] +pub fn clear_resolved_graph_memo_for_test(index: &MultiEntryIndex) { + index.resolved_graph_memo.borrow_mut().clear(); +} + +/// Install a schedule retention armed over `per_entry` with an EXPLICIT eviction switch, +/// bypassing the process-global `GUNBC_SCHEDULE_RETENTION_EVICT` env — so the D0.4 +/// retain-all (`evict_enabled=false`) baseline can be exercised on a real index without +/// racing that env against concurrently-running tests. +#[cfg(any(test, feature = "interp_test_witness"))] +pub fn install_schedule_retention_for_test( + index: &MultiEntryIndex, + per_entry: Vec<(String, Vec)>, + evict_enabled: bool, +) { + *index.schedule_retention.borrow_mut() = + Some(ScheduleRetention::armed(per_entry, evict_enabled)); +} + #[cfg(any(test, feature = "interp_test_witness"))] pub fn both_closure_edges_initialized_for_test(index: &MultiEntryIndex) -> bool { index.both_closure_edges.borrow().is_some() @@ -6587,6 +6660,14 @@ impl ScheduleRetention { pub fn resolved_graph_evictions(&self) -> u64 { self.resolved_graph_evictions } + /// The eviction switch this retention was armed with (false = + /// `GUNBC_SCHEDULE_RETENTION_EVICT=0`, the retain-all measurement pole). Consumers + /// outside the bookkeeper — the index's resolved-graph pin drop — gate on it so the + /// pole retains resolved graphs too, not just the per-module caches (D0.4: the pre-M2 + /// baseline was invalid because it kept dropping graphs unconditionally). + pub fn evict_enabled(&self) -> bool { + self.evict_enabled + } pub fn note_graph_eviction(&mut self) { self.resolved_graph_evictions += 1; } @@ -6619,16 +6700,33 @@ fn index_arm_schedule_retention(index: &MultiEntryIndex, rows: &[DiscoveryRow]) if !seen.insert(row.entry.as_str()) { continue; } - // CHEAP closure: a BFS over the prebuilt selection adjacency (import + strict - // reference edges; no per-entry source loading, no #6848 both-closure fixpoint - // walk) — keyed by the repo-relative path the reconcile loop records - // (`decl_file`). The prior `collect_both_closure_module_names_for_entry` - // front-loaded the source-loading fixpoint for every distinct entry (the - // compiler-affected worst case is ~694), a cost the affected-set path otherwise - // skips or defers. The wider selection tier over-retains (never premature-evicts - // a reference-reached module into a recompute-on-miss); a module reached by no - // edge at all surfaces as counted RetentionUnknown at reconcile (retained, §5). - let paths = selection_closure_live_paths_with_facts(&row.entry, &index.module_graph_facts); + // Arm from the LOADER's EXACT closure — `load_sources_for_entry_with_pool`, the + // same function the discovery reconcile path runs (`resolve_entry_with_parse_cache`). + // This is the ONE closure authority the arming and the loader now share (D0.3 / + // the #6985 Class-B root, third appearance): the prior CHEAP + // `selection_closure_live_paths_with_facts` walked only `selection_adjacency` + // (import edges + strict reference edges for import-LESS files), so it OMITTED + // modules the loader reaches via qualified-projection references emitted by + // import-BEARING files. Such a module, being present in some OTHER entry's global + // refcount, was counted neither `RetentionUnknown` nor re-evicted after it was + // re-cached on load — an invisible resident leak. Consuming the loader's own output + // makes the armed closure ⊇ the reconciled set BY CONSTRUCTION, not by two closures + // happening to agree (the doc's "name the one closure authority, don't add a third + // adjacency"). Cost is neutral, not the front-load the old comment feared: the pool + // loader MEMOIZES into `entry_closure_sources`, so this pre-load is exactly what + // discovery would compute and is reused on the entry's resolve, never doubled. Keyed + // by `workspace_relative_repo_path(&sf.path)` — the identical form the reconcile + // record site derives from `resolved.module.span.file` (`decl_file`), so refcount + // keys and record keys still coincide. A closure that cannot be loaded is skipped: + // its modules become counted `RetentionUnknown` at reconcile (retained — §5 + // fail-closed), so arming still never fails the run. + let paths = match load_sources_for_entry_with_pool(index, &row.entry) { + Ok(sources) => sources + .iter() + .map(|sf| workspace_relative_repo_path(&sf.path)) + .collect::>(), + Err(_) => continue, + }; per_entry.push((row.entry.clone(), paths)); } let evict_enabled = schedule_retention_evict_enabled(); @@ -6670,6 +6768,173 @@ fn index_record_schedule_module( } } +/// Kept so `gunbc.observation_emit_census` roster hygiene cannot go stale after the +/// raw `[typecheck-attribution]` key=value shape dissolves into the observation projection. +#[allow(dead_code)] +pub const TYPECHECK_ATTRIBUTION_CENSUS_MARKER: &str = "[typecheck-attribution]"; + +/// Kept so `gunbc.observation_emit_census` roster hygiene cannot go stale after the +/// raw `[measurement] … bytes (VmHWM)` dump dissolves into `ci_measurement_rss_line`. +#[allow(dead_code)] +pub const MEASUREMENT_CENSUS_MARKER: &str = "[measurement]"; + +const MEASUREMENT_UNREADABLE_CAUSE: &str = "no /proc/self/status"; + +fn measurement_emoji() -> bool { + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true") +} + +fn measurement_human_bytes(bytes: u64) -> String { + // Mirror of ci_gibibyte_tenths: (bytes * 10) / gibibyte_scale_factor_bytes (2^30). + let tenths = (bytes.saturating_mul(10)) / 1_073_741_824; + format!("{}.{} GiB", tenths / 10, tenths % 10) +} + +/// Mirror of `gunbc.observation_seed_render.seed_peak_rss_line` — +/// `ci_measurement_rss_line ∘ ci_render_line`. Identity-first label, human GiB. +/// When `rss_bytes` is `None`, `unreadable_cause` names the MeasuredUnavailable cause +/// (seed default: `no /proc/self/status`). +pub fn render_peak_rss_line_mirror(label: &str, rss_bytes: Option, emoji: bool) -> String { + render_peak_rss_line_mirror_with_cause(label, rss_bytes, MEASUREMENT_UNREADABLE_CAUSE, emoji) +} + +pub fn render_peak_rss_line_mirror_with_cause( + label: &str, + rss_bytes: Option, + unreadable_cause: &str, + emoji: bool, +) -> String { + let _ = MEASUREMENT_CENSUS_MARKER; + let glyph = if emoji { "🕐" } else { "◷" }; + let rss = match rss_bytes { + Some(b) => measurement_human_bytes(b), + None => format!("unreadable ({unreadable_cause})"), + }; + format!("{glyph} {label} — {rss}") +} + +const TYPECHECK_MINUTE_SWITCH_SECONDS: u64 = 90; + +fn typecheck_emoji() -> bool { + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true") +} + +fn typecheck_human_duration(ms: u64) -> String { + if ms < 1_000 { + format!("{ms}ms") + } else if ms < TYPECHECK_MINUTE_SWITCH_SECONDS * 1_000 { + format!("{} seconds", ms / 1_000) + } else { + format!("{} minutes", ms / 60_000) + } +} + +/// Mirror of `gunbc.observation_seed_render.typecheck_begin_line` — +/// leaf text is `typecheck ` (ModuleSegment + PhaseSegment). +pub fn render_typecheck_begin_line_mirror(module_path: &str, emoji: bool) -> String { + let glyph = if emoji { "🔄" } else { "◐" }; + format!("{glyph} started typecheck {module_path}") +} + +/// Mirror of `gunbc.observation_seed_render.typecheck_concluded_line`. +pub fn render_typecheck_concluded_line_mirror( + module_path: &str, + elapsed_ms: u64, + emoji: bool, +) -> String { + let glyph = if emoji { "✅" } else { "✓" }; + format!( + "{glyph} typecheck {module_path} done in {}", + typecheck_human_duration(elapsed_ms) + ) +} + +/// Snapshot of the floor's active-batch progress, sampled by the detached +/// floor-memory heartbeat thread. Armed only when `entry_total` is known and +/// non-zero — never a fabricated 0-of-0 (observation law 2 / §5). +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct HeartbeatFeedSnapshot { + pub batch_index: u64, + pub batch_label: String, + pub entry_done: u64, + pub entry_total: u64, +} + +struct HeartbeatFeedState { + batch_index: u64, + batch_label: String, + /// `None` while a discovery batch has entered but its roster entry-count is + /// not yet known — the heartbeat thread skips emit until this is `Some(n>0)`. + entry_total: Option, + entry_done: AtomicU64, +} + +/// Process-wide feed the floor-memory heartbeat samples. Updated at batch-enter +/// (label + total when known) and at each `index_schedule_entry_completed` (and +/// SingleClaim completion) — the existing per-entry point, never a parallel counter. +static HEARTBEAT_FEED: Mutex> = Mutex::new(None); + +/// Enter a floor batch. `entry_total = Some(n)` arms the feed when `n > 0`; +/// `None` leaves it pending (discovery: total filled once the roster is known). +/// Resets `entry_done` to 0. A zero total is refused (§5: never fabricate 0-of-0). +pub fn heartbeat_feed_enter_batch(batch_index: u64, label: &str, entry_total: Option) { + let total = match entry_total { + Some(0) => None, + other => other, + }; + let mut g = HEARTBEAT_FEED.lock().unwrap_or_else(|p| p.into_inner()); + *g = Some(HeartbeatFeedState { + batch_index, + batch_label: label.to_string(), + entry_total: total, + entry_done: AtomicU64::new(0), + }); +} + +/// Fill the entry total once a discovery roster's entry-group count is known. +/// No-op when `total == 0` (refuses to arm a 0-of-0). No-op when no batch is open. +pub fn heartbeat_feed_set_entry_total(total: u64) { + if total == 0 { + return; + } + let mut g = HEARTBEAT_FEED.lock().unwrap_or_else(|p| p.into_inner()); + if let Some(state) = g.as_mut() { + state.entry_total = Some(total); + } +} + +/// Record one completed entry (discovery source-file grain, or one SingleClaim). +/// Caps at `entry_total` when known so a late double-complete cannot invent +/// "entry N+1 of N". +pub fn heartbeat_feed_entry_completed() { + let g = HEARTBEAT_FEED.lock().unwrap_or_else(|p| p.into_inner()); + if let Some(state) = g.as_ref() { + let prev = state.entry_done.fetch_add(1, Ordering::Relaxed); + if let Some(total) = state.entry_total { + if prev >= total { + // Undo the overshoot — saturating at total. + state.entry_done.store(total, Ordering::Relaxed); + } + } + } +} + +/// The armed snapshot, or `None` when no batch is open / total still pending. +/// The heartbeat thread skips emit on `None` rather than printing a fabricated +/// progress line. +pub fn heartbeat_feed_snapshot() -> Option { + let g = HEARTBEAT_FEED.lock().unwrap_or_else(|p| p.into_inner()); + let state = g.as_ref()?; + let entry_total = state.entry_total?; + let entry_done = state.entry_done.load(Ordering::Relaxed).min(entry_total); + Some(HeartbeatFeedSnapshot { + batch_index: state.batch_index, + batch_label: state.batch_label.clone(), + entry_done, + entry_total, + }) +} + /// Drive one entry-completion: decrement the entry's closure refcounts, drop the /// per-module state that reached zero from every per-module cache (typed, parse, /// normalize-diag, ownership-diag, source-hash), AND drop the entry's assembled @@ -6684,13 +6949,19 @@ fn index_schedule_entry_completed( entry: &str, subject: Option<&str>, ) -> Result<(), String> { - let batch = { + let (batch, evict_enabled) = { let mut slot = index.schedule_retention.borrow_mut(); match slot.as_mut() { - Some(sr) => sr.entry_completed(entry)?, - None => return Ok(()), + Some(sr) => (sr.entry_completed(entry)?, sr.evict_enabled()), + None => { + // Schedule unarmed — the drain still advanced an entry; feed the heartbeat. + heartbeat_feed_entry_completed(); + return Ok(()); + } } }; + // The same per-entry point feeds the observation heartbeat — one counter, not a fork. + heartbeat_feed_entry_completed(); if !batch.typed_keys.is_empty() { let mut cache = index.typed_module_cache.borrow_mut(); for key in &batch.typed_keys { @@ -6712,13 +6983,17 @@ fn index_schedule_entry_completed( // Drop the completed entry's assembled graph. The entry's own `InterpContext` still // holds it until the next resolve, so this only removes the memo's pin; a rare later // entry with the identical closure re-resolves (a memo miss, cost only). + // Gate the resolved-graph pin drop on the SAME eviction switch as the per-module drops + // above: with GUNBC_SCHEDULE_RETENTION_EVICT=0 the retain-all measurement pole must + // retain EVERYTHING — graphs included — so it faithfully reproduces pre-M2 peak + // retention (D0.4). Before this the pole understated peak by silently freeing graphs. let graph_evicted = match subject { - Some(subj) => index + Some(subj) if evict_enabled => index .resolved_graph_memo .borrow_mut() .remove(subj) .is_some(), - None => false, + _ => false, }; let (sched_evictions, retention_unknown, graph_evictions) = { let mut slot = index.schedule_retention.borrow_mut(); @@ -6749,6 +7024,44 @@ fn index_schedule_entry_completed( Ok(()) } +#[cfg(test)] +mod heartbeat_feed_red_controls { + use super::{ + heartbeat_feed_enter_batch, heartbeat_feed_entry_completed, heartbeat_feed_set_entry_total, + heartbeat_feed_snapshot, + }; + + #[test] + fn never_arms_a_fabricated_zero_of_zero() { + heartbeat_feed_enter_batch(0, "witness discovery", Some(0)); + assert_eq!( + heartbeat_feed_snapshot(), + None, + "entry_total=0 must not arm the feed" + ); + heartbeat_feed_enter_batch(0, "witness discovery", None); + assert_eq!( + heartbeat_feed_snapshot(), + None, + "pending total must not arm the feed" + ); + heartbeat_feed_set_entry_total(0); + assert_eq!( + heartbeat_feed_snapshot(), + None, + "set_entry_total(0) must refuse to arm" + ); + heartbeat_feed_set_entry_total(602); + let snap = heartbeat_feed_snapshot().expect("armed after real total"); + assert_eq!(snap.batch_label, "witness discovery"); + assert_eq!(snap.entry_done, 0); + assert_eq!(snap.entry_total, 602); + heartbeat_feed_entry_completed(); + let snap = heartbeat_feed_snapshot().expect("still armed"); + assert_eq!(snap.entry_done, 1); + } +} + #[cfg(test)] mod schedule_retention_red_controls { use super::{ScheduleRetention, SCHEDULE_RETENTION_TUNABLE_COUNT}; @@ -6915,9 +7228,9 @@ mod schedule_retention_red_controls { } /// END-TO-END WIRING (real index, real closure computation, real cache eviction): - /// two entries sharing one library module. Arm from the schedule, resolve both - /// entries (populating the process-shared caches through the actual reconcile - /// path that records each module), then drive per-entry completion. The + /// two entries sharing one library module. PREWARM the caches, arm from the schedule + /// AFTER prewarming, then re-resolve both entries through reconcile's all-hits probe + /// (the D0.2 registration path), then drive per-entry completion. The /// shared module survives entry A's completion (still reachable by B) and drops /// only when B completes — proving the mechanism frees real per-module state on a /// live `MultiEntryIndex`, not just in the pure bookkeeper above. @@ -6965,9 +7278,24 @@ mod schedule_retention_red_controls { reads_live_tree: false, }, ]; + // PREWARM the per-module typed cache WITHOUT arming — the production order: + // compile-clean warms the shared index before discovery arms retention. Then clear + // the per-entry resolved-graph memo (D0.1 makes compile-clean's warming Ephemeral: + // modules cached, no per-entry graph pin) so the arm-time re-resolves MISS the memo + // and go through reconcile's ALL-HITS probe — the exact path D0.2 fixed. The + // pre-D0.4 order (arm, THEN first resolve) sent every module cold through the slow + // recording path and never exercised the probe, so the all-hit no-registration + // defect passed green: the §7 order-blindness this control now closes. + super::resolve_entry_with_index(&index, &entry_a).expect("prewarm a"); + super::resolve_entry_with_index(&index, &entry_b).expect("prewarm b"); + super::clear_resolved_graph_memo_for_test(&index); + + // Arm AFTER prewarming, then re-resolve: per-module cache warm + graph memo cold ⇒ + // reconcile takes `try_reconcile_all_cache_hits`, which MUST register each hit's + // schedule keys (D0.2) or the completions below evict nothing and this test reds. super::index_arm_schedule_retention(&index, &rows); - super::resolve_entry_with_index(&index, &entry_a).expect("resolve a"); - super::resolve_entry_with_index(&index, &entry_b).expect("resolve b"); + super::resolve_entry_with_index(&index, &entry_a).expect("resolve a (all-hit)"); + super::resolve_entry_with_index(&index, &entry_b).expect("resolve b (all-hit)"); // Both closures reconciled → all three modules cached, and each entry's // assembled ResolvedGraph is memoized (the strong-Rc pin Fact #4 drops). @@ -7016,6 +7344,80 @@ mod schedule_retention_red_controls { let _ = std::fs::remove_dir_all(&dir); } + + /// RED (D0.4 retain-all baseline on a REAL index): with eviction DISABLED the pole + /// must retain EVERYTHING a completion would otherwise drop — the per-module caches + /// AND the assembled resolved graph. Before the D0.4 production fix the graph pin was + /// dropped UNCONDITIONALLY, so the "retain-all" baseline silently understated peak + /// retention (an invalid pre-M2 measurement pole). Installs a disabled retention + /// directly rather than racing the process-global `GUNBC_SCHEDULE_RETENTION_EVICT`. + #[test] + fn schedule_eviction_disabled_retains_resolved_graph_on_real_index() { + let dir = super::workspace_root() + .join("target") + .join(format!("sched_ret_probe_{}_retainall", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).expect("mkdir probe corpus"); + std::fs::write( + dir.join("shared_lib.dag"), + "module sched_ret_ra.shared_lib\n\ndata shared_val: Int = 7\n", + ) + .unwrap(); + std::fs::write( + dir.join("entry_a.dag"), + "module sched_ret_ra.entry_a\n\nimport sched_ret_ra.shared_lib { shared_val }\n\ndata a_val: Int = shared_val\n", + ) + .unwrap(); + let root = dir.to_string_lossy().to_string(); + let entry_a = dir.join("entry_a.dag").to_string_lossy().to_string(); + let index = super::build_multi_entry_index(&[root]); + + // Prewarm the per-module cache, then clear the graph memo (Ephemeral warming). + super::resolve_entry_with_index(&index, &entry_a).expect("prewarm a"); + super::clear_resolved_graph_memo_for_test(&index); + + // Arm with eviction DISABLED, keyed on the loader's exact closure (D0.3 authority). + let paths = super::load_sources_for_entry_with_pool(&index, &entry_a) + .unwrap() + .iter() + .map(|sf| super::workspace_relative_repo_path(&sf.path)) + .collect::>(); + super::install_schedule_retention_for_test(&index, vec![(entry_a.clone(), paths)], false); + + // Re-resolve so the all-hits probe records keys under the disabled retention, then + // snapshot the resident state right before completion. + super::resolve_entry_with_index(&index, &entry_a).expect("resolve a (all-hit)"); + let typed_before = super::typed_module_cache_len_for_test(&index); + let graphs_before = super::index_retention_snapshot(&index).resolved_graph_memo_entries; + assert!( + graphs_before >= 1, + "entry_a's graph memoized before completion" + ); + let subj_a = super::subject_digest_for_closure( + &super::load_sources_for_entry_with_pool(&index, &entry_a).unwrap(), + ); + + // Complete the entry: the disabled pole must drop NOTHING — not the per-module + // caches, not the resolved graph. + super::index_schedule_entry_completed(&index, &entry_a, Some(&subj_a)).expect("complete a"); + assert_eq!( + super::typed_module_cache_len_for_test(&index), + typed_before, + "retain-all pole: per-module cache must be unchanged" + ); + assert_eq!( + super::index_retention_snapshot(&index).resolved_graph_memo_entries, + graphs_before, + "retain-all pole: the resolved GRAPH must be retained too (D0.4) — before the \ + production fix a completion dropped it unconditionally" + ); + assert_eq!( + super::index_retention_snapshot(&index).schedule_evictions, + 0, + "disabled pole counts zero evictions" + ); + let _ = std::fs::remove_dir_all(&dir); + } } /// Interim width=1 floor-drain retention (v1-run-stability throughline, parent @@ -7601,8 +8003,27 @@ fn resolve_entry_with_parse_cache( let load_started = std::time::Instant::now(); let sources = load_sources_for_entry_with_pool(index, entry_file)?; resolve_stage_slot_add(|s| s.load += load_started.elapsed().as_nanos()); - resolved_graph_from_sources_with_index(index, sources, typecheck_gate, entry_file) - .map(|(graph, si, _compile_clean_diags)| (graph, si)) + resolved_graph_from_sources_with_index( + index, + sources, + typecheck_gate, + entry_file, + ResolvedGraphMemoShare::Memoize, + ) + .map(|(graph, si, _compile_clean_diags)| (graph, si)) +} + +/// Whether an assembled closure graph joins the in-process share tier +/// (`resolved_graph_memo`). Per-entry discovery resolves `Memoize` so a re-resolve of the +/// same subject is served by reference (the ReferenceTier). The compile-clean whole-tree +/// gate resolves `Ephemeral`: its aggregate graph strong-`Rc`-pins every `TypedModule` in +/// the tree and is never re-hit by discovery's smaller per-entry subjects, so memoizing it +/// is the 9.2GB-class resident-retention leak D0.1 removes (ci-two-tier §5). The per-module +/// `typed_module_cache` warming that IS the gate's purpose happens in reconcile regardless. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ResolvedGraphMemoShare { + Memoize, + Ephemeral, } fn compile_clean_diags_from_resolved_stages( @@ -7647,6 +8068,7 @@ fn resolved_graph_from_sources_with_index( sources: Vec>, typecheck_gate: ResolveTypecheckGate, phase_label: &str, + memo_share: ResolvedGraphMemoShare, ) -> Result< ( Rc, @@ -7868,15 +8290,22 @@ fn resolved_graph_from_sources_with_index( &ownership_diags, ); - // Install into the in-process share so same-subject re-resolves skip assembly. - index.resolved_graph_memo.borrow_mut().insert( - subject.clone(), - ( - typed.clone(), - source_indices.clone(), - compile_clean_diags.clone(), - ), - ); + // Install into the in-process share so same-subject re-resolves skip assembly — + // UNLESS this is an Ephemeral gate resolve (the compile-clean whole-tree gate): its + // aggregate graph strong-Rc-pins every TypedModule in the tree and is never re-hit by + // discovery's per-entry subjects, so memoizing it is the 9.2GB-class resident-retention + // leak D0.1 removes (ci-two-tier §5). Per-module typed-cache warming already happened + // above, in reconcile, and is unaffected. + if memo_share == ResolvedGraphMemoShare::Memoize { + index.resolved_graph_memo.borrow_mut().insert( + subject.clone(), + ( + typed.clone(), + source_indices.clone(), + compile_clean_diags.clone(), + ), + ); + } if let Some(cache_root) = resolved_graph_cache_root_from_env() { // A failed store write is a disclosed refusal, never a silent shrug — // the swallowed error hid that big closures never landed on disk (only @@ -8216,6 +8645,15 @@ fn try_reconcile_all_cache_hits( resolve_stage_slot_add(|s| s.assembly_probe += probe_started.elapsed().as_nanos()); return Ok(None); }; + // Record this confirmed hit's cache keys with the armed schedule retention + // (idempotent; no-op when unarmed) — the all-hits PROBE must register keys just + // like the slow reconcile loop at `reconcile_with_typed_cache`, else a prewarmed + // all-hit run (compile-clean warms the index, deliberately making all-hit the + // common case) arms retention that references NOTHING: completion then reports + // evictions while removing nothing (D0.2 retention-truth fix, ci-two-tier §5). + // Same key forms as the slow path — `decl_file` for the schedule refcount, the + // raw `span.file` for the parse/normalize/ownership/source-hash caches. + index_record_schedule_module(index, &decl_file, &typed_key, &resolved.module.span.file); note_interface_hash(&mut interface_hash_by_name, mod_name, &tc_result); results[slot] = Some(tc_result); progressed = true; @@ -8748,7 +9186,11 @@ fn reconcile_with_typed_cache( // Once-per-node receipt (§6.2): count only genuine computes (cache misses). bump_typecheck_compute_count(); if phase_profile::phase_profile_enabled() { - eprintln!("[typecheck-attribution] module={mod_name} start"); + let _ = TYPECHECK_ATTRIBUTION_CENSUS_MARKER; + eprintln!( + "{}", + render_typecheck_begin_line_mirror(&mod_name, typecheck_emoji()) + ); } let module_tc_started = std::time::Instant::now(); // Same-tree bare underlay for the module being typechecked @@ -8788,8 +9230,14 @@ fn reconcile_with_typed_cache( }); let module_tc_ms = module_tc_elapsed.as_millis(); if module_tc_ms >= 2_000 { + let _ = TYPECHECK_ATTRIBUTION_CENSUS_MARKER; eprintln!( - "[typecheck-attribution] module={mod_name} ms={module_tc_ms}" + "{}", + render_typecheck_concluded_line_mirror( + &mod_name, + module_tc_ms as u64, + typecheck_emoji(), + ) ); } let computed = index_insert_typed(index, typed_key.clone(), computed)?; @@ -9085,11 +9533,13 @@ fn install_effect_stream_policy(ctx: &v1_interpreter::InterpContext, verbose: bo // otherwise be silently defaulted, which is the fabricated-plausible-output the // funnel exists to avoid. Leave the policy uninstalled instead — the documented // fallback table is honest about being a fallback. - let (Some(ess), Some(esf), Some(efs), Some(eff)) = ( + let (Some(ess), Some(esf), Some(efs), Some(eff), Some(ods), Some(odf)) = ( disposition("expect_success_observed_success"), disposition("expect_success_observed_failure"), disposition("expect_failure_observed_success"), disposition("expect_failure_observed_failure"), + disposition("outcome_is_data_observed_success"), + disposition("outcome_is_data_observed_failure"), ) else { return; }; @@ -9097,7 +9547,7 @@ fn install_effect_stream_policy(ctx: &v1_interpreter::InterpContext, verbose: bo return; }; v1_interpreter::set_effect_stream_policy(InstalledEffectStreamPolicy { - dispositions: [ess, esf, efs, eff], + dispositions: [ess, esf, efs, eff, ods, odf], subject_line_guard: guard.to_string(), }); } @@ -10106,6 +10556,7 @@ pub fn handle_ci() { Some("dag/tools/gunbc_ci.dag".to_string()), false, false, + Vec::new(), ); } @@ -10190,7 +10641,14 @@ pub fn handle_run( entry_file: Option, claim_run: bool, ) { - handle_run_with_options(source_roots, function, entry_file, false, claim_run); + handle_run_with_options( + source_roots, + function, + entry_file, + false, + claim_run, + Vec::new(), + ); } /// adhoc-c328b166-bca residual-hunt instrumentation dump: printed periodically @@ -10249,18 +10707,55 @@ fn dump_residual_hunt_instrumentation() { } } +/// Parse repeated `--arg name=value` into the interpreter's named-argument +/// channel (`run_in_context_with_args`, v1_interpreter.rs:1564 — already the +/// channel `claim_executor` uses internally; `Commands::Run` simply never grew +/// the flag, which is why every parameter-shaped value reached `.dag` entries +/// through the process environment instead). +/// +/// Named-only by construction: a `.dag` entry's parameters are named, so +/// positional order across the CLI boundary would be an unchecked coincidence. +/// A missing `=` refuses (§5) rather than guessing a position. Values enter as +/// `Value::Str` — no coercion is fabricated here; a parameter wanting another +/// type is the typed-argument follow-on, not a silent conversion. +fn parse_run_args(raw: &[String]) -> Result, v1_interpreter::Value)>, String> { + raw.iter() + .map(|spec| match spec.split_once('=') { + Some((name, value)) if !name.is_empty() => Ok(( + Some(name.to_string()), + v1_interpreter::Value::Str(value.to_string()), + )), + Some(_) => Err(format!("--arg `{spec}`: empty parameter name before `=`")), + None => Err(format!( + "--arg `{spec}`: expected `name=value` (named arguments only)" + )), + }) + .collect() +} + pub fn handle_run_with_options( source_roots: Vec, function: String, entry_file: Option, dry_run: bool, claim_run: bool, + args: Vec, ) { if source_roots.is_empty() { eprintln!("error: provide at least one --source-root"); std::process::exit(1); } + // Refuse a malformed --arg before the compile, so the diagnostic is the + // first thing printed rather than the last. + let run_args = match parse_run_args(&args) { + Ok(parsed) => parsed, + Err(message) => { + eprintln!("error: {message}"); + std::process::exit(2); + } + }; + if claim_run && entry_file.is_none() { eprintln!( "error: --claim-run requires --entry (scoped import closure; \ @@ -10269,6 +10764,14 @@ pub fn handle_run_with_options( std::process::exit(1); } + // Same install as claim_executor / claim_batch: without it, host-effect traces + // fall back to Full and the merge-admission stamp (`gunbc run … merge_admission_stamp`) + // dumps every Ambient Begin/Done as emoji shell noise (operator live-log + // 2026-07-25). Install before any Wet eval so ShellTrace Suppressed-at-Normal + // collapses Ambient scaffolding; Anomaly Failed still surfaces via disposition. + install_output_policy(&source_roots); + install_group_syntax(&source_roots); + if let Ok(secs) = std::env::var("GUNBC_FLATTEN_SITE_DUMP_SECS") { if let Ok(secs) = secs.parse::() { std::thread::spawn(move || loop { @@ -10346,7 +10849,10 @@ pub fn handle_run_with_options( let ctx = v1_interpreter::InterpContext::new(graph, result.source_indices.clone(), execution_mode); v1_interpreter::with_active_context(&ctx, || { - let run_outcome = v1_interpreter::run_in_context(&ctx, &function, !claim_run); + // One path, not two: with an empty `--arg` list this is byte-identical + // to `run_in_context`, which passes the same empty slice. + let run_outcome = + v1_interpreter::run_in_context_with_args(&ctx, &function, &run_args, !claim_run); v1_interpreter::print_eval_recompute_trace(&ctx); match run_outcome { Ok(val) => { @@ -15235,6 +15741,12 @@ fn run_discovery_corpus_with_options_inner( // true (§6). One build covers the run; workers only read the process-wide memo. prime_witness_execution_legs(&index, rows.iter().map(|row| row.entry.as_str())); + // Arm the observation heartbeat's entry total at the same grain the drain walks + // (entry-groups), now that the roster is known — never a fabricated 0-of-0, and + // never earlier (batch-enter only had the opaque DiscoveryBatch runnable). + let discovery_entry_total = entry_row_groups(&rows).len() as u64; + heartbeat_feed_set_entry_total(discovery_entry_total); + let floor_color = floor_color_enabled(); let floor_stream = floor_stream_enabled(); return match width_policy { @@ -23725,6 +24237,26 @@ pub fn non_fold_residue_synthetic_unrostered_red_holds() -> bool { sites.contains(&site.to_string()) && !non_fold_residue_site_is_rostered(site) } +#[cfg(test)] +mod nfr_observation_roster_test { + use super::non_fold_residue_site_is_rostered; + + // Green-by-execution for the one observation-stack wildcard site + // (ci_hold_cause_text over SchedulerHold, merged via #7168): the roster now + // carries it, so the corpus nfr witness's unrostered count no longer counts it. + // Reds if the roster row's key drifts from the scan's `{rel}::{fn}` key, or if + // the hand edit malformed the frontier list (the reader panics on a bad list). + #[test] + fn observation_hold_cause_wildcard_is_rostered() { + assert!( + non_fold_residue_site_is_rostered( + "dag/gunbc/observation_ci_render.dag::ci_hold_cause_text" + ), + "the observation ci_hold_cause_text wildcard must be rostered after the fix" + ); + } +} + // ── Non-fold-residue census (DESIGN §6) ────────────────────────────────────────────────────────── // // Audits the corpus for `match` expressions whose scrutinee is a function parameter with a declared diff --git a/src/v1/stage0/src/main.rs b/src/v1/stage0/src/main.rs index 2dbaebbe721..daa6519f102 100644 --- a/src/v1/stage0/src/main.rs +++ b/src/v1/stage0/src/main.rs @@ -68,6 +68,10 @@ enum Commands { /// TestClaim / witness run: Bool false → exit 1; requires --entry #[arg(long)] claim_run: bool, + /// Named argument for the entry function, repeatable: `--arg name=value`. + /// Values enter as String; a missing `=` refuses rather than guessing. + #[arg(long = "arg")] + args: Vec, }, /// Apply a host's typed converge policy in-process @@ -630,8 +634,16 @@ fn main() { function, entry, claim_run, + args, } => { - cli_run::handle_run_with_options(source_roots, function, entry, cli.dry_run, claim_run); + cli_run::handle_run_with_options( + source_roots, + function, + entry, + cli.dry_run, + claim_run, + args, + ); } Commands::Converge { host } => { diff --git a/src/v1/stage0/src/memory_governor.rs b/src/v1/stage0/src/memory_governor.rs index 372f4f80b45..74385417115 100644 --- a/src/v1/stage0/src/memory_governor.rs +++ b/src/v1/stage0/src/memory_governor.rs @@ -45,6 +45,39 @@ use std::path::{Path, PathBuf}; use std::sync::Mutex; +/// Kept so `gunbc.observation_emit_census` roster hygiene cannot go stale after the +/// raw `[governor]` key=value shapes dissolve into the observation projection. +#[allow(dead_code)] +pub const GOVERNOR_CENSUS_MARKER: &str = "[governor]"; + +fn governor_emoji() -> bool { + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true") +} + +fn mirror_ci_tenths_text(tenths: u64) -> String { + format!("{}.{}", tenths / 10, tenths % 10) +} + +fn mirror_ci_human_bytes(bytes: u64) -> String { + let tenths = (bytes.saturating_mul(10)) / 1_073_741_824; + format!("{} GiB", mirror_ci_tenths_text(tenths)) +} + +fn mirror_ci_human_percent(bp: u64) -> String { + format!("{}%", mirror_ci_tenths_text(bp / 10)) +} + +fn render_governor_info_line(text: &str, emoji: bool) -> String { + let glyph = if emoji { "🕐" } else { "◷" }; + format!("{glyph} {text}") +} + +fn render_governor_done_line(text: &str, emoji: bool) -> String { + // Glyph discipline (operator live-log 2026-07-25): a receipt is *data*, not an + // outcome — StatusPulse, never the Done/success glyph. + render_governor_info_line(text, emoji) +} + /// Multiplicative-decrease divisor is 2 (halve), additive increase is +1 — classic AIMD. /// The remaining thresholds are POLICY (like TCP's), not measurements of any workload: /// they scale with the budget or are dimensionless, so no per-corpus constant returns. @@ -115,6 +148,8 @@ pub enum HoldReason { impl HoldReason { fn describe(&self) -> String { + // Internal diagnostic text retained for tests/debug; log projection goes through + // `render_governor_hold_line_mirror` (ci_hold_cause_text authority). match self { HoldReason::WindowFull { active, target } => { format!("window full (active={active} target={target})") @@ -144,6 +179,15 @@ impl HoldReason { ), } } + fn emit_hold_line(&self, old_target: usize, new_target: usize) -> String { + let _ = GOVERNOR_CENSUS_MARKER; + let base = render_governor_hold_line_mirror(self, governor_emoji()); + if old_target != new_target { + format!("{base} — target_width {old_target}→{new_target}") + } else { + base + } + } fn is_memory_creep(&self) -> bool { !matches!( self, @@ -155,6 +199,39 @@ impl HoldReason { } } +/// Mirror of `gunbc.observation_ci_render.ci_hold_cause_text` over the seed's HoldReason +/// (lockstep with SchedulerHold). Proven byte-equal to the seed oracle for the two +/// narrated arms (PsiPressure, CurrentHighWater); other variants share the model's +/// generic "blocked on scheduler admission" text. +pub fn mirror_ci_hold_cause_text(hold: &HoldReason) -> String { + match hold { + HoldReason::CurrentHighWater { + current, + high_water, + } => format!( + "blocked on the memory high-water line ({} of {})", + mirror_ci_human_bytes(*current), + mirror_ci_human_bytes(*high_water) + ), + HoldReason::PsiPressure { avg10 } => { + // avg10 is percent with one decimal; basis points = percent × 100. + let bp = (*avg10 * 100.0).round() as u64; + format!( + "blocked on memory reclaim (pressure {})", + mirror_ci_human_percent(bp) + ) + } + _ => "blocked on scheduler admission".to_string(), + } +} + +/// Mirror of `gunbc.observation_seed_render.seed_governor_hold_line` — +/// `ci_hold_cause_text ∘ StatusBlocked ∘ ci_render_line`. +pub fn render_governor_hold_line_mirror(hold: &HoldReason, emoji: bool) -> String { + let glyph = if emoji { "⏳" } else { "◷" }; + format!("{glyph} {}", mirror_ci_hold_cause_text(hold)) +} + #[derive(Debug, Clone, Copy, PartialEq)] pub enum AdmitDecision { /// A worker slot was granted. `forced_serial` marks the width-1 progress floor firing @@ -258,17 +335,16 @@ struct HardEvent { } impl HardEvent { - fn describe(&self) -> String { + fn emit_line(&self) -> String { + let _ = GOVERNOR_CENSUS_MARKER; + let glyph = if governor_emoji() { "⏳" } else { "◷" }; let cause = if self.budget_exceeded { - "memory.current exceeded the declared budget".to_string() + "blocked on the declared memory budget" } else { - format!( - "memory.events high +{} oom_kill +{}", - self.high_delta, self.oom_kill_delta - ) + "blocked on memory reclaim" }; format!( - "[governor] hard back-off: {cause} — target_width {}→{} (workers drain between units)", + "{glyph} {cause} — target_width {}→{}", self.old_target, self.new_target ) } @@ -665,18 +741,25 @@ impl MemoryGovernor { budget_source: source_label, max_width: max_width.max(1), }; + let _ = GOVERNOR_CENSUS_MARKER; eprintln!( - "[governor] adaptive width (AIMD): budget={} source={} max_width={} sensors={}", - limits - .budget_bytes - .map(|b| b.to_string()) - .unwrap_or_else(|| "unknown".into()), - limits.budget_source, - limits.max_width, - sensor_dir - .as_ref() - .map(|d| d.display().to_string()) - .unwrap_or_else(|| "none (creep checks inert)".into()), + "{}", + render_governor_info_line( + &format!( + "governor adaptive width — budget={} source={} max_width={} sensors={}", + limits + .budget_bytes + .map(|b| b.to_string()) + .unwrap_or_else(|| "unknown".into()), + limits.budget_source, + limits.max_width, + sensor_dir + .as_ref() + .map(|d| d.display().to_string()) + .unwrap_or_else(|| "none (creep checks inert)".into()), + ), + governor_emoji() + ) ); MemoryGovernor { limits, @@ -695,16 +778,11 @@ impl MemoryGovernor { let new_target = core.target_width; drop(core); if let Some(h) = hard { - eprintln!("{}", h.describe()); + eprintln!("{}", h.emit_line()); } if let AdmitDecision::Hold(reason) = &decision { if reason.is_memory_creep() && !was_holding { - eprintln!( - "[governor] creep back-off: {} — target_width {}→{} (admissions held; workers drain between units)", - reason.describe(), - old_target, - new_target - ); + eprintln!("{}", reason.emit_hold_line(old_target, new_target)); } } if let AdmitDecision::Admit { @@ -712,7 +790,11 @@ impl MemoryGovernor { } = &decision { eprintln!( - "[governor] progress floor: signals hot but zero workers active — admitting one (counted forced_serial)" + "{}", + render_governor_info_line( + "governor progress floor — signals hot but zero workers active; admitting one (counted forced_serial)", + governor_emoji() + ) ); } decision @@ -749,12 +831,11 @@ impl MemoryGovernor { let (hard, creep_backoff) = note_completion(&mut core, &sig, &self.limits, kind); drop(core); if let Some(h) = hard { - eprintln!("{}", h.describe()); + eprintln!("{}", h.emit_line()); } if let Some((old, new)) = creep_backoff { - eprintln!( - "[governor] creep back-off (observed at completion): target_width {old}→{new} (admissions held; workers drain between units)" - ); + let glyph = if governor_emoji() { "⏳" } else { "◷" }; + eprintln!("{glyph} blocked on scheduler admission — target_width {old}→{new}"); } } @@ -768,7 +849,14 @@ impl MemoryGovernor { if !had_share { if let Some(s) = share { eprintln!( - "[governor] measured worker share: {s} bytes (first slot's cost over the pool baseline) — headroom gate armed" + "{}", + render_governor_info_line( + &format!( + "governor measured worker share — {} (first slot cost over pool baseline); headroom gate armed", + mirror_ci_human_bytes(s) + ), + governor_emoji() + ) ); } } @@ -803,32 +891,36 @@ impl MemoryGovernor { /// The end-of-run receipt: the counted degradations (§5 — observable, prioritizable). pub fn receipt_line(&self) -> String { + let _ = GOVERNOR_CENSUS_MARKER; let core = self.core.lock().unwrap(); - format!( - "[governor] receipt: budget={} source={} max_width_reached={} admissions={} \ + render_governor_done_line( + &format!( + "governor receipt — budget={} source={} max_width_reached={} admissions={} \ width_growths={} creep_backoffs={} pacing_holds={} headroom_holds={} \ ceiling_holds={} hard_backoffs={} budget_exceeded={} forced_serial={} \ peak_current={}", - self.limits - .budget_bytes - .map(|b| b.to_string()) - .unwrap_or_else(|| "unknown".into()), - self.limits.budget_source, - core.max_width_reached, - core.admissions, - core.width_growths, - core.creep_backoffs, - core.pacing_holds, - core.headroom_holds, - core.ceiling_holds, - core.hard_backoffs, - core.budget_exceeded_backoffs, - core.forced_serial_admissions, - if core.peak_current_bytes == 0 { - "unreadable".to_string() - } else { - core.peak_current_bytes.to_string() - }, + self.limits + .budget_bytes + .map(|b| b.to_string()) + .unwrap_or_else(|| "unknown".into()), + self.limits.budget_source, + core.max_width_reached, + core.admissions, + core.width_growths, + core.creep_backoffs, + core.pacing_holds, + core.headroom_holds, + core.ceiling_holds, + core.hard_backoffs, + core.budget_exceeded_backoffs, + core.forced_serial_admissions, + if core.peak_current_bytes == 0 { + "unreadable".to_string() + } else { + core.peak_current_bytes.to_string() + }, + ), + governor_emoji(), ) } } diff --git a/src/v1/stage0/src/std_measure.rs b/src/v1/stage0/src/std_measure.rs index 907ad33fcec..21c7bff8faa 100644 --- a/src/v1/stage0/src/std_measure.rs +++ b/src/v1/stage0/src/std_measure.rs @@ -99,6 +99,10 @@ pub fn kibi_factor() -> Nat { 1024 } +pub fn milliseconds_per_second() -> Nat { + 1000 +} + pub fn seconds_per_minute() -> Nat { 60 } diff --git a/src/v1/stage0/src/v1_compiler_emit_rust.rs b/src/v1/stage0/src/v1_compiler_emit_rust.rs index e4bdbc1a7d7..a25f1ffcd5f 100644 --- a/src/v1/stage0/src/v1_compiler_emit_rust.rs +++ b/src/v1/stage0/src/v1_compiler_emit_rust.rs @@ -29955,6 +29955,15 @@ pub fn emit_subcommand_enum( "#[arg(long)]\n".to_string(), make_indent((depth.clone() + 2)), "claim_run: bool,\n".to_string(), + make_indent((depth.clone() + 2)), + "/// Named argument for the entry function, repeatable: `--arg name=value`.\n" + .to_string(), + make_indent((depth.clone() + 2)), + "/// Values enter as String; a missing `=` refuses rather than guessing.\n".to_string(), + make_indent((depth.clone() + 2)), + "#[arg(long = \"arg\")]\n".to_string(), + make_indent((depth.clone() + 2)), + "args: Vec,\n".to_string(), make_indent((depth.clone() + 1)), "},".to_string(), ]); @@ -30240,7 +30249,7 @@ pub fn emit_compile_match_arm(crate_name: String) -> String { } pub fn emit_run_match_arm(crate_name: String) -> String { - v1_rt::concat(v1_rt::concat("\n Commands::Run { source_roots, function, entry, claim_run } => {\n".to_string(), " cli_run::handle_run_with_options(source_roots, function, entry, cli.dry_run, claim_run);\n".to_string()), " },".to_string()) + v1_rt::concat(v1_rt::concat("\n Commands::Run { source_roots, function, entry, claim_run, args } => {\n".to_string(), " cli_run::handle_run_with_options(source_roots, function, entry, cli.dry_run, claim_run, args);\n".to_string()), " },".to_string()) } pub fn emit_main_pipeline_fns(crate_name: String) -> String { diff --git a/src/v1/stage0/src/v1_compiler_runtime_rust.rs b/src/v1/stage0/src/v1_compiler_runtime_rust.rs index d4bedb13f9b..6eb2851e142 100644 --- a/src/v1/stage0/src/v1_compiler_runtime_rust.rs +++ b/src/v1/stage0/src/v1_compiler_runtime_rust.rs @@ -92,7 +92,7 @@ pub fn rt_filesystem() -> String { } pub fn rt_hash_ops() -> String { - v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat("const FNV1A64_OFFSET: u64 = 0xcbf29ce484222325;\n".to_string(), "const FNV1A64_PRIME: u64 = 0x100000001b3;\n\n".to_string()), "fn fnv1a64(bytes: &[u8]) -> u64 {\n".to_string()), " let mut hash = FNV1A64_OFFSET;\n".to_string()), " for b in bytes {\n".to_string()), " hash ^= *b as u64;\n".to_string()), " hash = hash.wrapping_mul(FNV1A64_PRIME);\n".to_string()), " }\n".to_string()), " hash\n".to_string()), "}\n\n".to_string()), "pub type Hash = String;\n\n".to_string()), "const HASH_DIGEST_LEN: usize = 16;\n\n".to_string()), "pub fn is_hash_digest(s: &str) -> bool {\n".to_string()), " s.len() == HASH_DIGEST_LEN && s.bytes().all(|b| b.is_ascii_hexdigit())\n".to_string()), "}\n\n".to_string()), "fn expect_hash_digest(s: &str, arg: &str) {\n".to_string()), " if !is_hash_digest(s) {\n".to_string()), " panic!(\"{} must be a 16-char hex Hash digest\", arg);\n".to_string()), " }\n".to_string()), "}\n\n".to_string()), "/// Stage-boundary trace mark — the v1-seed interim realization of the v2 per-RealizedStep\n".to_string()), "/// CostAccount (std.realization_measurement). One stderr line per mark; consecutive marks\n".to_string()), "/// define the segments of a natural Gantt read directly off any run log.\n".to_string()), "///\n".to_string()), "/// **Dissolution trigger (DESIGN §6):** delete this fn, the `trace_mark` registry row in\n".to_string()), "/// `04_method.dag` (+ hand-synced twin `v1_compiler_infer_method.rs`), the nine\n".to_string()), "/// `trace_mark(...)` marks in `compile.dag` (+ hand-synced `v1_compiler_compile.rs`), and\n".to_string()), "/// the interpreter arm in `v1_interpreter.rs` when realization_measurement_loop **Phase 0**\n".to_string()), "/// (`docs/plans/realization-measurement-loop.md`) lands a `.dag` `PerformanceReceipt`\n".to_string()), "/// per-stage carrier that a floor witness consumes by execution (the same retirement event\n".to_string()), "/// as `phase_profile.rs` / `GUNBC_FLOOR_GANTT`, per `docs/plans/ci-floor-fractal-gantt.md`\n".to_string()), "/// § dissolution). Receipt = that witness green with these marks deleted and stage walls\n".to_string()), "/// still attributable from the model path.\n".to_string()), "pub fn trace_mark(label: String) {\n".to_string()), " use std::sync::OnceLock;\n".to_string()), " use std::time::Instant;\n".to_string()), " static TRACE_T0: OnceLock = OnceLock::new();\n".to_string()), " let ms = TRACE_T0.get_or_init(Instant::now).elapsed().as_millis();\n".to_string()), " let mut rss_mib = String::from(\"absent\");\n".to_string()), " if let Ok(status) = std::fs::read_to_string(\"/proc/self/status\") {\n".to_string()), " for line in status.lines() {\n".to_string()), " if let Some(rest) = line.strip_prefix(\"VmRSS:\") {\n".to_string()), " if let Some(kib) = rest\n".to_string()), " .split_whitespace()\n".to_string()), " .next()\n".to_string()), " .and_then(|k| k.parse::().ok())\n".to_string()), " {\n".to_string()), " rss_mib = (kib / 1024).to_string();\n".to_string()), " }\n".to_string()), " }\n".to_string()), " }\n".to_string()), " }\n".to_string()), " eprintln!(\"[gantt] {} t_ms={} rss_mib={}\", label, ms, rss_mib);\n".to_string()), "}\n\n".to_string()), "/// Content hash over raw bytes — the byte-level single authority. `atom_identity_hash`\n".to_string()), "/// is the `String` projection of this. Use this directly for arbitrary binary content\n".to_string()), "/// (e.g. an executable or serialized payload): routing bytes through `String`/\n".to_string()), "/// `from_utf8_lossy` first collapses every invalid UTF-8 sequence to U+FFFD, so distinct\n".to_string()), "/// byte sequences would hash equal — a §5 silent-collision fail-open for content-addressing.\n".to_string()), "pub fn bytes_identity_hash(bytes: &[u8]) -> Hash {\n".to_string()), " format!(\"{:016x}\", fnv1a64(bytes))\n".to_string()), "}\n\n".to_string()), "pub fn atom_identity_hash(s: String) -> Hash {\n".to_string()), " bytes_identity_hash(s.as_bytes())\n".to_string()), "}\n\n".to_string()), "pub fn hash_combine(a: Hash, b: Hash) -> Hash {\n".to_string()), " expect_hash_digest(&a, \"a\");\n".to_string()), " expect_hash_digest(&b, \"b\");\n".to_string()), " let mut bytes = a.into_bytes();\n".to_string()), " bytes.push(0);\n".to_string()), " bytes.extend_from_slice(b.as_bytes());\n".to_string()), " format!(\"{:016x}\", fnv1a64(&bytes))\n".to_string()), "}\n\n".to_string()) + v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat(v1_rt::concat("const FNV1A64_OFFSET: u64 = 0xcbf29ce484222325;\n".to_string(), "const FNV1A64_PRIME: u64 = 0x100000001b3;\n".to_string()), "\n".to_string()), "fn fnv1a64(bytes: &[u8]) -> u64 {\n".to_string()), " let mut hash = FNV1A64_OFFSET;\n".to_string()), " for b in bytes {\n".to_string()), " hash ^= *b as u64;\n".to_string()), " hash = hash.wrapping_mul(FNV1A64_PRIME);\n".to_string()), " }\n".to_string()), " hash\n".to_string()), "}\n".to_string()), "\n".to_string()), "pub type Hash = String;\n".to_string()), "\n".to_string()), "const HASH_DIGEST_LEN: usize = 16;\n".to_string()), "\n".to_string()), "pub fn is_hash_digest(s: &str) -> bool {\n".to_string()), " s.len() == HASH_DIGEST_LEN && s.bytes().all(|b| b.is_ascii_hexdigit())\n".to_string()), "}\n".to_string()), "\n".to_string()), "fn expect_hash_digest(s: &str, arg: &str) {\n".to_string()), " if !is_hash_digest(s) {\n".to_string()), " panic!(\"{} must be a 16-char hex Hash digest\", arg);\n".to_string()), " }\n".to_string()), "}\n".to_string()), "\n".to_string()), "/// Kept so `gunbc.observation_emit_census` roster hygiene cannot go stale after the\n".to_string()), "/// raw gantt key=value shape (process-absolute millis and rss-mib fields) dissolves into the observation projection.\n".to_string()), "#[allow(dead_code)]\n".to_string()), "pub const GANTT_CENSUS_MARKER: &str = \"[gantt]\";\n".to_string()), "\n".to_string()), "/// Mirror of `gunbc.observation_ci_render.ci_minute_switch_seconds`.\n".to_string()), "const OBS_MINUTE_SWITCH_SECONDS: u64 = 90;\n".to_string()), "\n".to_string()), "fn obs_human_duration(ms: u64) -> String {\n".to_string()), " if ms < 1_000 {\n".to_string()), " format!(\"{ms}ms\")\n".to_string()), " } else if ms < OBS_MINUTE_SWITCH_SECONDS * 1_000 {\n".to_string()), " format!(\"{} seconds\", ms / 1_000)\n".to_string()), " } else {\n".to_string()), " format!(\"{} minutes\", ms / 60_000)\n".to_string()), " }\n".to_string()), "}\n".to_string()), "\n".to_string()), "fn obs_phase_emoji() -> bool {\n".to_string()), " std::env::var(\"GITHUB_ACTIONS\").as_deref() == Ok(\"true\")\n".to_string()), "}\n".to_string()), "\n".to_string()), "/// Pure Rust mirror of `gunbc.observation_seed_render.phase_begin_line` —\n".to_string()), "/// `ci_event_line(Begin) ∘ ci_render_line`. Justified divergence from the\n".to_string()), "/// interpreter seed boundary used by floor prelude marks: `trace_mark` runs\n".to_string()), "/// inside compile itself (hand-synced path and interpreter intrinsic), and\n".to_string()), "/// calling back into the interpreter to render would recurse. Proven\n".to_string()), "/// byte-equal to the `.dag` oracle by the seed RED.\n".to_string()), "pub fn render_phase_begin_line_mirror(phase: &str, emoji: bool) -> String {\n".to_string()), " let glyph = if emoji { \"🔄\" } else { \"◐\" };\n".to_string()), " format!(\"{glyph} started {phase}\")\n".to_string()), "}\n".to_string()), "\n".to_string()), "/// Pure Rust mirror of `gunbc.observation_seed_render.phase_concluded_line` —\n".to_string()), "/// `ci_event_line(Concluded{Done}) ∘ ci_render_line`. Same justified-divergence\n".to_string()), "/// note as `render_phase_begin_line_mirror`.\n".to_string()), "pub fn render_phase_concluded_line_mirror(phase: &str, elapsed_ms: u64, emoji: bool) -> String {\n".to_string()), " let glyph = if emoji { \"✅\" } else { \"✓\" };\n".to_string()), " format!(\"{glyph} {phase} done in {}\", obs_human_duration(elapsed_ms))\n".to_string()), "}\n".to_string()), "\n".to_string()), "/// Stage-boundary trace mark — the v1-seed interim realization of the v2 per-RealizedStep\n".to_string()), "/// CostAccount (std.realization_measurement). Wired (gantt flip): projects Begin/Concluded\n".to_string()), "/// on a PhaseSegment via the observation mirrors of `ci_event_line`. Segment wall on\n".to_string()), "/// Concluded (matched `.begin`→`.done`, else delta since last mark) — never the old\n".to_string()), "/// process-absolute millis field. RSS stays on heartbeat/measurement (`ci_event_line` does\n".to_string()), "/// not project `event.rss` — explicit divergence from the deleted rss-mib field).\n".to_string()), "///\n".to_string()), "/// **Dissolution trigger (DESIGN §6):** delete this fn, the `trace_mark` registry row in\n".to_string()), "/// `04_method.dag` (+ hand-synced twin `v1_compiler_infer_method.rs`), the nine\n".to_string()), "/// `trace_mark(...)` marks in `compile.dag` (+ hand-synced `v1_compiler_compile.rs`), and\n".to_string()), "/// the interpreter arm in `v1_interpreter.rs` when realization_measurement_loop **Phase 0**\n".to_string()), "/// (`docs/plans/realization-measurement-loop.md`) lands a `.dag` `PerformanceReceipt`\n".to_string()), "/// per-stage carrier that a floor witness consumes by execution (the same retirement event\n".to_string()), "/// as `phase_profile.rs` / `GUNBC_FLOOR_GANTT`, per `docs/plans/ci-floor-fractal-gantt.md`\n".to_string()), "/// § dissolution). Receipt = that witness green with these marks deleted and stage walls\n".to_string()), "/// still attributable from the model path.\n".to_string()), "pub fn trace_mark(label: String) {\n".to_string()), " use std::sync::{Mutex, OnceLock};\n".to_string()), " use std::time::Instant;\n".to_string()), " static TRACE_T0: OnceLock = OnceLock::new();\n".to_string()), " static LAST_MARK: OnceLock> = OnceLock::new();\n".to_string()), " static OPENS: OnceLock>> = OnceLock::new();\n".to_string()), " let t0 = TRACE_T0.get_or_init(Instant::now);\n".to_string()), " let last = LAST_MARK.get_or_init(|| Mutex::new(*t0));\n".to_string()), " let opens = OPENS.get_or_init(|| Mutex::new(std::collections::HashMap::new()));\n".to_string()), " let emoji = obs_phase_emoji();\n".to_string()), " let now = Instant::now();\n".to_string()), " let _ = GANTT_CENSUS_MARKER;\n".to_string()), " if let Some(phase) = label.strip_suffix(\".begin\") {\n".to_string()), " if let Ok(mut map) = opens.lock() {\n".to_string()), " map.insert(phase.to_string(), now);\n".to_string()), " }\n".to_string()), " if let Ok(mut l) = last.lock() {\n".to_string()), " *l = now;\n".to_string()), " }\n".to_string()), " eprintln!(\"{}\", render_phase_begin_line_mirror(phase, emoji));\n".to_string()), " } else if let Some(phase) = label.strip_suffix(\".done\") {\n".to_string()), " let elapsed_ms = {\n".to_string()), " let started = opens.lock().ok().and_then(|mut m| m.remove(phase));\n".to_string()), " match started {\n".to_string()), " Some(t) => now.saturating_duration_since(t).as_millis() as u64,\n".to_string()), " None => {\n".to_string()), " let prev = last.lock().map(|l| *l).unwrap_or(*t0);\n".to_string()), " now.saturating_duration_since(prev).as_millis() as u64\n".to_string()), " }\n".to_string()), " }\n".to_string()), " };\n".to_string()), " if let Ok(mut l) = last.lock() {\n".to_string()), " *l = now;\n".to_string()), " }\n".to_string()), " eprintln!(\n".to_string()), " \"{}\",\n".to_string()), " render_phase_concluded_line_mirror(phase, elapsed_ms, emoji)\n".to_string()), " );\n".to_string()), " } else {\n".to_string()), " let prev = last.lock().map(|l| *l).unwrap_or(*t0);\n".to_string()), " let elapsed_ms = now.saturating_duration_since(prev).as_millis() as u64;\n".to_string()), " if let Ok(mut l) = last.lock() {\n".to_string()), " *l = now;\n".to_string()), " }\n".to_string()), " eprintln!(\n".to_string()), " \"{}\",\n".to_string()), " render_phase_concluded_line_mirror(&label, elapsed_ms, emoji)\n".to_string()), " );\n".to_string()), " }\n".to_string()), "}\n".to_string()), "\n".to_string()), "/// Content hash over raw bytes — the byte-level single authority. `atom_identity_hash`\n".to_string()), "/// is the `String` projection of this. Use this directly for arbitrary binary content\n".to_string()), "/// (e.g. an executable or serialized payload): routing bytes through `String`/\n".to_string()), "/// `from_utf8_lossy` first collapses every invalid UTF-8 sequence to U+FFFD, so distinct\n".to_string()), "/// byte sequences would hash equal — a §5 silent-collision fail-open for content-addressing.\n".to_string()), "pub fn bytes_identity_hash(bytes: &[u8]) -> Hash {\n".to_string()), " format!(\"{:016x}\", fnv1a64(bytes))\n".to_string()), "}\n".to_string()), "\n".to_string()), "pub fn atom_identity_hash(s: String) -> Hash {\n".to_string()), " bytes_identity_hash(s.as_bytes())\n".to_string()), "}\n".to_string()), "\n".to_string()), "pub fn hash_combine(a: Hash, b: Hash) -> Hash {\n".to_string()), " expect_hash_digest(&a, \"a\");\n".to_string()), " expect_hash_digest(&b, \"b\");\n".to_string()), " let mut bytes = a.into_bytes();\n".to_string()), " bytes.push(0);\n".to_string()), " bytes.extend_from_slice(b.as_bytes());\n".to_string()), " format!(\"{:016x}\", fnv1a64(&bytes))\n".to_string()), "}\n".to_string()), "\n".to_string()) } pub fn rt_resolution_silent_pick_telemetry() -> String { diff --git a/src/v1/stage0/src/v1_compiler_tokenize.rs b/src/v1/stage0/src/v1_compiler_tokenize.rs index 5279b7740c4..59843b4d589 100644 --- a/src/v1/stage0/src/v1_compiler_tokenize.rs +++ b/src/v1/stage0/src/v1_compiler_tokenize.rs @@ -834,6 +834,53 @@ pub fn process_escapes(raw: String) -> String { process_escapes_loop(raw.clone(), 0, Rc::new(vec![])) } +pub fn hex_escape_note() -> String { + thread_local! { + static CACHED: String = { + "\\xNN decodes to the byte NN names. Before this arm every \\x fell through to the unknown-escape branch below, which preserves the backslash literally — so every ANSI code authored in .dag (extdeps.render.ansi.csi_esc, extdeps.render.terminal's reset, gunbc.ci_render's red/reset) emitted the six characters backslash-x-1-b instead of ESC, and no colour this repo authored had ever rendered. The witness that was supposed to catch it compared one undecoded literal against another, so it agreed with itself and could not fail (DESIGN §5 — a check satisfied by editing the declaration while the realization lies).".to_string() + }; + } + CACHED.with(|c: &String| c.clone()) +} + +pub fn unknown_escape_passthrough_frontier() -> String { + thread_local! { + static CACHED: String = { + "🟡 dissolve-on: tokenizer_unknown_escape_strict_close (opened 2026-07-25) — the else-arm below resolves an UNRECOGNIZED escape to concat(backslash, next), so \\s silently means backslash-s and is indistinguishable from a typo'd escape. That is a closed-vocabulary violation (DESIGN §4: the vocabulary is closed; §5: a failure arm refuses, never widens) and it is knowingly RETAINED, not accepted: refusing today would red 142 occurrences across 38 files (textual census 2026-07-25 — grep over .dag, so an upper bound, and DESIGN's own caution about grep census applies), overwhelmingly regex fragments (\\' 28, \\. 19, \\| 13, \\/ 12, \\B 10) that intend the backslash to survive. NOT the four sites a \\x-only count suggests. DISSOLVES WHEN a raw-string form lands to carry regex fragments, the 142 occurrences migrate onto it, and this arm becomes a typed located parse refusal (the Rust rule: unknown escape is an error, a literal backslash is spelled \\\\). Because that is a PARSE-SEMANTICS change over a tokenizer every lane's probes compile through, the strict close lands with a corpus-wide compile receipt, never the site migration alone.".to_string() + }; + } + CACHED.with(|c: &String| c.clone()) +} + +pub fn hex_digit_value(ch: String) -> Option { + { + let cp = v1_rt::code_point(ch.clone()); + if ((cp.clone() >= 48) && (cp.clone() <= 57)) { + Some((cp.clone() - 48)) + } else { + if ((cp.clone() >= 97) && (cp.clone() <= 102)) { + Some((cp.clone() - 87)) + } else { + if ((cp.clone() >= 65) && (cp.clone() <= 70)) { + Some((cp.clone() - 55)) + } else { + None + } + } + } + } +} + +pub fn hex_escape_char(hi: String, lo: String) -> Option { + match hex_digit_value(hi.clone()) { + Some(h) => match hex_digit_value(lo.clone()) { + Some(l) => Some(v1_rt::from_code_point(((h.clone() * 16) + l.clone()))), + None => None, + }, + None => None, + } +} + pub fn process_escapes_loop(mut source: String, mut pos: i64, mut acc: Rc>) -> String { loop { if (pos.clone() >= v1_rt::string_length(&source)) { @@ -844,37 +891,64 @@ pub fn process_escapes_loop(mut source: String, mut pos: i64, mut acc: Rc { + let __tco_0 = (pos + 4); + let __tco_1 = v1_rt::rc_list_push(acc, decoded.clone()); + pos = __tco_0; + acc = __tco_1; + continue; + } + None => { + let __tco_0 = (pos + 2); + let __tco_1 = v1_rt::rc_list_push( + acc, + v1_rt::concat("\\".to_string(), next.clone()), + ); + pos = __tco_0; + acc = __tco_1; + continue; + } + } } else { - if (next.clone() == "\\".to_string()) { - "\\".to_string() + let resolved = if (next.clone() == "\"".to_string()) { + "\"".to_string() } else { - if (next.clone() == "n".to_string()) { - "\n".to_string() + if (next.clone() == "\\".to_string()) { + "\\".to_string() } else { - if (next.clone() == "t".to_string()) { - "\t".to_string() + if (next.clone() == "n".to_string()) { + "\n".to_string() } else { - if (next.clone() == "{".to_string()) { - "{".to_string() + if (next.clone() == "t".to_string()) { + "\t".to_string() } else { - if (next.clone() == "}".to_string()) { - "}".to_string() + if (next.clone() == "{".to_string()) { + "{".to_string() } else { - v1_rt::concat("\\".to_string(), next.clone()) + if (next.clone() == "}".to_string()) { + "}".to_string() + } else { + v1_rt::concat("\\".to_string(), next.clone()) + } } } } } + }; + { + let __tco_0 = (pos + 2); + let __tco_1 = v1_rt::rc_list_push(acc, resolved.clone()); + pos = __tco_0; + acc = __tco_1; + continue; } - }; - { - let __tco_0 = (pos + 2); - let __tco_1 = v1_rt::rc_list_push(acc, resolved.clone()); - pos = __tco_0; - acc = __tco_1; - continue; } } else { { diff --git a/src/v1/stage0/src/v1_interpreter.rs b/src/v1/stage0/src/v1_interpreter.rs index 35ce3f74ae1..c3441a8b875 100644 --- a/src/v1/stage0/src/v1_interpreter.rs +++ b/src/v1/stage0/src/v1_interpreter.rs @@ -5,6 +5,7 @@ use std::collections::BTreeSet; use std::fmt; use std::hash::{Hash, Hasher}; use std::rc::Rc; +use std::sync::atomic::{AtomicBool, Ordering}; use std::time::Instant; use im::HashMap as HamtMap; @@ -4161,15 +4162,28 @@ fn eval_method_call(node: &Rc, env: &Rc, ctx: &InterpContext) -> Inte if let Some(MethodSemantics::ServiceMethodSemantics { service_name, .. }) = semantics.as_deref() { let extra_args = method_arg_nodes(node.clone()); - let named_args: Vec<(Option, Value)> = extra_args - .iter() - .map(|a| { - let name = arg_name_at(a.clone(), ctx.si()); - let val = eval_expr(&arg_value(a.clone()), env, ctx)?; - Ok((name, val)) - }) - .collect::>()?; - return eval_service_call(service_name, &method_name, &named_args, env, ctx); + // The declared-expectation edge is read off THIS node — the call site — and + // partitioned out before anything becomes a bound param. See + // EFFECT_EXPECTATION_ARG for why the call node is the only correct home. + let mut named_args: Vec<(Option, Value)> = Vec::new(); + let mut declared_expectation: Option = None; + for a in extra_args.iter() { + let name = arg_name_at(a.clone(), ctx.si()); + let val = eval_expr(&arg_value(a.clone()), env, ctx)?; + if name.as_deref() == Some(EFFECT_EXPECTATION_ARG) { + declared_expectation = Some(expectation_from_declared_arg(&val)?); + continue; + } + named_args.push((name, val)); + } + return eval_service_call( + service_name, + &method_name, + &named_args, + env, + ctx, + declared_expectation.unwrap_or(ExpectedOutcome::ExpectSuccess), + ); } let receiver_val = eval_expr(&method_receiver(node.clone()), env, ctx)?; @@ -5108,7 +5122,18 @@ fn wet_service_call( msg: format!("no transport for service {}", key), })?; let param_env = build_service_param_env(op_node, args, env, ctx)?; - dispatch_service_wet(service_node, op_node, transport, ¶m_env, ctx) + // No call node here: this is the interpreter's own internal seam (Clock.UnixSecs + // and friends), not a .dag call site, so it declares the default explicitly + // rather than inheriting one. + dispatch_service_wet( + service_node, + op_node, + transport, + ¶m_env, + ctx, + &key, + ExpectedOutcome::ExpectSuccess, + ) } fn unix_secs_from_clock_value( @@ -5145,14 +5170,10 @@ fn realize_clock_unix_secs_transport() -> Result Option { let s = std::env::var(name).ok()?.trim().to_string(); if s.is_empty() { @@ -5165,7 +5186,14 @@ fn wet_env_var(name: &str) -> Option { fn resolve_env_var_token(ctx: &InterpContext, var_name: &str) -> Option { if ctx.service_ops.contains_key("shell.Env.Get") { let args = [(Some("name".to_string()), Value::Str(var_name.to_string()))]; - match eval_service_call("shell.Env", "Get", &args, &Env::empty(), ctx) { + match eval_service_call( + "shell.Env", + "Get", + &args, + &Env::empty(), + ctx, + ExpectedOutcome::ExpectSuccess, + ) { Ok(Value::Record { fields, .. }) => ctx.field(&fields, "value").and_then(|v| match v { Value::Str(s) if !s.is_empty() => Some(s.clone()), _ => None, @@ -5235,6 +5263,7 @@ fn eval_service_call( args: &[(Option, Value)], env: &Rc, ctx: &InterpContext, + expected: ExpectedOutcome, ) -> InterpResult { ctx.effect_dispatch_count .set(ctx.effect_dispatch_count.get().wrapping_add(1)); @@ -5290,7 +5319,15 @@ fn eval_service_call( .map(|requested| hermetic_checkout_read_disposition(&requested).is_ok()) .unwrap_or(false); if confirmed_checkout_input { - return dispatch_service_wet(service_node, op_node, transport, ¶m_env, ctx); + return dispatch_service_wet( + service_node, + op_node, + transport, + ¶m_env, + ctx, + &key, + expected, + ); } } let published = ctx.published_mock_keys()?; @@ -5335,7 +5372,15 @@ fn eval_service_call( return eval_mock_response(op_node, ctx); } - let result = dispatch_service_wet(service_node, op_node, transport, ¶m_env, ctx)?; + let result = dispatch_service_wet( + service_node, + op_node, + transport, + ¶m_env, + ctx, + &key, + expected, + )?; if ctx.execution_mode.is_record() { let store = ctx @@ -5364,9 +5409,21 @@ fn dispatch_service_wet( transport: &Rc, param_env: &Rc, ctx: &InterpContext, + intent: &str, + expected: ExpectedOutcome, ) -> InterpResult { + // Local Env.Get: reading THIS process's own environment is not a host effect + // (shell-to-dag residual census §0b / DESIGN §3(b)). `printenv` was the wrong + // single hardwired transport — unset vars exited 1 and, under ExpectSuccess, + // every optional floor_diff_observe injection (GUNBC_CI_DIFF_*) painted Anomaly + // Failed lines (operator live-log 2026-07-25). Native handler; shell printenv + // remains the remote-target realization. + if intent == "shell.Env.Get" { + return dispatch_env_get_native(op_node, param_env, ctx); + } + if is_shell_transport(transport.clone()) { - let result = dispatch_shell(transport, param_env, ctx)?; + let result = dispatch_shell(transport, param_env, ctx, intent, expected)?; return map_shell_outputs(&result, op_node, ctx); } @@ -5378,6 +5435,37 @@ fn dispatch_service_wet( dispatch_rest(service_node, op_node, transport, param_env, ctx) } +/// Native realization of `shell.Env.Get` for OnTarget locality — same Absent/Present +/// semantics as printenv (unset/empty → Null optional; value trimmed), with no +/// ObservationEvent and no child process. +fn dispatch_env_get_native( + op_node: &Rc, + param_env: &Rc, + ctx: &InterpContext, +) -> InterpResult { + let name = match param_env.lookup(ctx.sym("name")) { + Some(Value::Str(s)) => s.clone(), + Some(other) => { + return Err(InterpError::TypeError { + msg: format!("shell.Env.Get name must be String, got {other}"), + }); + } + None => { + return Err(InterpError::TypeError { + msg: "shell.Env.Get missing name parameter".to_string(), + }); + } + }; + let value = match wet_env_var(&name) { + Some(s) => Value::Str(s), + None => Value::Null, + }; + Ok(Value::Record { + type_name: ctx.sym(&authored_name_at(ctx.si(), op_node.clone())), + fields: Rc::new(vec![(ctx.sym("value"), value)]), + }) +} + fn build_service_param_env( op_node: &Rc, args: &[(Option, Value)], @@ -5681,6 +5769,64 @@ pub fn output_decision(channel: OutputChannel) -> OutputDecision { pub enum ExpectedOutcome { ExpectSuccess, ExpectFailure, + /// The exit code is an OBSERVATION consumed by a typed downstream verdict, not a + /// pass/fail judgment of its own — a probe whose non-zero exit means "subject + /// absent" rather than "something broke". Renders ambient regardless of code; + /// only dispatch failure (the probe could not run) stays anomalous. Admissible + /// ONLY where the annotated helper returns a typed observation or verdict, never + /// unit — see `gunbc.output_policy` outcome_is_data_note for the guard. + OutcomeIsData, +} + +/// The reserved call-site argument through which a caller DECLARES what it expects +/// an effect to do — the sibling edge on the CALL node (operator decision +/// 2026-07-25), resolving the open question this file carried at `dispatch_shell`. +/// +/// It is stripped from the argument list before `build_service_param_env`, so it +/// never becomes a bound param and therefore never reaches +/// `content_hash_service_inputs` — which iterates `op_node.params` and looks each up +/// in `param_env`. The exclusion is structural, not a remembered filter, which is +/// exactly why a service-op `input {}` was the wrong home: that IS a param, so it +/// would join the digest and two invocations differing only in what the caller +/// expected would become different cache identities for the same request. +/// +/// It is equally NOT a transport property, despite `transport_stdin` / +/// `transport_response_format` being the nearest local precedent. `dispatch_service_wet` +/// receives the transport from `op_node.transport.or(service_node.transport)` — the +/// extdeps service-op DECLARATION, shared by every caller. Hanging the expectation +/// there would make it a per-operation fact, so a red control and a genuine check +/// both calling `shell.Test.IsFile` could not differ, and caller policy declared in +/// extdeps is the DESIGN §3 layer inversion. That shape would have typechecked and +/// read as green while being wrong in both directions. +pub const EFFECT_EXPECTATION_ARG: &str = "expect"; + +/// Read the caller's declared expectation off the reserved argument's value. +/// +/// Absent is handled by the caller as `ExpectSuccess` — the DECLARED migration +/// default, behaviour-identical to the untyped `exit != 0` proxy this replaces, so +/// no existing call site changes meaning. A PRESENT but unreadable value REFUSES +/// instead of falling back: `gunbc.output_policy` deliberately has no `ExpectAny` +/// arm, so a value we cannot read is ignorance, and answering it with a default +/// would be ⊤-as-answer conflated with ⊤-as-ignorance (DESIGN §5 — a failure arm +/// refuses, never widens). +fn expectation_from_declared_arg(val: &Value) -> InterpResult { + match val { + Value::Variant { variant_name, .. } => match resolve_sym(*variant_name).as_str() { + "ExpectSuccess" => Ok(ExpectedOutcome::ExpectSuccess), + "ExpectFailure" => Ok(ExpectedOutcome::ExpectFailure), + "OutcomeIsData" => Ok(ExpectedOutcome::OutcomeIsData), + other => Err(InterpError::TypeError { + msg: format!( + "`{EFFECT_EXPECTATION_ARG}:` must be an ExpectedOutcome (ExpectSuccess | ExpectFailure | OutcomeIsData), got variant `{other}`" + ), + }), + }, + _ => Err(InterpError::TypeError { + msg: format!( + "`{EFFECT_EXPECTATION_ARG}:` must be an ExpectedOutcome (ExpectSuccess | ExpectFailure), got a non-variant value" + ), + }), + } } /// Carrier for `gunbc.output_policy.StreamDisposition` — what becomes of an effect's @@ -5701,7 +5847,7 @@ pub enum StreamDisposition { #[derive(Clone)] pub struct InstalledEffectStreamPolicy { /// Indexed by `stream_policy_index(expected, observed_success)`. - pub dispositions: [StreamDisposition; 4], + pub dispositions: [StreamDisposition; 6], pub subject_line_guard: String, } @@ -5714,11 +5860,14 @@ static EFFECT_STREAM_POLICY: std::sync::OnceLock = /// zero", exactly what this file did before the expectation axis existed. /// `effect_stream_policy_mirror_matches_dag_authority` pins it to the same golden /// the `.dag` witness asserts, so the two cannot drift silently. -const EFFECT_STREAM_POLICY_FALLBACK: [StreamDisposition; 4] = [ +const EFFECT_STREAM_POLICY_FALLBACK: [StreamDisposition; 6] = [ StreamDisposition::SummarizeCounts, // ExpectSuccess × observed success StreamDisposition::SurfaceContent, // ExpectSuccess × observed failure StreamDisposition::SurfaceContent, // ExpectFailure × observed success StreamDisposition::SummarizeCounts, // ExpectFailure × observed failure + StreamDisposition::SummarizeCounts, // OutcomeIsData × observed success + StreamDisposition::SummarizeCounts, // OutcomeIsData × observed failure — the exit is + // an answer, so neither pole is an anomaly ]; /// Mirror of `extdeps.github.log_annotations.subject_text_line_guard`, used when no @@ -5731,6 +5880,8 @@ fn stream_policy_index(expected: ExpectedOutcome, observed_success: bool) -> usi (ExpectedOutcome::ExpectSuccess, false) => 1, (ExpectedOutcome::ExpectFailure, true) => 2, (ExpectedOutcome::ExpectFailure, false) => 3, + (ExpectedOutcome::OutcomeIsData, true) => 4, + (ExpectedOutcome::OutcomeIsData, false) => 5, } } @@ -5800,20 +5951,30 @@ pub fn host_trace_grouping_active() -> bool { || output_decision(OutputChannel::Instrumentation) != OutputDecision::Suppressed) } +/// Tracks an open host-effect group so `group_end` is idempotent — law 4 closes the +/// group before an Anomaly shell failure, and the batch-end `group_end` must not emit +/// a second `::endgroup::`. +static GROUP_OPEN: AtomicBool = AtomicBool::new(false); + /// Open a titled group on stderr — the same stream the host-effect trace lines use, /// so the runner folds those lines under the marker. No-op when no syntax is /// installed. Pair with `group_end`; the caller must keep the bracket tight (open → /// run+join the effectful work → close) and defer non-trace output (PASS/FAIL) until -/// after `group_end` so it stays OUTSIDE the collapsed section. +/// after `group_end` so it stays outside the collapsed section. pub fn group_begin(title: &str) { if let Some(s) = GROUP_SYNTAX.get() { eprintln!("{}{}{}", s.open_prefix, title, s.open_suffix); + GROUP_OPEN.store(true, Ordering::SeqCst); } } /// Close the current group. Emits the close line only when the target defines one -/// (GitHub Actions); a plain terminal closes implicitly and prints nothing. +/// (GitHub Actions) and a group is actually open. Idempotent: a second call is a +/// no-op (law 4 may have already closed for an Anomaly). pub fn group_end() { + if !GROUP_OPEN.swap(false, Ordering::SeqCst) { + return; + } if let Some(s) = GROUP_SYNTAX.get() { if let Some(close) = &s.close_line { eprintln!("{close}"); @@ -5832,90 +5993,134 @@ fn trace_emit(channel: OutputChannel, line: &str) { } } -// The funnel for the ShellTrace channel. Consumes the installed -// `gunbc.output_policy` ShellTrace decision (Suppressed / Condensed / Full) -// rather than re-deriving it from verbosity — keeps CI logs readable instead of -// dumping every `sh -c` script. -fn render_shell_trace(argv: &[String]) { - match output_decision(OutputChannel::ShellTrace) { - OutputDecision::Suppressed => {} - OutputDecision::Full => eprintln!("[shell] {}", argv.join(" ")), - OutputDecision::Condensed => { - // Collapse newlines/runs of whitespace into a single readable line, - // then truncate so a multiline `sh -c` script is one tidy summary. - let collapsed: String = argv - .join(" ") - .split_whitespace() - .collect::>() - .join(" "); - // Fallback column bound (no Viewport at the trace site); the single - // authority is `gunbc.output_policy.shell_trace_summary_max_columns`. - const MAX: usize = 100; - let summary = if collapsed.chars().count() > MAX { - let head: String = collapsed.chars().take(MAX).collect(); - format!("{head}…") - } else { - collapsed - }; - eprintln!("{}", paint(&format!(" $ {summary}"), sgr::DIM)); - } +/// Census hygiene marker for the `[shell]` emit family — kept after the wiring flip so +/// the observation_emit_census roster cannot go stale. +pub const SHELL_CENSUS_MARKER: &str = "[shell]"; + +/// Collapse argv into one readable line — runs of whitespace become a single space — +/// so a multiline `sh -c` script reads as one command. Used in Failed.error (uncapped: +/// an anomaly expands fully). Ambient subjects are named intents, not argv. +fn shell_argv_collapsed(argv: &[String]) -> String { + argv.join(" ") + .split_whitespace() + .collect::>() + .join(" ") +} + +fn shell_obs_emoji() -> bool { + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true") +} + +fn shell_obs_human_duration(ms: u64) -> String { + if ms < 1_000 { + format!("{ms}ms") + } else if ms < 60_000 { + format!("{} seconds", ms / 1_000) + } else { + format!("{} minutes", ms / 60_000) } } -fn shell_completion_trace_line( - exit_code: i32, - stdout_bytes: usize, - stderr_bytes: usize, - wall: std::time::Duration, +/// Mirror of `gunbc.observation_seed_render.shell_effect_begin_line`. +pub fn render_shell_effect_begin_line_mirror(intent: &str, emoji: bool) -> String { + let _ = SHELL_CENSUS_MARKER; + let glyph = if emoji { "🔄" } else { "◐" }; + format!("{glyph} started {intent}") +} + +/// Mirror of `gunbc.observation_seed_render.shell_effect_done_line`. +pub fn render_shell_effect_done_line_mirror(intent: &str, elapsed_ms: u64, emoji: bool) -> String { + let _ = SHELL_CENSUS_MARKER; + let glyph = if emoji { "✅" } else { "✓" }; + format!( + "{glyph} {intent} done in {}", + shell_obs_human_duration(elapsed_ms) + ) +} + +/// Mirror of `gunbc.observation_seed_render.shell_effect_failed_line`. +/// Subject is the named intent; `argv_collapsed` (WITHOUT `$ `) feeds Failed.error only. +pub fn render_shell_effect_failed_line_mirror( + intent: &str, + argv_collapsed: &str, + exit_code: u64, + elapsed_ms: u64, + emoji: bool, ) -> String { + let _ = SHELL_CENSUS_MARKER; + let glyph = if emoji { "❌" } else { "✗" }; format!( - "[shell] done exit={exit_code} stdout={stdout_bytes} stderr={stderr_bytes} bytes wall={:.3}s", - wall.as_secs_f64() + "{glyph} {intent} failed: $ {argv_collapsed} (exit={exit_code}) in {}", + shell_obs_human_duration(elapsed_ms) ) } -/// Post-wait completion trace for every shell transport: exit, stdout/stderr bytes, -/// spawn-to-wait wall seconds. Pairs with `render_shell_trace` (pre-spawn). +// Ambient shell Begin — named intent subject, ShellTrace-gated (Suppressed at Normal). +fn render_shell_trace(intent: &str) { + if output_decision(OutputChannel::ShellTrace) == OutputDecision::Suppressed { + return; + } + let line = render_shell_effect_begin_line_mirror(intent, shell_obs_emoji()); + trace_emit(OutputChannel::ShellTrace, &line); +} + +/// Post-wait completion: Ambient Done via ShellTrace, or Anomaly Failed via +/// `effect_stream_disposition` alone (never silenced by ShellTrace Suppressed). +/// Law 4: `group_end` before an Anomaly so it lands OutsideGroup. /// -/// Whether the captured stderr CONTENT is surfaced (tail-bounded) or left as a byte -/// count is the `.dag` authority's `effect_stream_disposition`, keyed on what the -/// caller DECLARED this effect would do: divergence surfaces, agreement counts. At -/// the migration default `ExpectSuccess` that is the previous behaviour exactly — -/// content on non-zero exit (a failing op whose error text is discarded is an -/// undiagnosable failure, DESIGN §5; a whole self-host build failure was once -/// invisible in CI because only `stderr=N bytes` was logged), counts on success so -/// benign compiler warnings do not drown the trace. What the expectation axis adds -/// is the other direction: an effect declared `ExpectFailure` that SUCCEEDS is -/// divergence, and now surfaces instead of passing silently. +/// Subject is the typed service.op intent. Failed.error carries self-describing +/// `$ (exit=N)`; empty stderr still surfaces via the Failed line alone. fn render_shell_completion_trace( expected: ExpectedOutcome, exit_code: i32, - stdout_bytes: usize, + _stdout_bytes: usize, stderr: &[u8], wall: std::time::Duration, + argv: &[String], + intent: &str, ) { - trace_emit( - OutputChannel::ShellTrace, - &shell_completion_trace_line(exit_code, stdout_bytes, stderr.len(), wall), - ); let disposition = effect_stream_disposition(expected, exit_code); - if let Some(block) = shell_completion_stderr_trace_block(disposition, exit_code, stderr) { - trace_emit(OutputChannel::ShellTrace, &block); + let emoji = shell_obs_emoji(); + let elapsed_ms = wall.as_millis() as u64; + let collapsed = shell_argv_collapsed(argv); + + if disposition == StreamDisposition::SurfaceContent { + group_end(); + let code = if exit_code < 0 { + exit_code.unsigned_abs() as u64 + } else { + exit_code as u64 + }; + let line = + render_shell_effect_failed_line_mirror(intent, &collapsed, code, elapsed_ms, emoji); + eprintln!("{line}"); + if let Some(block) = shell_completion_stderr_content(stderr) { + eprintln!("{block}"); + } + return; + } + + if output_decision(OutputChannel::ShellTrace) == OutputDecision::Suppressed { + return; } + let line = render_shell_effect_done_line_mirror(intent, elapsed_ms, emoji); + trace_emit(OutputChannel::ShellTrace, &line); } -/// Pure tail-bounding of captured stderr for the completion trace. Returns `None` when the -/// disposition is not `SurfaceContent`, or when there is no stderr to surface; `Some(block)` -/// is the `[shell] stderr` diagnostic, its content tail-bounded with a leading elision marker -/// when it exceeds the cap and every subject line guarded so relayed text cannot mint workflow -/// commands in the parent run. Kept pure (no `trace_emit`) so the surfacing decision is -/// unit-testable with a RED control. -fn shell_completion_stderr_trace_block( - disposition: StreamDisposition, - exit_code: i32, - stderr: &[u8], -) -> Option { - if disposition != StreamDisposition::SurfaceContent || stderr.is_empty() { +/// Whether a SurfaceContent failure should emit a Failed observation line. +/// Pure: disposition alone — never the ShellTrace channel. Empty stderr still +/// returns true (the Failed line is the sole signal). RED control for the +/// installed-policy path: pair with `effect_stream_disposition`. +fn shell_failure_surfaces(disposition: StreamDisposition) -> bool { + disposition == StreamDisposition::SurfaceContent +} + +/// Pure tail-bounding of captured stderr that follows a Failed observation line. +/// Returns `None` when there is no stderr to surface; `Some(block)` is the content +/// only (the Failed line already carries `$ argv (exit=N)`). Subject lines are +/// guarded so relayed text cannot mint workflow commands in the parent run. +fn shell_completion_stderr_content(stderr: &[u8]) -> Option { + if stderr.is_empty() { return None; } const MAX_STDERR_TRACE: usize = 16384; @@ -5933,11 +6138,9 @@ fn shell_completion_stderr_trace_block( String::new() }; Some(format!( - "[shell] stderr (exit={exit_code}):\n{prefix}{}", + "{prefix}{}", // Trailing newlines are stripped before guarding so a subject whose stderr // ends in `\n` (almost all of them) does not render a stray guard-only line. - // This is presentation of the block, not a change to the guard rule: the - // .dag authority still prefixes every line of whatever text it is given. neutralize_workflow_commands(String::from_utf8_lossy(tail).trim_end_matches('\n')) )) } @@ -6085,17 +6288,13 @@ fn dispatch_shell( transport: &Rc, param_env: &Rc, ctx: &InterpContext, + intent: &str, + expected: ExpectedOutcome, ) -> InterpResult { - // Migration default: every effect issues as `ExpectSuccess`, which makes the - // dispatch below behaviour-identical to the untyped `exit != 0` proxy it - // replaces. This single binding is the seam a declared per-call-site - // expectation threads through, so probes can annotate `ExpectFailure`; where - // that declaration LIVES is still open (a service-op `input {}` would put it in - // `param_env` here, but it would also join `content_hash_service_inputs`, and - // two invocations differing only in what the caller expected are the same - // request — a cache-identity impurity, plus caller policy declared in extdeps - // is the layer inversion DESIGN §3 names). - let expected = ExpectedOutcome::ExpectSuccess; + // `expected` is the caller's DECLARED expectation, threaded from the sibling edge + // on the call node (EFFECT_EXPECTATION_ARG). Absent at the call site it is + // ExpectSuccess, which keeps every undeclared site behaviour-identical to the + // untyped `exit != 0` proxy this replaces. let argv_nodes = &transport.children; let mut argv: Vec = Vec::new(); for node in argv_nodes.iter() { @@ -6109,7 +6308,7 @@ fn dispatch_shell( }); } - render_shell_trace(&argv); + render_shell_trace(intent); // Arg-size wall: a single argv token over the host MAX_ARG_STRLEN would make // the spawn below die with an opaque `os error 7` (E2BIG). Refuse here with a @@ -6167,6 +6366,8 @@ fn dispatch_shell( output.stdout.len(), &output.stderr, wall_start.elapsed(), + &argv, + intent, ); output } else { @@ -6188,6 +6389,8 @@ fn dispatch_shell( output.stdout.len(), &output.stderr, wall_start.elapsed(), + &argv, + intent, ); output }; @@ -7254,7 +7457,14 @@ fn eval_filesystem_read_builtin(path: String, ctx: &InterpContext) -> InterpResu } let args = [(Some("path".to_string()), Value::Str(path))]; - let result = eval_service_call("Filesystem", "Read", &args, &Env::empty(), ctx)?; + let result = eval_service_call( + "Filesystem", + "Read", + &args, + &Env::empty(), + ctx, + ExpectedOutcome::ExpectSuccess, + )?; let (content, success, error) = match result { Value::Record { fields, .. } => { @@ -10284,13 +10494,16 @@ mod dispatch_rest_decision_tests { mod shell_completion_trace_tests { use super::hermetic_checkout_read_disposition_under; use super::neutralize_workflow_commands; - use super::shell_completion_stderr_trace_block; - use super::shell_completion_trace_line; + use super::render_shell_effect_begin_line_mirror; + use super::render_shell_effect_done_line_mirror; + use super::render_shell_effect_failed_line_mirror; + use super::shell_argv_collapsed; + use super::shell_completion_stderr_content; + use super::shell_failure_surfaces; use super::{ effect_stream_disposition, ExpectedOutcome, StreamDisposition, EFFECT_STREAM_POLICY_FALLBACK, SUBJECT_LINE_GUARD_FALLBACK, }; - use std::time::Duration; /// Convenience for the tests below: the disposition a failing effect gets when /// its caller declared success — the migration default, and what every call @@ -10299,61 +10512,95 @@ mod shell_completion_trace_tests { StreamDisposition::SurfaceContent } + fn av(parts: &[&str]) -> Vec { + parts.iter().map(|s| s.to_string()).collect() + } + #[test] - fn shell_completion_trace_line_formats_exit_stdout_stderr_wall() { - let line = shell_completion_trace_line(0, 1234, 56, Duration::from_millis(5150)); - assert_eq!( - line, - "[shell] done exit=0 stdout=1234 stderr=56 bytes wall=5.150s" - ); + fn shell_effect_begin_mirror_formats_started_subject() { + let line = render_shell_effect_begin_line_mirror("shell.Exec.Run", true); + assert_eq!(line, "🔄 started shell.Exec.Run"); + let unicode = render_shell_effect_begin_line_mirror("git.Core.HeadCommit", false); + assert_eq!(unicode, "◐ started git.Core.HeadCommit"); } #[test] - fn stderr_block_surfaces_content_on_nonzero_exit() { - let block = - shell_completion_stderr_trace_block(surfacing(), 101, b"error: manifest not found\n") - .expect("non-zero exit with stderr must surface a diagnostic block"); - assert!(block.starts_with("[shell] stderr (exit=101):\n")); - assert!(block.contains("error: manifest not found")); + fn shell_effect_done_mirror_formats_duration() { + let line = render_shell_effect_done_line_mirror("shell.Exec.Run", 5150, true); + assert_eq!(line, "✅ shell.Exec.Run done in 5 seconds"); } #[test] - fn stderr_block_none_when_disposition_is_not_surface_content() { - // RED control: the block is gated on the .dag disposition, not on a local - // `exit != 0` re-derivation — a non-surfacing disposition yields nothing even - // with a non-zero exit and non-empty stderr, and an empty stderr yields - // nothing even when the disposition says surface. - assert_eq!( - shell_completion_stderr_trace_block( - StreamDisposition::SummarizeCounts, - 1, - b"error: real failure\n" - ), - None - ); + fn shell_effect_failed_mirror_is_self_describing() { + // Failed.error carries `$ (exit=N)` so the line stands alone when + // stderr is empty (the common CI miss). + let line = + render_shell_effect_failed_line_mirror("shell.Exec.Run", "echo hi", 1, 2000, true); assert_eq!( - shell_completion_stderr_trace_block( - StreamDisposition::StreamSuppressed, - 1, - b"error: real failure\n" - ), - None + line, + "❌ shell.Exec.Run failed: $ echo hi (exit=1) in 2 seconds" ); + } + + #[test] + fn shell_argv_collapsed_squeezes_whitespace() { assert_eq!( - shell_completion_stderr_trace_block(surfacing(), 1, b""), - None + shell_argv_collapsed(&av(&["sh", "-c", "echo hi\nthere"])), + "sh -c echo hi there" ); } #[test] - fn stderr_block_tail_bounds_and_marks_elision() { + fn stderr_content_surfaces_body_on_nonzero_exit() { + let block = shell_completion_stderr_content(b"error: manifest not found\n") + .expect("non-empty stderr must surface a content block"); + assert!(block.contains("error: manifest not found")); + assert!(!block.contains("[shell]")); + } + + #[test] + fn stderr_content_none_when_empty() { + assert_eq!(shell_completion_stderr_content(b""), None); + } + + #[test] + fn failure_surfaces_from_disposition_alone_not_channel() { + // RED control: the Failed observation is gated on the .dag disposition, not + // on a local `exit != 0` re-derivation — and empty stderr does NOT suppress it + // (the Failed line is the sole signal once Ambient counts are observation Done). + assert!(shell_failure_surfaces(surfacing())); + assert!(!shell_failure_surfaces(StreamDisposition::SummarizeCounts)); + assert!(!shell_failure_surfaces(StreamDisposition::StreamSuppressed)); + } + + #[test] + fn at_normal_a_failing_effect_surfaces_its_command_a_passing_one_is_silent() { + // Composes the .dag disposition (ExpectSuccess × exit → the Normal four + // corners via the uninstalled fallback that mirrors Normal) with the + // failure-surfaces predicate. GREEN: passing → SummarizeCounts → silent. + // Discriminating RED: failing → SurfaceContent → Failed line must fire + // even with empty stderr (self-describing `$ argv`). + assert!(!shell_failure_surfaces(effect_stream_disposition( + ExpectedOutcome::ExpectSuccess, + 0 + ))); + assert!(shell_failure_surfaces(effect_stream_disposition( + ExpectedOutcome::ExpectSuccess, + 1 + ))); + let collapsed = shell_argv_collapsed(&av(&["git", "rev-parse", "--show-toplevel"])); + let line = + render_shell_effect_failed_line_mirror("git.Core.Toplevel", &collapsed, 1, 0, false); + assert!(line.contains("failed: $ git rev-parse --show-toplevel (exit=1)")); + assert!(line.starts_with("✗ git.Core.Toplevel failed:")); + assert_eq!(shell_completion_stderr_content(b""), None); + } + + #[test] + fn stderr_content_tail_bounds_and_marks_elision() { let big = vec![b'x'; 16384 + 500]; - let block = shell_completion_stderr_trace_block(surfacing(), 1, &big) - .expect("oversized stderr surfaces"); + let block = shell_completion_stderr_content(&big).expect("oversized stderr surfaces"); assert!(block.contains("<500 earlier stderr bytes elided>")); - // Only the 16384-byte tail is carried, not the full 16884-byte body: the trailing - // contiguous run of stderr bytes is exactly the cap. The guard prefix is a - // line-initial insertion, so it does not disturb the trailing run. assert_eq!(block.chars().rev().take_while(|c| *c == 'x').count(), 16384); } @@ -10362,13 +10609,11 @@ mod shell_completion_trace_tests { // The priced incident: a child's stderr legitimately carrying `::error::` // annotated the PARENT run as failing. Every relayed line is guarded, so no // subject text can occupy the line-initial `::` position GitHub parses. - let block = shell_completion_stderr_trace_block( - surfacing(), - 1, + let block = shell_completion_stderr_content( b"::error::build verification: artifact absent\n::warning::next\n", ) .expect("failing effect surfaces its stderr"); - for line in block.lines().skip(1) { + for line in block.lines() { assert!( !line.trim_start().starts_with("::"), "relayed subject line is command-bearing: {line:?}" @@ -10379,7 +10624,7 @@ mod shell_completion_trace_tests { #[test] fn effect_stream_policy_mirror_matches_dag_authority() { - // Mirror pin for the uninstalled fallback: these are the four corners the + // Mirror pin for the uninstalled fallback: these are the six corners the // .dag witness `w_shell_trace_stream_policy_projects_the_four_corners` // asserts at Normal verbosity, and the guard literal // `extdeps.github.log_annotations.subject_text_line_guard` publishes. If the @@ -10392,6 +10637,9 @@ mod shell_completion_trace_tests { StreamDisposition::SurfaceContent, StreamDisposition::SurfaceContent, StreamDisposition::SummarizeCounts, + // OutcomeIsData: neither pole is an anomaly — the exit is an answer. + StreamDisposition::SummarizeCounts, + StreamDisposition::SummarizeCounts, ] ); assert_eq!(SUBJECT_LINE_GUARD_FALLBACK, "| "); @@ -10421,15 +10669,6 @@ mod shell_completion_trace_tests { ); } - #[test] - fn shell_completion_trace_line_surfaces_nonzero_exit() { - let line = shell_completion_trace_line(1, 0, 4096, Duration::from_secs(2)); - assert_eq!( - line, - "[shell] done exit=1 stdout=0 stderr=4096 bytes wall=2.000s" - ); - } - #[test] fn hermetic_checkout_read_admits_relative_path_under_root() { let dir = @@ -10635,8 +10874,8 @@ mod argv_arg_limit_test { use crate::v1_std_core::{make_span, make_text_part_node, shell_transport_node, Node}; use super::{ - argv_arg_limit_refusal, dispatch_shell, Env, ExecutionMode, InterpContext, InterpError, - HOST_ARG_MAX_STRLEN_BYTES, + argv_arg_limit_refusal, dispatch_shell, Env, ExecutionMode, ExpectedOutcome, InterpContext, + InterpError, HOST_ARG_MAX_STRLEN_BYTES, }; fn argv_limit_test_context() -> InterpContext { @@ -10726,7 +10965,13 @@ mod argv_arg_limit_test { let ctx = argv_limit_test_context(); let transport = shell_check_style_transport(&"x".repeat(HOST_ARG_MAX_STRLEN_BYTES + 1)); let env = Env::empty(); - match dispatch_shell(&transport, &env, &ctx) { + match dispatch_shell( + &transport, + &env, + &ctx, + "shell.Exec.Check", + ExpectedOutcome::ExpectSuccess, + ) { Err(InterpError::ArgvExceedsHostArgMax { actual_bytes, limit_bytes, @@ -10750,7 +10995,13 @@ mod argv_arg_limit_test { let ctx = argv_limit_test_context(); let transport = shell_check_style_transport("true"); let env = Env::empty(); - match dispatch_shell(&transport, &env, &ctx) { + match dispatch_shell( + &transport, + &env, + &ctx, + "shell.Exec.Check", + ExpectedOutcome::ExpectSuccess, + ) { Err(InterpError::ArgvExceedsHostArgMax { .. }) => { panic!("small argv must not trip the arg-size wall") } diff --git a/src/v1/stage0/src/v1_rt.rs b/src/v1/stage0/src/v1_rt.rs index 3211dd7c99a..50bb6a0a1b0 100644 --- a/src/v1/stage0/src/v1_rt.rs +++ b/src/v1/stage0/src/v1_rt.rs @@ -760,9 +760,53 @@ fn expect_hash_digest(s: &str, arg: &str) { } } +/// Kept so `gunbc.observation_emit_census` roster hygiene cannot go stale after the +/// raw gantt key=value shape (process-absolute millis and rss-mib fields) dissolves into the observation projection. +#[allow(dead_code)] +pub const GANTT_CENSUS_MARKER: &str = "[gantt]"; + +/// Mirror of `gunbc.observation_ci_render.ci_minute_switch_seconds`. +const OBS_MINUTE_SWITCH_SECONDS: u64 = 90; + +fn obs_human_duration(ms: u64) -> String { + if ms < 1_000 { + format!("{ms}ms") + } else if ms < OBS_MINUTE_SWITCH_SECONDS * 1_000 { + format!("{} seconds", ms / 1_000) + } else { + format!("{} minutes", ms / 60_000) + } +} + +fn obs_phase_emoji() -> bool { + std::env::var("GITHUB_ACTIONS").as_deref() == Ok("true") +} + +/// Pure Rust mirror of `gunbc.observation_seed_render.phase_begin_line` — +/// `ci_event_line(Begin) ∘ ci_render_line`. Justified divergence from the +/// interpreter seed boundary used by floor prelude marks: `trace_mark` runs +/// inside compile itself (hand-synced path and interpreter intrinsic), and +/// calling back into the interpreter to render would recurse. Proven +/// byte-equal to the `.dag` oracle by the seed RED. +pub fn render_phase_begin_line_mirror(phase: &str, emoji: bool) -> String { + let glyph = if emoji { "🔄" } else { "◐" }; + format!("{glyph} started {phase}") +} + +/// Pure Rust mirror of `gunbc.observation_seed_render.phase_concluded_line` — +/// `ci_event_line(Concluded{Done}) ∘ ci_render_line`. Same justified-divergence +/// note as `render_phase_begin_line_mirror`. +pub fn render_phase_concluded_line_mirror(phase: &str, elapsed_ms: u64, emoji: bool) -> String { + let glyph = if emoji { "✅" } else { "✓" }; + format!("{glyph} {phase} done in {}", obs_human_duration(elapsed_ms)) +} + /// Stage-boundary trace mark — the v1-seed interim realization of the v2 per-RealizedStep -/// CostAccount (std.realization_measurement). One stderr line per mark; consecutive marks -/// define the segments of a natural Gantt read directly off any run log. +/// CostAccount (std.realization_measurement). Wired (gantt flip): projects Begin/Concluded +/// on a PhaseSegment via the observation mirrors of `ci_event_line`. Segment wall on +/// Concluded (matched `.begin`→`.done`, else delta since last mark) — never the old +/// process-absolute millis field. RSS stays on heartbeat/measurement (`ci_event_line` does +/// not project `event.rss` — explicit divergence from the deleted rss-mib field). /// /// **Dissolution trigger (DESIGN §6):** delete this fn, the `trace_mark` registry row in /// `04_method.dag` (+ hand-synced twin `v1_compiler_infer_method.rs`), the nine @@ -774,25 +818,54 @@ fn expect_hash_digest(s: &str, arg: &str) { /// § dissolution). Receipt = that witness green with these marks deleted and stage walls /// still attributable from the model path. pub fn trace_mark(label: String) { - use std::sync::OnceLock; + use std::sync::{Mutex, OnceLock}; use std::time::Instant; static TRACE_T0: OnceLock = OnceLock::new(); - let ms = TRACE_T0.get_or_init(Instant::now).elapsed().as_millis(); - let mut rss_mib = String::from("absent"); - if let Ok(status) = std::fs::read_to_string("/proc/self/status") { - for line in status.lines() { - if let Some(rest) = line.strip_prefix("VmRSS:") { - if let Some(kib) = rest - .split_whitespace() - .next() - .and_then(|k| k.parse::().ok()) - { - rss_mib = (kib / 1024).to_string(); + static LAST_MARK: OnceLock> = OnceLock::new(); + static OPENS: OnceLock>> = OnceLock::new(); + let t0 = TRACE_T0.get_or_init(Instant::now); + let last = LAST_MARK.get_or_init(|| Mutex::new(*t0)); + let opens = OPENS.get_or_init(|| Mutex::new(std::collections::HashMap::new())); + let emoji = obs_phase_emoji(); + let now = Instant::now(); + let _ = GANTT_CENSUS_MARKER; + if let Some(phase) = label.strip_suffix(".begin") { + if let Ok(mut map) = opens.lock() { + map.insert(phase.to_string(), now); + } + if let Ok(mut l) = last.lock() { + *l = now; + } + eprintln!("{}", render_phase_begin_line_mirror(phase, emoji)); + } else if let Some(phase) = label.strip_suffix(".done") { + let elapsed_ms = { + let started = opens.lock().ok().and_then(|mut m| m.remove(phase)); + match started { + Some(t) => now.saturating_duration_since(t).as_millis() as u64, + None => { + let prev = last.lock().map(|l| *l).unwrap_or(*t0); + now.saturating_duration_since(prev).as_millis() as u64 } } + }; + if let Ok(mut l) = last.lock() { + *l = now; + } + eprintln!( + "{}", + render_phase_concluded_line_mirror(phase, elapsed_ms, emoji) + ); + } else { + let prev = last.lock().map(|l| *l).unwrap_or(*t0); + let elapsed_ms = now.saturating_duration_since(prev).as_millis() as u64; + if let Ok(mut l) = last.lock() { + *l = now; } + eprintln!( + "{}", + render_phase_concluded_line_mirror(&label, elapsed_ms, emoji) + ); } - eprintln!("[gantt] {} t_ms={} rss_mib={}", label, ms, rss_mib); } /// Content hash over raw bytes — the byte-level single authority. `atom_identity_hash` diff --git a/src/v2/test/claim/ci_floor_plan_witness_test.dag b/src/v2/test/claim/ci_floor_plan_witness_test.dag index 9ab6e93b040..e2a92fd1d7c 100644 --- a/src/v2/test/claim/ci_floor_plan_witness_test.dag +++ b/src/v2/test/claim/ci_floor_plan_witness_test.dag @@ -45,10 +45,13 @@ import v2.workflow.ci_floor_plan { falsifier_known_red_probe_batch_enrolled, falsifier_silent_pick_gate_batch_enrolled, falsifier_rehomed_bin_wet_batch_enrolled, + falsifier_cadence_witness_batch_enrolled, + falsifier_cadence_witness_entries, falsifier_substrate_long_lane_batch_enrolled, gunbc_ci_floor_schedule_lens_holds, - witness_floor_batch_wall_budgets_cover_schedule, - witness_floor_batch_wall_budgets_all_positive, + witness_floor_batch_clamp_params_cover_schedule, + witness_floor_batch_clamp_overhead_all_positive, + witness_floor_batch_clamp_rate_all_nonneg, } import gunbc.ci_layer_roots { witness_layer_roots, witness_discovery_scan_dirs, known_red_probe_entries, falsifier_silent_pick_gate_entries, falsifier_substrate_long_lane_entries } import v2.compiler.self_host.wet_receipt_enrollment { @@ -472,6 +475,24 @@ fn falsifier_self_host_wet_budget_declared_holds() -> Bool { && gunbc_ci_fast_lane_eval_budget_ms() == 5000 } +fn falsifier_cadence_witness_batch_holds(r: Runnable) -> Bool { + match r { + RunnableDiscoveryBatch { source_roots: _, scan_dirs: dirs, explicit_entries: entries, node_frontier_selection: mode, exclude_substrings: _, discovery_scope_dirs: _, profile: p } => + match mode { + SelectionApplied => true + SelectionPredictOnly => false + SelectionOff => false + } + && (length(xs: dirs) == 0) + && (length(xs: entries) == length(xs: falsifier_cadence_witness_entries())) + && discovery_batch_declares_wet(r: r) + && p.spawns_host_compiler + && (p.heavy_whole_tree_resolve == false) + RunnableSingleClaim { entry: _, function: _, profile: _ } => false + RunnableKernelWorkload { fused_op_count: _ } => false + } +} + fn falsifier_plan_structure_holds() -> Bool { let batches = gunbc_falsifier_batches() let predict_ok = (batches.first().length() == 1) @@ -482,7 +503,8 @@ fn falsifier_plan_structure_holds() -> Bool { let silent_pick_gate_enrolled = falsifier_silent_pick_gate_batch_enrolled() let rehomed_enrolled = falsifier_rehomed_bin_wet_batch_enrolled() let substrate_long_enrolled = falsifier_substrate_long_lane_batch_enrolled() - let expected_batches = 1 + (if wet_enrolled { 1 } else { 0 }) + (if wet_known_red_enrolled { 1 } else { 0 }) + (if probe_enrolled { 1 } else { 0 }) + (if silent_pick_gate_enrolled { 1 } else { 0 }) + (if rehomed_enrolled { 1 } else { 0 }) + (if substrate_long_enrolled { 1 } else { 0 }) + let cadence_enrolled = falsifier_cadence_witness_batch_enrolled() + let expected_batches = 1 + (if wet_enrolled { 1 } else { 0 }) + (if wet_known_red_enrolled { 1 } else { 0 }) + (if probe_enrolled { 1 } else { 0 }) + (if silent_pick_gate_enrolled { 1 } else { 0 }) + (if rehomed_enrolled { 1 } else { 0 }) + (if substrate_long_enrolled { 1 } else { 0 }) + (if cadence_enrolled { 1 } else { 0 }) let wet_ok = if wet_enrolled { let wet_batch = skip(xs: batches, n: 1).first() (wet_batch.length() == 1) @@ -529,7 +551,15 @@ fn falsifier_plan_structure_holds() -> Bool { } else { true } - (length(xs: batches) == expected_batches) && predict_ok && wet_ok && wet_known_red_ok && probe_ok && silent_pick_gate_ok && rehomed_ok && substrate_long_ok + let cadence_ok = if cadence_enrolled { + let cadence_offset = 1 + (if wet_enrolled { 1 } else { 0 }) + (if wet_known_red_enrolled { 1 } else { 0 }) + (if probe_enrolled { 1 } else { 0 }) + (if silent_pick_gate_enrolled { 1 } else { 0 }) + (if rehomed_enrolled { 1 } else { 0 }) + (if substrate_long_enrolled { 1 } else { 0 }) + let cadence_batch = skip(xs: batches, n: cadence_offset).first() + (cadence_batch.length() == 1) + && falsifier_cadence_witness_batch_holds(r: cadence_batch.first()) + } else { + true + } + (length(xs: batches) == expected_batches) && predict_ok && wet_ok && wet_known_red_ok && probe_ok && silent_pick_gate_ok && rehomed_ok && substrate_long_ok && cadence_ok } fn witness_cheap_gates_are_resource_anchors() -> Bool { @@ -559,8 +589,9 @@ test fn ci_floor_plan_witnesses() -> Bool { && witness_hermetic_floor_posture_declared() && witness_floor_arm_time_budget_refusal_plan_roster_authority() && witness_floor_arm_time_budget_refusal_materialized_roster_matches_seed() - && witness_floor_batch_wall_budgets_cover_schedule() - && witness_floor_batch_wall_budgets_all_positive() + && witness_floor_batch_clamp_params_cover_schedule() + && witness_floor_batch_clamp_overhead_all_positive() + && witness_floor_batch_clamp_rate_all_nonneg() && falsifier_rehomed_bin_wet_batch_enrolled() && falsifier_substrate_long_lane_batch_enrolled() } diff --git a/src/v2/test/claim/ci_placement_witness_test.dag b/src/v2/test/claim/ci_placement_witness_test.dag new file mode 100644 index 00000000000..476c0d72dcb --- /dev/null +++ b/src/v2/test/claim/ci_placement_witness_test.dag @@ -0,0 +1,34 @@ +module v2.test.claim.ci_placement_witness + +import std.logic { Bool } +import v2.workflow.ci_placement { + Placement, PrTier, Gauntlet, + PlacementClassification, Hermetic, Unclassified, + WarmCostReceipt, + placement_is_admissible +} + +test fn ci_placement_gauntlet_always_admissible_holds() -> Bool { + placement_is_admissible(p: Gauntlet { reason: "awaiting D2 warm-cost measurement" }) +} + +test fn ci_placement_valid_pr_tier_admissible_holds() -> Bool { + placement_is_admissible(p: PrTier { + warm_cost: WarmCostReceipt { measured_ms: 1600, run_ref: "synthetic-warm-receipt" }, + classification: Hermetic + }) +} + +test fn ci_placement_over_budget_pr_tier_refused_holds() -> Bool { + !placement_is_admissible(p: PrTier { + warm_cost: WarmCostReceipt { measured_ms: 6000, run_ref: "synthetic-over-budget" }, + classification: Hermetic + }) +} + +test fn ci_placement_unclassified_pr_tier_refused_holds() -> Bool { + !placement_is_admissible(p: PrTier { + warm_cost: WarmCostReceipt { measured_ms: 100, run_ref: "synthetic-unclassified" }, + classification: Unclassified + }) +} diff --git a/src/v2/test/claim/long/gauntlet_lane_enrollment_witness_test.dag b/src/v2/test/claim/long/gauntlet_lane_enrollment_witness_test.dag new file mode 100644 index 00000000000..91294ce71bc --- /dev/null +++ b/src/v2/test/claim/long/gauntlet_lane_enrollment_witness_test.dag @@ -0,0 +1,101 @@ +module test.claim.long.gauntlet_lane_enrollment_witness + +import gunbc.ci_layer_roots { SubstrateLongLaneRow, falsifier_substrate_long_lane_rows } +import gunbc.commit_workflow { + CommitCheckEnrollment, CommitCheckKind, CommitWitnessClaim, CommitCargoFmtCheck, + CommitWorkflowSurface, GithubActionsCiJob, GithubActionsCiRegenJob, + GitPrePushHook, FalsifierCadenceJob, + roster_has_no_bare_deenrolled_witness, + live_roster_has_no_bare_deenrolled_witness, + live_roster_falsifier_witness_entries +} +import std.realization_schedule { + ScheduleWitnessEntry, ExecutionWitnessKind, SpanUndeclared +} +import v2.std.algebra { any, length } +import v2.std.collection { List } +import v2.std.logic { Bool } +import v2.std.text { String } + +data gauntlet_lane_enrollment_note: String = "Executing consumer for the FalsifierCadenceJob lane (gunbc.commit_workflow falsifier_cadence_surface_note / bare_deenrollment_wall_note). It exists because the lane's whole value is the difference between 're-homed' and 'dropped', and that difference is invisible in a diff: both look like a surface list edit. Three claims, the third a RED control so the first two cannot pass vacuously. This file lives under test/claim/long/ for HOMING consistency with the rows it guards, but is enrolled per-PR on GithubActionsCiJob deliberately — it is a pure roster fold with no live-tree read and no host effect, measured well inside the fast-lane budget, and its whole point is to red on the PR that de-enrolls something rather than up to 4 hours later." + +fn synthetic_rehomed_row() -> CommitCheckEnrollment { + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/long/synthetic_rehomed_witness_test.dag", + check_fns: ["synthetic_rehomed_holds"], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, + surfaces: [FalsifierCadenceJob] + } +} + +fn synthetic_pr_row() -> CommitCheckEnrollment { + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/synthetic_pr_witness_test.dag", + check_fns: ["synthetic_pr_holds"], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, + surfaces: [GithubActionsCiJob] + } +} + +fn synthetic_bare_deenrolled_row() -> CommitCheckEnrollment { + CommitCheckEnrollment { + check: CommitWitnessClaim { + entry: "src/v2/test/claim/long/synthetic_dropped_witness_test.dag", + check_fns: ["synthetic_dropped_holds"], + kind: ExecutionWitnessKind, + span: SpanUndeclared + }, + surfaces: [GitPrePushHook, GithubActionsCiRegenJob] + } +} + +fn falsifier_entries_contain(entries: List, f: String) -> Bool { + any(entries, e => e.function == f) +} + +data substrate_long_lane_is_a_sibling_cadence_note: String = "wave1_gate1_d_census_subsumption_witness_holds moved off FalsifierCadenceJob onto falsifier_substrate_long_lane_rows (nightly falsifier batch 6) when main landed the Class C split. That is a re-home between two cadence lanes, NOT a de-enrollment, so the invariant this file guards still holds — but the pin has to read the lane the row actually sits on. Dropping the pin instead would have been the narrowing this witness exists to prevent: the row would then be free to vanish from BOTH lanes with nothing red. The check below therefore asserts substrate-long-lane membership by function name, which is the same claim ('this re-homed row still executes on a cadence') read against the correct authority." + +fn substrate_long_lane_contains(f: String) -> Bool { + any(falsifier_substrate_long_lane_rows, row => row.function == f) +} + +fn witness_rehomed_rows_are_cadence_enrolled() -> Bool { + let entries = live_roster_falsifier_witness_entries() + substrate_long_lane_contains(f: "wave1_gate1_d_census_subsumption_witness_holds") + && falsifier_entries_contain(entries: entries, f: "wave1_gate1_d_ingested_bind_executes_witness_holds") + && falsifier_entries_contain(entries: entries, f: "wave1_gate1_d_ingested_loop_executes_witness_holds") + && falsifier_entries_contain(entries: entries, f: "lens_affected_set_provenance_producer_rejects_malformed_ingest_holds") + && falsifier_entries_contain(entries: entries, f: "lens_affected_set_provenance_producer_rejects_beyond_surface_ingest_holds") + && falsifier_entries_contain(entries: entries, f: "lens_affected_set_provenance_producer_coverage_receipt_holds") + && falsifier_entries_contain(entries: entries, f: "witness_parse_binding_fidelity_distinct_identifiers_holds") +} + +fn witness_split_entries_keep_cheap_half_per_pr() -> Bool { + let cadence = live_roster_falsifier_witness_entries() + !falsifier_entries_contain(entries: cadence, f: "lens_affected_set_provenance_producer_git_diff_wire_holds") + && !falsifier_entries_contain(entries: cadence, f: "witness_resolve_distinct_param_bindings_holds") +} + +test fn gauntlet_lane_rehomed_rows_are_cadence_enrolled_holds() -> Bool { + witness_rehomed_rows_are_cadence_enrolled() + && witness_split_entries_keep_cheap_half_per_pr() +} + +test fn gauntlet_lane_rehome_is_never_bare_deenrollment_holds() -> Bool { + live_roster_has_no_bare_deenrolled_witness() + && roster_has_no_bare_deenrolled_witness( + roster: [synthetic_pr_row(), synthetic_rehomed_row()] + ) +} + +test fn gauntlet_lane_rehome_bare_deenrollment_RED_control_holds() -> Bool { + !roster_has_no_bare_deenrolled_witness( + roster: [synthetic_pr_row(), synthetic_rehomed_row(), synthetic_bare_deenrolled_row()] + ) +} diff --git a/src/v2/test/fixture/floor_skip/budget_red_control_plan.dag b/src/v2/test/fixture/floor_skip/budget_red_control_plan.dag index 695028c8492..ab7ad0fd1f8 100644 --- a/src/v2/test/fixture/floor_skip/budget_red_control_plan.dag +++ b/src/v2/test/fixture/floor_skip/budget_red_control_plan.dag @@ -8,7 +8,7 @@ import std.realization_schedule { import v2.std.collection { List } import std.types { Int } -data budget_red_control_plan_note: String = "THE COST WALL's by-execution RED-control fixture (endgame D5): a minimal plan whose fn is NAMED gunbc_ci_floor_batches so claim_executor arms the per-batch wall budgets against it — one tiny hermetic batch, one generous budget row. RED direction: GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS=0 tightens the budget to zero, the batch overruns by construction, and the run must exit nonzero with the typed FLOOR-BATCH-OVER-BUDGET refusal naming batch 1. GREEN direction: the same invocation without the env exits zero. The injection is tighten-only (min), so this fixture can force the refusal but can never widen a real budget — a fault injection, not an escape hatch. Recipe: target/release/claim_executor --source-root dag --source-root src/v2 --plan-entry src/v2/test/fixture/floor_skip/budget_red_control_plan.dag --plan-function gunbc_ci_floor_batches. Lives under test/fixture/floor_skip/ (discovery-excluded at dir grain) and declares no test fns, so it is invisible to the witness corpus — driven only by this recipe." +data budget_red_control_plan_note: String = "THE COST WALL's by-execution RED-control fixture (endgame D5): a minimal plan whose fn is NAMED gunbc_ci_floor_batches so claim_executor arms the per-batch derived clamp against it — one tiny hermetic batch, one generous overhead row (600s) with rate 0, so the computed clamp is 600s. RED direction: GUNBC_FLOOR_BATCH_BUDGET_TIGHTEN_MS=0 tightens the budget to zero, the batch overruns by construction, and the run must exit nonzero with the typed FLOOR-BATCH-OVER-BUDGET refusal naming batch 1. GREEN direction: the same invocation without the env exits zero. The injection is tighten-only (min), so this fixture can force the refusal but can never widen a real budget — a fault injection, not an escape hatch. Recipe: target/release/claim_executor --source-root dag --source-root src/v2 --plan-entry src/v2/test/fixture/floor_skip/budget_red_control_plan.dag --plan-function gunbc_ci_floor_batches. Lives under test/fixture/floor_skip/ (discovery-excluded at dir grain) and declares no test fns, so it is invisible to the witness corpus — driven only by this recipe." fn budget_red_control_claim() -> Runnable { RunnableSingleClaim { @@ -22,6 +22,10 @@ fn gunbc_ci_floor_batches() -> List> { [[budget_red_control_claim()]] } -fn gunbc_ci_floor_batch_wall_budgets_seconds() -> List { +fn gunbc_ci_floor_batch_clamp_overhead_seconds() -> List { [600] } + +fn gunbc_ci_floor_batch_clamp_per_unit_ms() -> List { + [0] +} diff --git a/src/v2/workflow/ci_floor_plan.dag b/src/v2/workflow/ci_floor_plan.dag index 506b021d558..a58185efd76 100644 --- a/src/v2/workflow/ci_floor_plan.dag +++ b/src/v2/workflow/ci_floor_plan.dag @@ -17,9 +17,10 @@ import gunbc.ci_spec { floor_plan_function, plan_artifact_plan_function, FloorBatchStopPolicy, - gunbc_ci_floor_batch_wall_budget_seconds, + gunbc_ci_floor_batch_clamp_params, } import gunbc.falsifier_workflow { falsifier_plan_function } +import gunbc.commit_workflow { commit_gate_roster, project_falsifier_witness_entries } import gunbc.ci_gate { Gate, EmitHostGate, CheapClaimPoolGate, ExtdepsExternalAuthorityGate, DagCompileCleanGate, GeneratedArtifactDriftGate, @@ -434,16 +435,24 @@ fn gunbc_ci_fast_lane_eval_budget_ms() -> v2.std.nat.Nat { second_count(s: gunbc_ci_fast_lane_witness_eval_budget) * 1000 } -fn gunbc_ci_floor_batch_wall_budgets_seconds() -> List { - gunbc_ci_floor_batch_wall_budget_seconds +fn gunbc_ci_floor_batch_clamp_overhead_seconds() -> List { + map(gunbc_ci_floor_batch_clamp_params, c => c.overhead_seconds) } -fn witness_floor_batch_wall_budgets_cover_schedule() -> Bool { - length(xs: gunbc_ci_floor_batch_wall_budgets_seconds()) == length(xs: gunbc_ci_floor_batches()) +fn gunbc_ci_floor_batch_clamp_per_unit_ms() -> List { + map(gunbc_ci_floor_batch_clamp_params, c => c.per_unit_ms) } -fn witness_floor_batch_wall_budgets_all_positive() -> Bool { - fold(gunbc_ci_floor_batch_wall_budgets_seconds(), init: true, f: fn(ok, b) { ok && (b > 0) }) +fn witness_floor_batch_clamp_params_cover_schedule() -> Bool { + length(xs: gunbc_ci_floor_batch_clamp_params) == length(xs: gunbc_ci_floor_batches()) +} + +fn witness_floor_batch_clamp_overhead_all_positive() -> Bool { + fold(gunbc_ci_floor_batch_clamp_overhead_seconds(), init: true, f: fn(ok, b) { ok && (b > 0) }) +} + +fn witness_floor_batch_clamp_rate_all_nonneg() -> Bool { + fold(gunbc_ci_floor_batch_clamp_per_unit_ms(), init: true, f: fn(ok, b) { ok && (b >= 0) }) } data gunbc_falsifier_self_host_wet_receipt_budget_note: String = "Whole-receipt wall-time budget for the nightly falsifier Wet self-host lane (operator ruling 2026-07-15, Wave 2 CI substrate STEP 3; kill-at-deadline 2026-07-25): bounds emit+cargo total wall time per witness — the wet receipt IS each module's Definition-of-Done, and cargo dominates (std/logic template ~101s wall, 06_translate ~52s measured Refinement-1). claim_executor arms this budget for discovery batches whose explicit entries intersect falsifier_self_host_wet_entries OR falsifier_self_host_wet_known_red_entries — never for silent-pick (owns gunbc_falsifier_silent_pick_gate_receipt_wall_budget) or rehomed bin. Primary refusal is WitnessWallBudgetExceeded: shell waits kill the process group at the ceiling (Finding 1 kill-at-deadline); wall_budget_completion_outcome remains the completion-side backstop. Secondary gunbc_falsifier_self_host_wet_interp_eval_budget arms the cooperative CPU eval deadline for interpreter-wedge detection only (subprocess cargo I/O is wall-counted, not CPU-counted). Receipted ceilings: logic template 101s, 06_translate 52s — 600s per receipt leaves headroom for larger modules while keeping 11-module upper bound (~9.5min measured) far under the 270m falsifier step budget. A raise requires a dated re-basis note (same discipline as floor batch walls) — never to absorb a cache-cold host." @@ -1009,11 +1018,43 @@ fn gunbc_falsifier_batches() -> List> { enrolled: falsifier_rehomed_bin_wet_batch_enrolled(), batch: gunbc_falsifier_rehomed_bin_wet_batch() ) - falsifier_batches_append_if_enrolled( + let with_substrate_long = falsifier_batches_append_if_enrolled( acc: with_rehomed, enrolled: falsifier_substrate_long_lane_batch_enrolled(), batch: gunbc_falsifier_substrate_long_lane_batch() ) + falsifier_batches_append_if_enrolled( + acc: with_substrate_long, + enrolled: falsifier_cadence_witness_batch_enrolled(), + batch: gunbc_falsifier_cadence_witness_batch() + ) +} + +data falsifier_cadence_witness_batch_note: String = "The live consumer for gunbc.commit_workflow's FalsifierCadenceJob surface — the scheduled home for declared CommitWitnessClaim rows priced out of the per-PR floor (falsifier_cadence_surface_note). WHY THIS BATCH HAS TO EXIST IN THE SAME CHANGE that re-homes the rows: a surface tag with no batch reading it is enrollment-by-illusion, the exact ~54-dark-file state under test/claim/long/ that the re-home is supposed to stop adding to, and it would present as a clean green diff. So the enrollment predicate and this batch land together, and gauntlet_lane_enrollment_witness_test reds per-PR if a row ever sits on the surface without reaching here. Profile mirrors gunbc_falsifier_rehomed_bin_wet_batch (Wet, host compiler, substantial memory): the re-homed rows are execution-kind witnesses whose cost is exactly why they left the floor. SelectionApplied so the cadence run still records predictions — the falsifier's own divergence accounting stays intact for these rows rather than being exempted." + +fn falsifier_cadence_witness_entries() -> List { + project_falsifier_witness_entries(roster: commit_gate_roster) +} + +fn falsifier_cadence_witness_batch_enrolled() -> Bool { + length(xs: falsifier_cadence_witness_entries()) > 0 +} + +fn gunbc_falsifier_cadence_witness_batch() -> Runnable { + RunnableDiscoveryBatch { + source_roots: corpus_source_roots(), + scan_dirs: [], + explicit_entries: falsifier_cadence_witness_entries(), + node_frontier_selection: SelectionApplied, + exclude_substrings: [], + discovery_scope_dirs: [], + profile: runnable_resource_profile( + heavy_whole_tree_resolve: false, + spawns_host_compiler: true, + memory: runnable_memory_substantial(), + execution_mode: Wet + ) + } } data gunbc_pr_native_batch_note: String = "Per-PR native batch (P6 routing consumption, 2026-07-22): explicit entries = the equals_eval agreement legs of families the native-routing frontier marks NativeRouted — nothing else. The frontier is the ONE authority for both sides of the floor (this batch enrolls exactly the routed set; nothing else changes), and flips are parity-window-gated data, so at landing the routed set is empty and this appends no batch: zero behavior change until receipts flip a family. Wet profile: the admitted transport cargo-builds cold or runs warm against GUNBC_NATIVE_CACHE_ROOT." diff --git a/src/v2/workflow/ci_placement.dag b/src/v2/workflow/ci_placement.dag new file mode 100644 index 00000000000..ab31b789c80 --- /dev/null +++ b/src/v2/workflow/ci_placement.dag @@ -0,0 +1,44 @@ +module v2.workflow.ci_placement + +import std.logic { Bool } +import std.nat { Nat } +import v2.std.text { String } +import v2.workflow.ci_floor_plan { gunbc_ci_fast_lane_eval_budget_ms } + +data ci_placement_law_note: String = "The two-tier placement axis (ci-two-tier-placement-redesign.md D3, operator 2026-07-24): a check rides the PR path iff its measured warm cost is within the fast-lane budget OR its cost is proportional to the diff; everything else is a Gauntlet row (main pushes + the 4h falsifier cadence). Placement is DATA (PrTier | Gauntlet), never per-site prose. Admission is FAIL-CLOSED by construction: PrTier is admissible ONLY with (a) a measured warm-cost receipt within the fast-lane budget and (b) a hermetic/ephemeral classification; an unmeasured or unclassified check is inadmissible as PrTier and its only valid placement is Gauntlet — no row rides the fast path by taste. The 5s threshold is REUSED from the single fast-lane authority (gunbc_ci_fast_lane_eval_budget_ms, v2.workflow.ci_floor_plan gunbc_ci_fast_lane_rule_note), never a second 5s definition. The roster of real checks and their PrTier flips are FILLED from the D2 warm-cost probe on fleet srv hardware; until a check has a measured receipt it stays Gauntlet, so this mechanism is correct-by-construction with an empty PrTier set. Wiring the placement onto the live check rows (gates, ci-job steps, floor batches) and the Gauntlet workflow split ride PR-1 (D3b)." + +type PlacementClassification + = Hermetic + | Ephemeral + | LiveTreeReading + | Unclassified + +type WarmCostReceipt { + measured_ms: Nat + run_ref: String +} + +type Placement + = PrTier { warm_cost: WarmCostReceipt, classification: PlacementClassification } + | Gauntlet { reason: String } + +fn classification_admits_pr_tier(c: PlacementClassification) -> Bool { + match c { + Hermetic => true + Ephemeral => true + LiveTreeReading => false + Unclassified => false + } +} + +fn warm_cost_within_fast_lane(wc: WarmCostReceipt) -> Bool { + wc.measured_ms <= gunbc_ci_fast_lane_eval_budget_ms() +} + +fn placement_is_admissible(p: Placement) -> Bool { + match p { + Gauntlet { reason: _ } => true + PrTier { warm_cost: wc, classification: c } => + warm_cost_within_fast_lane(wc: wc) && classification_admits_pr_tier(c: c) + } +}