Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ A flaky or green-but-broken floor means no gate protects anything — so CI is u
- [x] **Section 1 spawn-width foundation — std.measure expressibility** (#5470/#5478) — the FLOOR family (measure_scale_fraction_floor + measure_fit_count_floor) and the demand-CEIL family completing the ceil-not-equal-floor money-pair landed, dissolving the measure unwrap→raw-arith→rewrap §3 fork. Authority: the expressibility-frontier spec #5467 [plan](docs/plans/expressibility-frontier.md)
- [ ] **scheduling: placement + materialization inert** — `Placement`/`Materialization` modeled + witness-passing, **no live consumer** (same band as the host-ops gap, substrate side)
- [ ] **scheduling: resource-aware (larger allocation → larger throughput)** — `spawn_width` is a side-channel derived from a static data row (`gunbc_ci_floor_measured_peak`, coarse whole_run÷concurrency estimate); goal: width derived from per-`Runnable` cost in the plan itself, self-calibrating from measured per-shard RSS. Four nodes: (A) measurement plumbing — claim_batch emits `[measurement] per-shard-peak-rss`, claim_executor collects max *(dispatched adhoc-97532cd3-dfe)*; (B) `CostEstimate.space` on `Runnable` in std; (C) scheduler reads cost → derives width, side-channel deleted; (D) calibration loop retires static data rows. Construction invariant: host memory increase → higher width → more throughput, no `.dag` edits required. [plan](docs/plans/resource-aware-scheduler.md)
- [ ] **caching forked** — sccache live · resolve-cache **always-on (#5789 — default `$TMPDIR/gunbc-rg-cache`, CI pending merge)** · ParseTable memo live · RecordedFixture · BuildBuddy opt-in → converge on `realize(subject)` (§2 P2)
- [ ] **caching forked** — sccache live · resolve-cache **opt-in (`GUNBC_RESOLVED_GRAPH_CACHE_DIR`; #5789 always-on default reverted — its whole-file JSON read/write OOMs the floor, must stream before it can default on)** · ParseTable memo live · RecordedFixture · BuildBuddy opt-in → converge on `realize(subject)` (§2 P2)

**Host-operation band — off-fabric, unmodeled, unenforced (G1–G3, NOW):**

Expand Down Expand Up @@ -157,7 +157,7 @@ Gate: uncached non-redundant work is an ERROR, not "slow". The cache-key-from-in
- [x] P1 honest keys by construction — warm==cold purity oracle **LANDED (#5429)**
- [ ] P2 one door: `realize(subject)` sole API — kernel inhabits `cache_interface.dag` (#5446); ParseTable dissolution is downstream of the dsl→v2 de-fork (§5)
- hermetic fixtures feed P2: [x] M4.1 universal hermetic corpus governance (#5236, [plan](docs/plans/m4-universal-hermetic-corpus.md)); [ ] M5 fixture-store onto one Realization kernel ([plan](docs/plans/m5-fixture-store-consolidation.md))
- [x] P3 **resolve-cache enable** — purity proven (616/616); #5789 makes cache **always-on** (default `$TMPDIR/gunbc-rg-cache`, no env var required; CI pending merge 2026-06-25) ← **core ask landed**
- [ ] P3 **resolve-cache enable** — purity proven (616/616); cache is correct but **opt-in** (`GUNBC_RESOLVED_GRAPH_CACHE_DIR`): #5789's always-on `$TMPDIR` default was reverted because the whole-file JSON read (hit) and write (miss) buffer ~11x the packed graph in memory and OOM the concurrent-shard floor. Re-enabling by default is gated on a streaming IO realization (binary + `deserialize_from`/`serialize_into`, no whole-file `Vec<u8>`)
- [ ] P4 economic tier (measured cost → `Materialization`) — instrument done (#5431); remaining = (a) space arm: per-shard RSS plumbing → `CostEstimate.space` on `Runnable` (nodes A+B, [plan](docs/plans/resource-aware-scheduler.md)); (b) time arm: consumer feedback → `CostAccount.time = Measured` feeds width-fold (node C+D); dissolves `cost_account_predicted_zero()` on the floor path
- [ ] P5 native `content(T) = content_hash(subgraph)` — gated on B2

Expand Down
4 changes: 2 additions & 2 deletions dsl/gunbc/roadmap_authority.dag
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ fn section_1() -> RoadmapSection {
derivable_doc(id: "1-measure-foundation", prs: [5470, 5478], title: "**Section 1 spawn-width foundation — std.measure expressibility**", description: "— the FLOOR family (measure_scale_fraction_floor + measure_fit_count_floor) and the demand-CEIL family completing the ceil-not-equal-floor money-pair landed, dissolving the measure unwrap→raw-arith→rewrap §3 fork. Authority: the expressibility-frontier spec #5467", path: "docs/plans/expressibility-frontier.md"),
authored(id: "1-placement-inert", done: false, content: "**scheduling: placement + materialization inert** — `Placement`/`Materialization` modeled + witness-passing, **no live consumer** (same band as the host-ops gap, substrate side)"),
authored(id: "1-sched-resource-aware", done: false, content: "**scheduling: resource-aware (larger allocation → larger throughput)** — `spawn_width` is a side-channel derived from a static data row (`gunbc_ci_floor_measured_peak`, coarse whole_run÷concurrency estimate); goal: width derived from per-`Runnable` cost in the plan itself, self-calibrating from measured per-shard RSS. Four nodes: (A) measurement plumbing — claim_batch emits `[measurement] per-shard-peak-rss`, claim_executor collects max *(dispatched adhoc-97532cd3-dfe)*; (B) `CostEstimate.space` on `Runnable` in std; (C) scheduler reads cost → derives width, side-channel deleted; (D) calibration loop retires static data rows. Construction invariant: host memory increase → higher width → more throughput, no `.dag` edits required. [plan](docs/plans/resource-aware-scheduler.md)"),
authored(id: "1-caching-forked", done: false, content: "**caching forked** — sccache live · resolve-cache **always-on (#5789 — default `$TMPDIR/gunbc-rg-cache`, CI pending merge)** · ParseTable memo live · RecordedFixture · BuildBuddy opt-in → converge on `realize(subject)` (§2 P2)"),
authored(id: "1-caching-forked", done: false, content: "**caching forked** — sccache live · resolve-cache **opt-in (`GUNBC_RESOLVED_GRAPH_CACHE_DIR`; #5789 always-on default reverted — its whole-file JSON read/write OOMs the floor, must stream before it can default on)** · ParseTable memo live · RecordedFixture · BuildBuddy opt-in → converge on `realize(subject)` (§2 P2)"),
],
edges: [],
),
Expand Down Expand Up @@ -241,7 +241,7 @@ fn section_2() -> RoadmapSection {
authored(id: "2-p1", done: true, content: "P1 honest keys by construction — warm==cold purity oracle **LANDED (#5429)**"),
authored(id: "2-p2", done: false, content: "P2 one door: `realize(subject)` sole API — kernel inhabits `cache_interface.dag` (#5446); ParseTable dissolution is downstream of the dsl→v2 de-fork (§5)"),
plain(id: "2-hermetic", content: "hermetic fixtures feed P2: [x] M4.1 universal hermetic corpus governance (#5236, [plan](docs/plans/m4-universal-hermetic-corpus.md)); [ ] M5 fixture-store onto one Realization kernel ([plan](docs/plans/m5-fixture-store-consolidation.md))"),
authored(id: "2-p3", done: true, content: "P3 **resolve-cache enable** — purity proven (616/616); #5789 makes cache **always-on** (default `$TMPDIR/gunbc-rg-cache`, no env var required; CI pending merge 2026-06-25) ← **core ask landed**"),
authored(id: "2-p3", done: false, content: "P3 **resolve-cache enable** — purity proven (616/616); cache is correct but **opt-in** (`GUNBC_RESOLVED_GRAPH_CACHE_DIR`): #5789's always-on `$TMPDIR` default was reverted because the whole-file JSON read (hit) and write (miss) buffer ~11x the packed graph in memory and OOM the concurrent-shard floor. Re-enabling by default is gated on a streaming IO realization (binary + `deserialize_from`/`serialize_into`, no whole-file `Vec<u8>`)"),
authored(id: "2-p4", done: false, content: "P4 economic tier (measured cost → `Materialization`) — instrument done (#5431); remaining = (a) space arm: per-shard RSS plumbing → `CostEstimate.space` on `Runnable` (nodes A+B, [plan](docs/plans/resource-aware-scheduler.md)); (b) time arm: consumer feedback → `CostAccount.time = Measured` feeds width-fold (node C+D); dissolves `cost_account_predicted_zero()` on the floor path"),
authored(id: "2-p5", done: false, content: "P5 native `content(T) = content_hash(subgraph)` — gated on B2"),
plain(id: "2-blockers", content: "blockers: [ ] B1 #5295 generic-instantiation (gates cross-shard `Share`) · [ ] B2 cross-tree content-hash (gates P5)"),
Expand Down
10 changes: 4 additions & 6 deletions src/v1/stage0/src/cli_run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ use crate::v1_std_core::{
use serde::Serialize;

use crate::resolved_graph_cache::{
lookup as cross_process_lookup, resolved_graph_cache_root, subject_digest_for_closure,
write as cross_process_write, CacheLookupResult,
lookup as cross_process_lookup, resolved_graph_cache_root_from_env,
subject_digest_for_closure, write as cross_process_write, CacheLookupResult,
};

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
Expand Down Expand Up @@ -404,8 +404,7 @@ fn resolve_entry_with_parse_cache(
> {
let sources = load_sources_for_entry_with_index(&index.source_files, entry_file)?;

{
let cache_root = resolved_graph_cache_root();
if let Some(cache_root) = resolved_graph_cache_root_from_env() {
let subject = subject_digest_for_closure(&sources);
match cross_process_lookup(&cache_root, &subject) {
CacheLookupResult::Hit(hit) => {
Expand Down Expand Up @@ -515,8 +514,7 @@ fn resolve_entry_with_parse_cache(
return Err(format_error_nodes(&ownership_diags, &source_indices));
}

{
let cache_root = resolved_graph_cache_root();
if let Some(cache_root) = resolved_graph_cache_root_from_env() {
let subject = subject_digest_for_closure(&sources);
let _ = cross_process_write(&cache_root, &subject, &typed, source_indices.as_ref());
}
Expand Down
24 changes: 15 additions & 9 deletions src/v1/stage0/src/resolved_graph_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -65,15 +65,21 @@ pub enum CacheWriteOutcome {
AlreadyExists,
}

pub fn resolved_graph_cache_root() -> PathBuf {
std::env::var_os("GUNBC_RESOLVED_GRAPH_CACHE_DIR")
.map(PathBuf::from)
.unwrap_or_else(|| {
let user_tag = std::env::var("USER")
.or_else(|_| std::env::var("USERNAME"))
.unwrap_or_else(|_| "shared".to_string());
std::env::temp_dir().join(format!("gunbc-rg-cache-{user_tag}"))
})
/// The cross-process resolved-graph disk cache is **opt-in**: it activates only
/// when `GUNBC_RESOLVED_GRAPH_CACHE_DIR` names a directory. With it unset the
/// cache is entirely off (`None` — no read and no write).
///
/// Why opt-in and not a `temp_dir()` default: under the CI floor the cache is a
/// net loss. Every commit changes the compiler binary, which is part of the
/// content digest, so the floor re-colds each run and is overwhelmingly a miss;
/// and both paths buffer a whole cache file in memory (a hit `read_to_end`s the
/// verbose-JSON file, ~11x the packed graph; a miss `to_vec`s the whole JSON),
/// so multi-GiB entries resolved across concurrent shards OOM the runner. The
/// cache pays its worst cost exactly where it collects ~no benefit. (#5789 made
/// it always-on via a `temp_dir()` default; this reverts to opt-in — its IO
/// realization must stream, not buffer, before it is safe to default on.)
pub fn resolved_graph_cache_root_from_env() -> Option<PathBuf> {
std::env::var_os("GUNBC_RESOLVED_GRAPH_CACHE_DIR").map(PathBuf::from)
}

fn extract_module_path(content: &str) -> Option<String> {
Expand Down