Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dsl/extdeps/languages/python/primitives.dag
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
// structurally today (rust_pilot_primitives is loaded as `ValueBody::List`
// by `src/v3/grounding_engine/src/lib.rs:7`, registered via
// `src/v3/std/extdeps_bootstrap_fixtures.dag:23`). What is *not* yet live
// for this file: no `ExtdepsBootstrapFixture` row has been registered for
// for this file: no `BootstrapFixture` row has been registered for
// `python_spec_predeclared_primitives` or `python_dag_unit_target_primitive`,
// and there is no Python-grounding engine consumer in-tree yet — so today
// these rows are loaded but unconsumed (Path-2 staging in the
Expand Down
43 changes: 28 additions & 15 deletions src/v3/compiler/src/bootstrap.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,23 @@
// so the compiler's own pipeline authority lives in the bootstrap Dag with
// the intended stage-body shape.
//
// **B4.4 extdeps-bootstrap fixture set.** The structural carrier
// `extdeps_bootstrap_fixture_authority` in
// **B4.4 bootstrap fixture set.** The structural carrier
// `bootstrap_fixture_authority` in
// `src/v3/std/extdeps_bootstrap_fixtures.dag` is the substrate authority for
// *which* extdeps files participate. The PB-1-e regen host filters
// `build.rs`-emitted `EXTDEPS_FILES` through [`EXTDEPS_BOOTSTRAP_PATH_KEYS`]
// (must stay in lockstep with that declaration's `virtual_path` fields).
// Only extdeps authorities whose content is pure structural **data**
// (target-primitive declarations consumed symbolically by the
// target-grounding engine as `Declaration`-shaped values; see
// `dsl/extdeps/languages/rust/primitives.dag`) are loaded. Arrow/realization
// *which* `.dag` files participate. The PB-1-e regen host filters
// `build.rs`-emitted `STAGED_FILES` (covering `src/v3/std/*.dag`) and
// `EXTDEPS_FILES` (covering `dsl/extdeps/**/*.dag`) through
// [`BOOTSTRAP_FIXTURE_PATH_KEYS`] (must stay in lockstep with that
// declaration's `virtual_path` fields). The fixture set is mixed-tree as of
// T-Ground-LanguageSpec scope E.1: per-target method-template-contract
// lists live under `src/v3/std/` (avoiding the v2 walker's `dsl/` traversal
// per the post-#1187 regression analysis), while target-primitive
// declarations remain under `dsl/extdeps/languages/`. Only authorities
// whose content is pure structural **data** are loaded. Arrow/realization
// files (`rust/emit.dag`, `rust/types.dag`, etc.) are deliberately excluded
// — their bodies stay per-target emitter-side, not in the bootstrap Dag.
// Expansion adds a field to the carrier + a key in `EXTDEPS_BOOTSTRAP_PATH_KEYS`
// once python/go primitives reach the same pilot stage.
// Expansion adds a field to the carrier + a key in `BOOTSTRAP_FIXTURE_PATH_KEYS`
// once new fixture authorities reach the same pilot stage.
//
// **Structural list load (Path 2 scoping).** The top-level
// `rust_pilot_primitives: List<RustPrimitive> = [...]` data declaration now
Expand Down Expand Up @@ -80,13 +83,23 @@ use crate::pipeline_authority::{ordered_pipeline_stages, PIPELINE_AUTHORITY_FILE
#[cfg_attr(not(feature = "bootstrap-regen-fresh"), allow(dead_code))]
const PIPELINE_REALIZATION_META: &str = "CompilerHostRealization";

/// Virtual paths selecting extdeps `.dag` sources for PB-1-e bootstrap regen.
/// Virtual paths selecting `.dag` sources for PB-1-e bootstrap regen.
///
/// Mixed-tree fixture set (per T-Ground-LanguageSpec scope E.1): paths
/// resolve against `STAGED_FILES` (covering `src/v3/std/*.dag`) and
/// `EXTDEPS_FILES` (covering `dsl/extdeps/**/*.dag`). The regen host
/// (`bootstrap_regen_fresh`) walks both sets to locate each path.
///
/// **Lockstep:** each entry must match a `virtual_path` on
/// `extdeps_bootstrap_fixture_authority` in `src/v3/std/extdeps_bootstrap_fixtures.dag`.
/// `Dag::extdeps_bootstrap_fixture_virtual_paths` is compared to this slice when the
/// `bootstrap_fixture_authority` in `src/v3/std/extdeps_bootstrap_fixtures.dag`.
/// `Dag::bootstrap_fixture_virtual_paths` is compared to this slice when the
/// committed bootstrap snapshots initialize (`dag.rs` LazyLock).
pub const EXTDEPS_BOOTSTRAP_PATH_KEYS: &[&str] = &["dsl/extdeps/languages/rust/primitives.dag"];
pub const BOOTSTRAP_FIXTURE_PATH_KEYS: &[&str] = &[
"dsl/extdeps/languages/rust/primitives.dag",
"src/v3/std/rust_method_template_contracts.dag",
"src/v3/std/python_method_template_contracts.dag",
"src/v3/std/go_method_template_contracts.dag",
];

/// v3-only inhabitance for kernel `Bool` (Class 5 / Lane 1e-2b Path A).
///
Expand Down
2,874 changes: 1,442 additions & 1,432 deletions src/v3/compiler/src/bootstrap_generated.rs

Large diffs are not rendered by default.

2,708 changes: 1,359 additions & 1,349 deletions src/v3/compiler/src/bootstrap_generated_without_parse_surface.rs

Large diffs are not rendered by default.

25 changes: 17 additions & 8 deletions src/v3/compiler/src/bootstrap_regen_fresh.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,16 @@ include!(concat!(env!("OUT_DIR"), "/v3_specs.rs"));
include!(concat!(env!("OUT_DIR"), "/v3_compiler_files.rs"));
include!(concat!(env!("OUT_DIR"), "/v3_extdeps_files.rs"));

fn extdeps_bootstrap_fixtures() -> impl Iterator<Item = (&'static str, &'static str)> {
// `BOOTSTRAP_FIXTURE_PATH_KEYS` may resolve against either STAGED_FILES or
// EXTDEPS_FILES (mixed-tree fixture set per T-Ground-LanguageSpec scope E.1).
// STAGED_FILES are already loaded via `staged_iter` in
// `load_runtime_bootstrap_authorities`, so this iterator returns only the
// extdeps-tree subset of the keys to avoid duplicate-declaration loads.
fn extdeps_keyed_bootstrap_fixtures() -> impl Iterator<Item = (&'static str, &'static str)> {
EXTDEPS_FILES
.iter()
.copied()
.filter(|(path, _)| crate::bootstrap::EXTDEPS_BOOTSTRAP_PATH_KEYS.contains(path))
.filter(|(path, _)| crate::bootstrap::BOOTSTRAP_FIXTURE_PATH_KEYS.contains(path))
}

// Single authority for staged-vs-`dsl/` name collision resolution during regen.
Expand Down Expand Up @@ -94,26 +99,30 @@ fn load_runtime_bootstrap_authorities(
.iter()
.copied()
.filter(|(path, _)| !excluded_compiler_paths.contains(path));
assert_extdeps_bootstrap_keys_resolve_against_extdeps_files();
assert_bootstrap_fixture_keys_resolve();
let fixtures: Vec<(&str, &str)> = staged_iter
.chain(V3_SPECS.iter().copied())
.chain(compiler_iter)
.chain(extdeps_bootstrap_fixtures())
.chain(extdeps_keyed_bootstrap_fixtures())
.collect();
load_fixtures(dag, &fixtures);
crate::bootstrap::materialize_pipeline_realizations(dag);
dag.populate_primitive_cache();
}

fn assert_extdeps_bootstrap_keys_resolve_against_extdeps_files() {
let missing: Vec<&str> = crate::bootstrap::EXTDEPS_BOOTSTRAP_PATH_KEYS
fn assert_bootstrap_fixture_keys_resolve() {
let missing: Vec<&str> = crate::bootstrap::BOOTSTRAP_FIXTURE_PATH_KEYS
.iter()
.copied()
.filter(|key| !EXTDEPS_FILES.iter().any(|(path, _)| *path == *key))
.filter(|key| {
!STAGED_FILES.iter().any(|(path, _)| *path == *key)
&& !EXTDEPS_FILES.iter().any(|(path, _)| *path == *key)
})
.collect();
assert!(
missing.is_empty(),
"EXTDEPS_BOOTSTRAP_PATH_KEYS must each appear in EXTDEPS_FILES (from build.rs): {missing:?}"
"BOOTSTRAP_FIXTURE_PATH_KEYS must each appear in STAGED_FILES or EXTDEPS_FILES \
(from build.rs): {missing:?}"
);
}

Expand Down
28 changes: 14 additions & 14 deletions src/v3/compiler/src/dag.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
use std::collections::{HashMap, HashSet};
use std::sync::LazyLock;

use crate::bootstrap::EXTDEPS_BOOTSTRAP_PATH_KEYS;
use crate::bootstrap::BOOTSTRAP_FIXTURE_PATH_KEYS;
use crate::diagnostics::{Diagnostic, DiagnosticTable, SourceSpan};
use crate::types::TypeShape;

Expand Down Expand Up @@ -3036,8 +3036,8 @@ impl Dag {

/// Typed accessor for the `rust_pilot_primitives` data declaration
/// from `dsl/extdeps/languages/rust/primitives.dag` (path authorized by
/// B4.4 `extdeps_bootstrap_fixture_authority` and the regen host's
/// `EXTDEPS_BOOTSTRAP_PATH_KEYS` filter over `EXTDEPS_FILES`). Returns the
/// B4.4 `bootstrap_fixture_authority` and the regen host's
/// `BOOTSTRAP_FIXTURE_PATH_KEYS` filter over `EXTDEPS_FILES`). Returns the
/// top-level `List<RustPrimitive>` declaration whose *type* the
/// target-grounding engine walks structurally (`RustPrimitive =
/// IntegerPrimitive | NonIntegerPrimitive {target_name, algebra,
Expand All @@ -3058,7 +3058,7 @@ impl Dag {
}

/// Virtual paths from the B4.4 extdeps-bootstrap fixture carrier
/// (`extdeps_bootstrap_fixture_authority` in
/// (`bootstrap_fixture_authority` in
/// `src/v3/std/extdeps_bootstrap_fixtures.dag`), in the order fields
/// appear on the lowered `ValueBody::Structural` body.
///
Expand All @@ -3067,11 +3067,11 @@ impl Dag {
/// has zero fields (degenerate); callers that require a non-empty set should
/// assert separately.
///
/// Used to keep the regen host's `EXTDEPS_BOOTSTRAP_PATH_KEYS` filter
/// Used to keep the regen host's `BOOTSTRAP_FIXTURE_PATH_KEYS` filter
/// aligned with the substrate declaration (compared in this module's bootstrap
/// `LazyLock` initializers).
pub fn extdeps_bootstrap_fixture_virtual_paths(&self) -> Option<Vec<String>> {
let decl = self.declaration_by_name("extdeps_bootstrap_fixture_authority")?;
pub fn bootstrap_fixture_virtual_paths(&self) -> Option<Vec<String>> {
let decl = self.declaration_by_name("bootstrap_fixture_authority")?;
let body = decl.value_body.as_ref()?;
let ValueBody::Structural { fields } = body else {
return None;
Expand Down Expand Up @@ -3608,14 +3608,14 @@ impl Default for Dag {
}
}

/// PB-1-e B4.4: the substrate `extdeps_bootstrap_fixture_authority` carrier and
/// [`EXTDEPS_BOOTSTRAP_PATH_KEYS`](crate::bootstrap::EXTDEPS_BOOTSTRAP_PATH_KEYS)
/// PB-1-e B4.4: the substrate `bootstrap_fixture_authority` carrier and
/// [`BOOTSTRAP_FIXTURE_PATH_KEYS`](crate::bootstrap::BOOTSTRAP_FIXTURE_PATH_KEYS)
/// must list the same virtual paths in the same order — the const is the regen
/// filter; the `.dag` declaration is runtime authority.
fn assert_extdeps_bootstrap_fixture_paths_match_regen_keys(dag: &Dag) {
let Some(paths) = dag.extdeps_bootstrap_fixture_virtual_paths() else {
let Some(paths) = dag.bootstrap_fixture_virtual_paths() else {
panic!(
"bootstrap snapshot must expose `extdeps_bootstrap_fixture_authority` as a \
"bootstrap snapshot must expose `bootstrap_fixture_authority` as a \
structural `ValueBody` with well-formed `virtual_path` fields on each fixture \
slot (missing declaration, non-structural body, or malformed fixture records). \
Regenerate via `regen_bootstrap` after editing \
Expand All @@ -3625,11 +3625,11 @@ fn assert_extdeps_bootstrap_fixture_paths_match_regen_keys(dag: &Dag) {
if !paths
.iter()
.map(String::as_str)
.eq(EXTDEPS_BOOTSTRAP_PATH_KEYS.iter().copied())
.eq(BOOTSTRAP_FIXTURE_PATH_KEYS.iter().copied())
{
panic!(
"`EXTDEPS_BOOTSTRAP_PATH_KEYS` must match `extdeps_bootstrap_fixture_authority` \
virtual_path fields in order.\n substrate: {paths:?}\n regen keys: {EXTDEPS_BOOTSTRAP_PATH_KEYS:?}"
"`BOOTSTRAP_FIXTURE_PATH_KEYS` must match `bootstrap_fixture_authority` \
virtual_path fields in order.\n substrate: {paths:?}\n regen keys: {BOOTSTRAP_FIXTURE_PATH_KEYS:?}"
);
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/v3/compiler/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -997,7 +997,7 @@ pub mod lens_structural_resolution {

mod bootstrap;

pub use bootstrap::EXTDEPS_BOOTSTRAP_PATH_KEYS;
pub use bootstrap::BOOTSTRAP_FIXTURE_PATH_KEYS;

#[cfg(feature = "bootstrap-regen-fresh")]
mod bootstrap_regen_fresh;
Expand Down
37 changes: 27 additions & 10 deletions src/v3/compiler/tests/integration/method_template_contract_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -124,10 +124,24 @@ fn assert_per_target_list_dag_method_unique(dag: &Dag, list_name: &str) {
.iter()
.find(|(label, _)| label == "dag_method")
.unwrap_or_else(|| panic!("row {idx} in `{list_name}` missing `dag_method` field"));
let FieldValue::Reference(decl_id) = dag_method else {
let FieldValue::Record(method_ref_fields) = dag_method else {
panic!(
"row {idx} in `{list_name}`: `dag_method` must be a \
`FieldValue::Reference(DeclarationId)`; got {dag_method:?}"
`FieldValue::Record` (the `MethodRef {{ decl }}` shape); \
got {dag_method:?}"
);
};
let (_, decl_field) = method_ref_fields
.iter()
.find(|(label, _)| label == "decl")
.unwrap_or_else(|| {
panic!("row {idx} in `{list_name}`: `MethodRef` missing `decl` field")
});
let FieldValue::Reference(decl_id) = decl_field else {
panic!(
"row {idx} in `{list_name}`: `MethodRef.decl` must be a \
`FieldValue::Reference(DeclarationId)` pointing at a \
`dsl/std/methods.dag` `MethodDeclaration`; got {decl_field:?}"
);
};
assert!(
Expand All @@ -140,17 +154,20 @@ fn assert_per_target_list_dag_method_unique(dag: &Dag, list_name: &str) {

#[test]
fn method_template_contract_per_target_dag_method_unique() {
// The uniqueness walker is intentionally present, but no target row-list
// authorities are loaded yet. Empty `dsl/extdeps/.../method_template_contracts.dag`
// scaffolds imported `v3.std.emit_model` from the shared extdeps tree and
// polluted the v2 loader; Grounding row-list authorities land after the
// Substrate method-declaration registry gives `dag_method` real targets.
// Until then this loop is a zero-authority placeholder, not a fake empty
// fixture.
// Phase 1 (T-Ground-LanguageSpec scope E.1): per-target row lists landed
// at `src/v3/std/{rust,python,go}_method_template_contracts.dag`,
// populated with registry-backed `MethodRef` rows referencing
// `dsl/std/methods.dag`. The walker is now load-bearing: each row
// references a top-level `MethodDeclaration` and uniqueness within each
// list is verified structurally.
let dag = generated_full_bootstrap_dag();
for list_name in EXPECTED_PER_TARGET_LISTS {
assert_per_target_list_dag_method_unique(&dag, list_name);
}
}

const EXPECTED_PER_TARGET_LISTS: &[&str] = &[];
const EXPECTED_PER_TARGET_LISTS: &[&str] = &[
"rust_method_template_contracts",
"python_method_template_contracts",
"go_method_template_contracts",
];
5 changes: 4 additions & 1 deletion src/v3/compiler/tests/integration/parse_corpus_manifest.txt
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,17 @@ src/v3/std/diagnostics.dag 10 12179 bc414c493551b464
src/v3/std/dimensions.dag 21 40239 d595746d2bda22f6
src/v3/std/effects.dag 53 97984 1acdbeaa68acfd2e
src/v3/std/emit_model.dag 35 65950 80de7505c56ef93a
src/v3/std/extdeps_bootstrap_fixtures.dag 4 4912 e2b6d96f5900a70a
src/v3/std/extdeps_bootstrap_fixtures.dag 4 12090 e307b3796658c074
src/v3/std/go_method_template_contracts.dag 5 93546 b383eacaafd0dc1c
src/v3/std/induction.dag 56 88794 546f74c63c189c50
src/v3/std/lens.dag 6 9852 28d5539a5b28716e
src/v3/std/list.dag 19 40269 8c648d4a4fd05c21
src/v3/std/lookup.dag 5 4734 f97d4e536b675b1b
src/v3/std/methods.dag 3 1388 48f145b21c115326
src/v3/std/parse_surface.dag 18 99510 b73114d5b178303d
src/v3/std/python_method_template_contracts.dag 5 120438 17c36841b44234d2
src/v3/std/resources.dag 5 3632 5303e31e96b01ee5
src/v3/std/rust_method_template_contracts.dag 5 61868 78c4638120922194
src/v3/std/substrate.dag 57 124531 9d03ca981cab59c3
src/v3/std/substrate_minimal.dag 8 5372 feeb314c735f4672
src/v3/std/termination.dag 18 36478 d010232f0373dc99
Expand Down
50 changes: 37 additions & 13 deletions src/v3/std/extdeps_bootstrap_fixtures.dag
Original file line number Diff line number Diff line change
@@ -1,30 +1,54 @@
module v3.std.extdeps_bootstrap_fixtures

// B4.4 — Structural extdeps-bootstrap fixture-set carrier.
// B4.4 — Structural bootstrap fixture-set carrier.
//
// Single substrate authority for *which* `dsl/extdeps/**/*.dag` files are
// loaded into the PB-1-e bootstrap Dag as structural primitive data (Path 2
// pilot). Arrow/realization authorities stay emitter-side; this set is the
// bounded exception documented in `bootstrap.rs`.
// Single substrate authority for *which* `.dag` files are loaded into the
// PB-1-e bootstrap Dag as structural primitive data (Path 2 pilot). The
// fixture set is **mixed-tree** as of T-Ground-LanguageSpec scope E.1:
// per-target render-template contract lists live under `src/v3/std/` (the
// v2 walker doesn't traverse that tree, avoiding the cross-tree-import
// regression #1187 surfaced); target primitives still live under
// `dsl/extdeps/languages/`. The carrier is the structural authority
// regardless of where each file physically lives.
//
// The regen host (`bootstrap_regen_fresh`) filters `EXTDEPS_FILES` (emitted by
// `build.rs`) through the `virtual_path` strings recorded here. At runtime the
// The regen host (`bootstrap_regen_fresh`) filters `STAGED_FILES`
// (`src/v3/std/*.dag`) and `EXTDEPS_FILES` (`dsl/extdeps/**/*.dag`)
// through the `virtual_path` strings recorded here. At runtime the
// committed bootstrap snapshot load compares those paths to
// `EXTDEPS_BOOTSTRAP_PATH_KEYS` in `bootstrap.rs` (see `dag.rs`).
// `BOOTSTRAP_FIXTURE_PATH_KEYS` in `bootstrap.rs` (see `dag.rs`).
//
// **Filename note.** This file is named
// `extdeps_bootstrap_fixtures.dag` for historical reasons (B4.4 landed
// under the extdeps-only framing). The name is preserved so the
// parse-corpus manifest path stays stable; the carrier-name rename
// (`Extdeps...` → unprefixed) reflects the now-mixed scope without
// incurring a manifest-row churn.

// One extdeps file authorized for bootstrap loading.
type ExtdepsBootstrapFixture {
// One source file authorized for bootstrap loading.
type BootstrapFixture {
virtual_path: String
}

// Named product of every pilot fixture row (extend with new fields when adding
// targets; cost of change stays this file + the parallel `const` filter).
type ExtdepsBootstrapFixtureSet {
rust_pilot_primitives: ExtdepsBootstrapFixture
type BootstrapFixtureSet {
rust_pilot_primitives: BootstrapFixture
rust_method_template_contracts: BootstrapFixture
python_method_template_contracts: BootstrapFixture
go_method_template_contracts: BootstrapFixture
}

data extdeps_bootstrap_fixture_authority: ExtdepsBootstrapFixtureSet = {
data bootstrap_fixture_authority: BootstrapFixtureSet = {
rust_pilot_primitives: {
virtual_path: "dsl/extdeps/languages/rust/primitives.dag"
},
rust_method_template_contracts: {
virtual_path: "src/v3/std/rust_method_template_contracts.dag"
},
python_method_template_contracts: {
virtual_path: "src/v3/std/python_method_template_contracts.dag"
},
go_method_template_contracts: {
virtual_path: "src/v3/std/go_method_template_contracts.dag"
}
}
Loading
Loading