N7-1b: std.algebra collection callback ops (map, all) realized through the one fold encoding, keyed on named template rows

dag/gunbc/recurring_failure_mode/collection_callback_operation_unrealized_in_v2_lowering.dag

@@ -0,0 +1,24 @@
+module gunbc.recurring_failure_mode.collection_callback_operation_unrealized_in_v2_lowering
+
+import std.types { NonEmptyStr }
+import std.decl_ref { DeclarationRef, WholeDeclaration }
+import gunbc.recurring_failure_mode { RecurringFailureMode }
+
+data collection_callback_operation_unrealized_in_v2_lowering: RecurringFailureMode = RecurringFailureMode {
+  identity: "collection_callback_operation_unrealized_in_v2_lowering" as NonEmptyStr,
+  receipts: [
+    "INVALID STATE: std.algebra models collection callback operations as template rows with callback_element_position Present (map, filter, flat_map, any, all, sort_by), but v2 body lowering realized only the fold family. A call of an unrealized row reaches v2.compiler.resolve as a bare name and refuses resolve_reason_unbound_symbol. HARM: v2.std.node calls `map(node.children, e => Edge { .. })`, so on the native route it refused its own resolve and dropped out of the closure index; about twenty other v2 modules call bare `map(`. The refusal is typed and located, so this is a capability gap on the loud side of the ladder, never a silent wrong answer.",
+    "DISTINGUISHING FACTS: the earliest unjustified boundary was the MODEL, not lowering. A realized roster call binds its head to the row's declaration (v2.compiler.resolve resolve_realized_declaration), and map / flat_map / any / all / sort_by were inline list elements in std.algebra free_monoid_collection_templates and finite_power_set_templates, copied once per list, with no declaration to bind to. They are now named shapes (collection_map_shape and siblings) referenced from both lists, as fold and filter already were.",
+    "CLIMB (N7-1b): map and all are realized through the one fold encoding, keyed on their rows in v2.compiler.fold_lowering collection_roster_rows. Each step is synthesized around the callback's own binder (CallbackRealization), and is O(1) per member: map conses one cell onto the carrier, and all joins one Bool. Controls: v2.test.claim.compiler.collection_callback_realization. all's body types through &&, which infer does not yet join (a separate frontier), so its control asserts the head binding and lowering, not infer acceptance.",
+    "COUNTED FRONTIER, 4 rows, by name. filter and any: v2.std.algebra DECLARES ordinary fns `filter` and `any`, and 41 modules each import and call them positionally or piped (measured at N7-1b's base, 2dae714bf4). Lowering classifies a head by its spelling without a binding, so a realized row would lower those calls to a Loop that v2.compiler.resolve resolve_realized_declaration refuses as realized_declaration_shadowed. That is a section 3 fork, the roster row and the corpus fn being one operation under two authorities. TRIGGER: the fork is consolidated (v2.std.algebra any/filter retired onto the roster operation, or a binding-aware realization under which a head bound to a corpus declaration stays an ordinary call), SUFFICIENT FOR an unbound `filter(xs, e => ..)` / `any(xs, e => ..)` to lower and bind its row while every imported call keeps binding. Its guard is ccr_declared_any_stays_an_ordinary_call. flat_map: its step is itself a fold over the callback's result (v2.std.algebra list_flat_map's nested fold_list_right), which the synthesized-step table cannot express as one cell. TRIGGER: a nested-fold step, i.e. a CallbackRealization arm whose step body is a fold encoding over the callback body with the outer accumulator as its init, SUFFICIENT FOR `flat_map(xs, e => f(e))` to lower and its head to bind collection_flat_map_shape. sort_by: cost_shape ShapeSortBody, which is not an iterate body, so it has no fold realization at all. TRIGGER: a sort realization the template's ShapeSortBody names, SUFFICIENT FOR `sort_by(xs, e => k(e))` to lower and its head to bind collection_sort_by_shape. RUNG: mitigatable (typed refusal at resolve). CEILING: structurally guaranteed: every fact is decidable from the row.",
+  ],
+  evidence: [
+    DeclarationRef { module_path: "std.algebra", decl_name: "collection_map_shape", field: WholeDeclaration },
+    DeclarationRef { module_path: "std.algebra", decl_name: "collection_flat_map_shape", field: WholeDeclaration },
+    DeclarationRef { module_path: "std.algebra", decl_name: "collection_sort_by_shape", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.compiler.fold_lowering", decl_name: "collection_roster_rows", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.compiler.fold_lowering", decl_name: "CallbackRealization", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.compiler.body_lowering_fold", decl_name: "body_lower_callback_encoding", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.test.claim.compiler.collection_callback_realization", decl_name: "ccr_map_over_records_reads_a_field", field: WholeDeclaration },
+  ],
+}

dag/gunbc/recurring_failure_mode/fold_realized_loop_carries_no_iteration_direction.dag

@@ -0,0 +1,26 @@
+module gunbc.recurring_failure_mode.fold_realized_loop_carries_no_iteration_direction
+
+import std.types { NonEmptyStr }
+import std.decl_ref { DeclarationRef, WholeDeclaration }
+import gunbc.recurring_failure_mode { RecurringFailureMode }
+
+data fold_realized_loop_carries_no_iteration_direction: RecurringFailureMode = RecurringFailureMode {
+  identity: "fold_realized_loop_carries_no_iteration_direction" as NonEmptyStr,
+  receipts: [
+    "INVALID STATE: a fold-realized Loop's encoding does not show its iteration direction. v2.compiler.fold_lowering fold_recurrence_encoding builds the identical Loop for the collection fold, fold_list (left), fold_list_right (right) and, since N7-1b, map, whose synthesized step conses onto the carrier and is order-preserving ONLY under right-to-left iteration (all joins with &&, which is order-insensitive). An executor that ran a map encoding left to right would return its results reversed: silent wrongness. CLIMBED IN gunbc#13069 (review 74982 asked that the PR introducing map not ship the reversible encoding with only this row): the direction is now a DECLARED FACT OF THE REALIZATION, v2.compiler.fold_lowering FoldIterationDirection, owned by the step synthesis that needs it (callback_realization_iteration_direction: map right to left) and by the authored forms (an authored fold and fold_list left, fold_list_right right), and derived for any Loop from the realized declaration it already carries (^loop_realized_declaration_edge) by fold_realized_iteration_direction. fold_list and fold_list_right are therefore no longer indistinguishable, and a path that names no collection fold derives no direction rather than a default.",
+    "WHAT REMAINS: no executor of a fold-realized Loop exists to consume the fact. Both installed Loop handlers REFUSE: v2.extdeps.runtimes.v2_evaluator v2_eval_step_loop and v2.program program_step_loop each return a rejected outcome, so v2.compiler.eval's eval_loop_node reaches a typed refusal, never a value, and the native route executes the seed's emission of the .dag source, not this encoding. So the residual is fail-closed, not latent silent wrongness. The obligation it leaves is on the executor that replaces those refusals: it must read fold_realized_iteration_direction, not assume an order.",
+    "DISTINGUISHING FACTS: this is not a cost defect. Each synthesized step is O(1) per member (one Cons onto the carrier, never snoc or append), held structurally by v2.test.claim.compiler.collection_callback_realization ccr_map_step_conses_onto_its_accumulator; a snoc step would make map left to right at quadratic cost, which is why the order is declared rather than designed away.",
+    "RUNG: the fact is structurally carried and its derivation is enforced by discriminating controls (ccr_map_loop_iterates_right_to_left on the real map route; ccr_fold_list_and_fold_list_right_are_told_apart; both red when map or fold_list_right is declared left to right, measured). CEILING: structurally impossible once execution exists. NEXT-RUNG TRIGGER, stated as the capability: an executor of fold-realized Loops replacing the refusing step_loop handlers that consumes fold_realized_iteration_direction, SUFFICIENT FOR map, fold_list and fold_list_right encodings to evaluate in their declared order, with an executing control that reverses the declared direction and goes red.",
+  ],
+  evidence: [
+    DeclarationRef { module_path: "v2.compiler.fold_lowering", decl_name: "fold_recurrence_encoding", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.compiler.fold_lowering", decl_name: "CallbackRealization", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.compiler.fold_lowering", decl_name: "FoldIterationDirection", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.compiler.fold_lowering", decl_name: "fold_realized_iteration_direction", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.extdeps.runtimes.v2_evaluator", decl_name: "v2_eval_step_loop", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.program", decl_name: "program_step_loop", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.test.claim.compiler.collection_callback_realization", decl_name: "ccr_map_loop_iterates_right_to_left", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.test.claim.compiler.collection_callback_realization", decl_name: "ccr_fold_list_and_fold_list_right_are_told_apart", field: WholeDeclaration },
+    DeclarationRef { module_path: "v2.test.claim.compiler.collection_callback_realization", decl_name: "ccr_map_step_conses_onto_its_accumulator", field: WholeDeclaration },
+  ],
+}

dag/std/algebra.dag

@@ -394,6 +394,31 @@ fn collection_fold_shape() -> AlgebraFieldTemplate {
   { name: "fold", param_types: [ReceiverSelf, AlgebraTypeVariable { id: "FoldAccumulator" }, CallableOf { params: [AlgebraTypeVariable { id: "FoldAccumulator" }, ReceiverElement], return_type: AlgebraTypeVariable { id: "FoldAccumulator" } }], return_type: AlgebraTypeVariable { id: "FoldAccumulator" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 1 } }
 }
 
+// The collection callback operations, each ONE template declared once and referenced from every profile
+// list that carries it, as filter and fold are. A named row is what a realized call head binds to
+// (v2.compiler.fold_lowering fold_family_realization), and one row per operation removes the copy each
+// list used to hold.
+
+fn collection_map_shape() -> AlgebraFieldTemplate {
+  { name: "map", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: AlgebraTypeVariable { id: "MappedElement" } }], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } }
+}
+
+fn collection_flat_map_shape() -> AlgebraFieldTemplate {
+  { name: "flat_map", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } } }], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } }
+}
+
+fn collection_any_shape() -> AlgebraFieldTemplate {
+  { name: "any", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: NamedTemplate { name: "Bool" } }], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } }
+}
+
+fn collection_all_shape() -> AlgebraFieldTemplate {
+  { name: "all", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: NamedTemplate { name: "Bool" } }], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } }
+}
+
+fn collection_sort_by_shape() -> AlgebraFieldTemplate {
+  { name: "sort_by", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: AlgebraTypeVariable { id: "SortKey" } }], return_type: ReceiverSelf, size_effect: Present { value: IdentityEffect }, cost_shape: Present { value: ShapeSortBody }, callback_element_position: Present { value: 0 } }
+}
+
 type StepPositionScan {
   next: Int
   found: Int?
@@ -958,11 +983,11 @@ fn finite_power_set_templates() -> List<AlgebraFieldTemplate> {
     { name: "member", param_types: [ReceiverSelf, ReceiverElement], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: none, callback_element_position: none },
     { name: "contains", param_types: [ReceiverSelf, ReceiverElement], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: none, callback_element_position: none },
     collection_filter_shape(),
-    { name: "map", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: AlgebraTypeVariable { id: "MappedElement" } }], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
-    { name: "flat_map", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } } }], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
+    collection_map_shape(),
+    collection_flat_map_shape(),
     collection_fold_shape(),
-    { name: "any", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: NamedTemplate { name: "Bool" } }], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
-    { name: "all", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: NamedTemplate { name: "Bool" } }], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
+    collection_any_shape(),
+    collection_all_shape(),
     { name: "count", param_types: [ReceiverSelf], return_type: NamedTemplate { name: "std.nat.Nat" }, size_effect: none, cost_shape: Present { value: ShapeLinearScan }, callback_element_position: none },
     { name: "length", param_types: [ReceiverSelf], return_type: NamedTemplate { name: "std.nat.Nat" }, size_effect: none, cost_shape: Present { value: ShapeLinearScan }, callback_element_position: none }
   ]
@@ -1002,19 +1027,19 @@ fn free_monoid_scalar_templates() -> List<AlgebraFieldTemplate> {
 
 fn free_monoid_collection_templates() -> List<AlgebraFieldTemplate> {
   [
-    { name: "map", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: AlgebraTypeVariable { id: "MappedElement" } }], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
+    collection_map_shape(),
     collection_filter_shape(),
-    { name: "flat_map", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } } }], return_type: ContainerOf { source: SameAsReceiver, element: AlgebraTypeVariable { id: "MappedElement" } }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
+    collection_flat_map_shape(),
     collection_fold_shape(),
-    { name: "any", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: NamedTemplate { name: "Bool" } }], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
-    { name: "all", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: NamedTemplate { name: "Bool" } }], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: Present { value: 0 } },
+    collection_any_shape(),
+    collection_all_shape(),
     { name: "count", param_types: [ReceiverSelf], return_type: NamedTemplate { name: "std.nat.Nat" }, size_effect: none, cost_shape: Present { value: ShapeLinearScan }, callback_element_position: none },
     { name: "first", param_types: [ReceiverSelf], return_type: OptionalOf { inner: ReceiverElement }, size_effect: Present { value: ProjectionEffect }, cost_shape: Present { value: ShapeLinearScan }, callback_element_position: none },
     { name: "last", param_types: [ReceiverSelf], return_type: OptionalOf { inner: ReceiverElement }, size_effect: Present { value: ProjectionEffect }, cost_shape: Present { value: ShapeLinearScan }, callback_element_position: none },
     { name: "get", param_types: [ReceiverSelf, NamedTemplate { name: "Int" }], return_type: OptionalOf { inner: ReceiverElement }, size_effect: Present { value: ProjectionEffect }, cost_shape: Present { value: ShapeLinearScan }, callback_element_position: none },
     { name: "skip", param_types: [ReceiverSelf, NamedTemplate { name: "Int" }], return_type: ReceiverSelf, size_effect: Present { value: ShrinkEffect }, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: none },
     { name: "take", param_types: [ReceiverSelf, NamedTemplate { name: "Int" }], return_type: ReceiverSelf, size_effect: none, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: none },
-    { name: "sort_by", param_types: [ReceiverSelf, CallableOf { params: [ReceiverElement], return_type: AlgebraTypeVariable { id: "SortKey" } }], return_type: ReceiverSelf, size_effect: Present { value: IdentityEffect }, cost_shape: Present { value: ShapeSortBody }, callback_element_position: Present { value: 0 } },
+    collection_sort_by_shape(),
     { name: "append", param_types: [ReceiverSelf, ReceiverElement], return_type: ReceiverSelf, size_effect: none, cost_shape: Present { value: ShapeConstant }, callback_element_position: none },
     { name: "contains", param_types: [ReceiverSelf, ReceiverElement], return_type: NamedTemplate { name: "Bool" }, size_effect: none, cost_shape: none, callback_element_position: none },
     { name: "enumerate", param_types: [ReceiverSelf], return_type: ContainerOf { source: SameAsReceiver, element: TupleOf { first: NamedTemplate { name: "Int" }, second: ReceiverElement } }, size_effect: Present { value: IdentityEffect }, cost_shape: Present { value: ShapeIterateBody }, callback_element_position: none },