Repository navigation
CI - #4091
Merged
Merged
CI#4091
Conversation
Address operator feedback: don't reinvent the wheel per layer (cache vs runner vs jobserver vs compiler-internal). Same Upsert<T> shape applies recursively across all of them — one pattern, one interpreter, one cache discipline. Add §4.0 establishing: - Upsert<T> as the unit at every layer (worked examples table) - srv1+srv2 = one logical computer at the modeling layer - Locality-aware resource-manager Upsert<T> sits between, owns the only knowledge of physical boundaries (L1 locality, memory pressure per box, cross-host network cost) Cross-reference §4.0 from each subsection (§4.2 caches, §4.5 runner pool, §4.6 jobserver, §4.7 locality) so they read as worked examples of the same pattern, not parallel designs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…straction Operator framing: "give me as much compute as the network wants to" — workload declares needs; the network/fabric decides scheduling/allocation. Don't pin to srv1/srv2 (or any specific physical substrate) at the workload layer. §4.0: rename "srv1+srv2 = one giant computer" framing to "workloads declare needs to a compute fabric." srv1+srv2 self-hosted is ONE instance of a ComputeFabric provider; Fargate, k8s, BuildBuddy RBE, GHA hosted are hypothetical other providers. Each declares its own capacity/locality/cost class. Workload sends opaque hints; provider decides. §4.5: rename to "Compute fabric — workloads declare needs, the network decides." Workload sends ResourceRequest + opaque hints; fabric returns a slot; workload adapts (jobserver scales to slot, doesn't pin to N cores). §4.7: locality is provider-private. Workload sends one opaque locality_hint; each provider type (stateful host-pool, stateless container, remote-execution CAS, GHA hosted) interprets or ignores per its own model. Workload doesn't switch on provider. §6: add Q7 (cross-provider L2 cache discipline) and Q8 (locality hint surface richness). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oster Operator framing: model srv1/srv2 intricately first; let the ComputeFabric abstraction emerge from comparing concrete instances. Rule-of-three (now 5) applies — don't lock in the abstraction until ubicloud, gcloud, Mac mini, and WSL are all modeled concretely. §4.0 becomes "Methodology" (renamed from "Framing"); the original framing moves to §4.0a marked provisional. New §4.0 content: - Bottom-up modeling note citing MODELING.md M9 + Cost-of-Change=1 - Intricate srv1/srv2 modeling agenda (10 structural facts not yet in ci.dag: shared $HOME, ctrl-build wrapper, ctrl-jobserver FIFO, sccache, storage topology, runner lifecycle, memory model, OS facts, failure modes) - Concrete provider roster: srv1+srv2, ubicloud, gcloud, Mac mini, WSL. Mac mini explicitly called out as the abstraction stress test (Darwin/macOS footprint forces real generality, not "two Linux pools with wrapping") - Perf-per-cost scheduling deferred as a separate meta-scheduler concern; out of scope until providers are concretely modeled §4.7 locality table updated to use the concrete roster (srv1/srv2, Mac mini, WSL, ubicloud/gcloud containers, BuildBuddy RBE) instead of hypothetical Fargate/k8s examples. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ceptance
Operator framing: don't model "machines" first. Model orthogonal physical/
execution facts (ProcessorKind / MemoryDevice / StorageDevice /
NetworkInterface / OperatingSystemSurface / ExecutionSurface) as primitives;
ComputeHost is a composition over those. "Machine" is at most a convenience
projection (MachineView), never primitive.
§4.0a Layer 3 (Demand): WorkDemand + ResourceEnvelope as orthogonal
COORDINATES of a record — not ComputeKind = CPU | GPU | Storage | Network
variants. A single workload inhabits multiple dimensions (GPU train needs
gpu + cpu + mem + storage + net). Coproduct-vs-coordinate check per
MODELING.md.
§4.0a Layer 4 (Supply): factored types replace the previous lumped
ComputeProvider { machines: List<ComputeMachine> }. ProcessorKind IS a
coproduct (a single device IS one of CPU/GPU/Accelerator); resource
REQUIREMENTS are coordinates. Eligibility via satisfies(supply, demand)
→ Witness<ComputeLeaseEligibility>, structural homomorphism.
§4.0d Forbidden: + ComputeKind coproduct, host/machine as eligibility
authority, Machine as primitive concept, scheduler fallback heuristics.
§4.0f NEW — Modeling DFS worksheet acceptance criteria: 8 falsification
cases (srv1/srv2, Mac mini, WSL, gcloud, GPU, storage-heavy, multi-match,
ineligibility) + canonical type-output list. Substrate lands in dsl/std/ /
src/v4/std/, not in src/v4/workflow/ci.dag — these are general-purpose,
not CI-specific.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Merged
3 tasks
…gonal facts Operator framing applied to caching: same "model facts, not abstractions" discipline. Don't pre-design CacheKind = GHA | sccache | BuildBuddy | Local. Model each existing cache interface as orthogonal facts; the common abstraction emerges from comparing the rows. §4.0g NEW: - Concrete cache-interface roster (GHA Actions Cache, sccache, BuildBuddy CAS, Cargo target/, ~/.cargo/registry/, rustup toolchain store, future Mac mini local sccache, future S3-backed sccache) — modeled as fact rows - Orthogonal fact dimensions (key derivation, value shape, persistence locality, eviction, atomicity, auth scope, read latency) — 4 of 4 (content-addressed × locality) combinations inhabited (independence proof) - Emergent CacheStore record (orthogonal coordinates, not kind enum) - Projection table: content_hash(canonical subject) → each backend's native key space - Derived layer-to-store mapping + cascade + two pairings (ctrl-build↔sccache, cargo-registry↔L2 mirror) §4.0d Forbidden expanded with cache anti-patterns: - CacheKind = GHA | sccache | BuildBuddy | Local coproduct (orthogonal coordinates, not exclusive variants) - GHA-specific / Rust-specific / sccache-specific semantics above CacheStore - Hand-authored cache keys at any layer (hashFiles/cache_key strings) - Bare-blob hits without CachedArtifactReceipt (kills wrong-cache-hit bug class) - Filesystem residue as proof of cache state §4.0f acceptance criteria extended (cases 9-15): - Each existing cache as a CacheStore row (GHA, sccache, BuildBuddy CAS, Cargo target/) - New cache backend addable as one fact row (no higher-layer changes) - Wrong-cache-hit protection via CachedArtifactReceipt - 4 of 4 (content-addressed × locality) combinations representable; no CacheKind coproduct anywhere Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
briansrls
marked this pull request as ready for review
May 31, 2026 06:47
briansrls
added a commit
that referenced
this pull request
May 31, 2026
… split + dispatch board + watchlist
Applies operator feedback on the dep graph refresh:
- §1 top table: P2 row renamed "TECHNICAL-PROVEN / AUTHORITY-BLOCKED" so the split between technical proof and authority deletion is visible at the top; P5 row reframed as two-bundle ("fixture/law" + "authority gate") to prevent "Wa-2 LANDED → P5 GREEN" misreads.
- §3.2 P2: explicit Layer 1 / Layer 2 framing matching the top table; P2-B authorization listed as decision-node.
- §3.3 P3 table: adds `state` column (routed / worksheet-needed / authoring / in-review / route-pending) + explanatory note on the 9-rows-for-8-classes structure cursor flagged on the prior commit.
- §3.5 P5: Layer 1 fixture/law bundle 3/3 CLOSED (Wa-2 #4080 dca5ce7 LANDED while operator was writing feedback); Layer 2 authority gate OPEN; explicit anti-misread sentence.
- §4 Path A: step 1 marked complete; structural-bridge step references #4091 §3.
- §6.5 NEW Current Dispatch Board: priority-ordered table per operator suggestion (Wa-2 done; structural-bridge DFS unstaffed; P2-B operator-gated; SG-RC-LAYERING authoring; SG-6 #4085 in cursor RC; SG-1b routed; #4091 ratification posture).
- §7: inverted "no dispatches needed" framing to name the two authoring blockers (SG-RC-LAYERING + structural-bridge replacement) as active items to monitor/escalate; operator-gated decisions surfaced.
- §8: removed Wa-2 risk (closed); added explicit "first concrete CiUpsertStep replacement under #4091" framing for the structural bridge.
- §10 renamed "Forecast (pre-bed read)" → "Watchlist (receipt-state, not time-based)" per operator's no-timeline discipline; consolidated PM-actionables into "decisions blocking forward progress" so it reads as receipt-state rather than time-adjacent.
No code paths changed; docs-only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 of 3 tasks
…l §4.0b-d Addresses cursor/composer-2.5 REQUEST_CHANGES on PR #4091: §3.4, §4.5, §4.6, §6 Q4, §7 Lane M, §8 still referenced the old per-step worker_count / timeout shape that §4.0b–d explicitly forbid. Reconciled to the canonical schema: - §2.5 infra-gotchas: timeout entry now cites ExecutionBudget split per §4.0c - §3.4 gap list: per-step timeout/worker_count entry replaced with "Declared parallelism + execution budget" — names ParallelismShape + ReducerLaws (§4.0b) and ExecutionBudget (§4.0c) as the canonical missing facts; flags that authored worker_count/timeout are forbidden (§4.0d) - §4.5 runner pool: parallelism comes from declared ParallelismShape, not authored worker_count; scheduler derives count at lease time - §4.6 jobserver: token cap derived from (ParallelismShape × provider_capacity × cache_locality), bounded by host_pool.jobserver_token_cap; no step.worker_count field - §6 Q4: reframed from "should we model per-step worker_count" to "how does ParallelismShape interact with provider capacity at lease time" (concrete: IndependentShards { shard_count: N } vs M < N free slots) - §7 Lane M: aligned with §4.0a canonical schema (WorkUnit, ComputeDemand, ComputeSupplyFacts, ComputeOffer, ComputeLease, ExecutionReceipt, ParallelismShape + ReducerLaws, ExecutionBudget) instead of worker_count/timeout - §8: prerequisite list now names ExecutionBudget / ParallelismShape + ReducerLaws instead of bare timeout/worker_count Doc now has one authoritative shape end-to-end. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
briansrls
added a commit
that referenced
this pull request
May 31, 2026
… receipt quality; split worksheets
Operator feedback on the §4.0g iteration: CacheStore was named like an
authority but methodology says it should EMERGE from concrete facts. Plus 8
specific additions to make the cache substrate honest and the receipt
substrate audit-able.
§4.0g cache section:
- Rename CacheStore → CacheInterfaceFacts (concrete row; lands first) +
CacheStoreView (derived projection; never first authority — same
discipline as MachineView in §4.0a Layer 4)
- Add vendor-fact discipline: rows must eventually carry cited source OR
runner-observed receipt; asserted-without-evidence forbidden at ratification
- Add ArtifactIdentity<T> (semantic) vs BackendCacheKey (transport)
separation, with explicit CacheKeyProjection<T> as the mapping. Prevents
backends from becoming identity authorities.
- Add CachedArtifactReceipt<T> as first-class type (was only mentioned in
forbidden list); requires verified_subject_digest + content_digest match
- Add CacheLookupSemantics / CacheWriteSemantics / CacheMissSemantics —
restore is not just key derivation; backends differ sharply in behavior
- Replace abstract KeyDerivation = {ContentAddressed | HandAuthored |
NativeInternal} with concrete KeyDerivationFacts: classification +
inputs_considered + overwritable + prefix_fallback_allowed +
content_verified_on_read + visibility_scope + invalidation_triggers
- Update layer-to-store mapping table and ctrl-build/Cargo registry pairings
to reference CacheInterfaceFacts rows (gha_actions_cache_facts,
sccache_local_facts, buildbuddy_cas_facts, cargo_target_dir_facts,
rustup_toolchain_store_facts)
§4.0a Layer 2 — add ingress-boundary discipline: ArtifactRef is the SCHEDULED
form, but the dependency model still starts from UpsertInputRef → ChangeSet
∩ FileSetSelector/NodeQuery → materialized ArtifactRef → WorkUnit input.
Don't erase that boundary too early — auditability requires the chain.
§4.0a Layer 5 — enrich PerformanceReceipt (sample_count,
measurement_context, confidence) + CostReceipt (pricing_source,
amortization_scope). Mechanically-checkable bar for perf-per-cost ranking:
all candidate offers must carry receipts with confidence ≥ Range (or
similar threshold, operator-bar).
§4.0f — split single worksheet into TWO orthogonal worksheets:
- Worksheet A: Compute fabric facts (cases 1-8) — type outputs in dsl/std/
- Worksheet B: Cache interface facts (cases 9-19) — type outputs in dsl/std/
- Independence acceptance: A and B compose ONLY through
ExecutionReceipt<T>.output and CachedArtifactReceipt<T>.producer; no other
cross-coupling.
- Added cases 17 (same ArtifactIdentity → different BackendCacheKeys),
18 (toolchain version change → new identity), 19 (vendor row evidence).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
briansrls
added a commit
that referenced
this pull request
May 31, 2026
… snapshot + #4091 anchor (#4094) * docs(planning): v4 dep graph refresh 2026-05-31T06:30Z — post-cascade snapshot + #4091 anchor Forward-projection update to docs/planning/v4-predicate-dependency-graph-2026-05-30.md (#4058) for the 2026-05-31 wave-collapse window. Captures: P2 scope (a) PROVEN via P2-A probe (44-source compiler closure, 0 diagnostics, no bridge); P3 7951→6991 rustc with 8 receipt-producing classes routed per #4086 catalog; P5-PROVEN 2/3 gates closed (Wa-1 #4079 + P5-D tranche-2 #4089 LANDED); Wave 1 EXIT achieved via #4082. Names Path A (P5 final-flip) as shortest critical path forward; identifies P2-B bridge deletion + structural-bridge replacement as latent operator gates. References #4091 elastic CI redesign exploration at §3.5 / §6 / §8 / §9 / §10 so any structural-bridge replacement authoring aligns with the in-flight design rather than being re-shaped post-ratification. Definitions cited per src/v4/TASKS.md:806-817. Authoritative state defers to merry-badger burn-down, sharp-otter MW-D8 ledger + P1 roster + post-SG-1 catalog (#4086), and quick-tern's tranche state. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(planning): #4094 tightening per operator feedback — P5 two-layer split + dispatch board + watchlist Applies operator feedback on the dep graph refresh: - §1 top table: P2 row renamed "TECHNICAL-PROVEN / AUTHORITY-BLOCKED" so the split between technical proof and authority deletion is visible at the top; P5 row reframed as two-bundle ("fixture/law" + "authority gate") to prevent "Wa-2 LANDED → P5 GREEN" misreads. - §3.2 P2: explicit Layer 1 / Layer 2 framing matching the top table; P2-B authorization listed as decision-node. - §3.3 P3 table: adds `state` column (routed / worksheet-needed / authoring / in-review / route-pending) + explanatory note on the 9-rows-for-8-classes structure cursor flagged on the prior commit. - §3.5 P5: Layer 1 fixture/law bundle 3/3 CLOSED (Wa-2 #4080 dca5ce7 LANDED while operator was writing feedback); Layer 2 authority gate OPEN; explicit anti-misread sentence. - §4 Path A: step 1 marked complete; structural-bridge step references #4091 §3. - §6.5 NEW Current Dispatch Board: priority-ordered table per operator suggestion (Wa-2 done; structural-bridge DFS unstaffed; P2-B operator-gated; SG-RC-LAYERING authoring; SG-6 #4085 in cursor RC; SG-1b routed; #4091 ratification posture). - §7: inverted "no dispatches needed" framing to name the two authoring blockers (SG-RC-LAYERING + structural-bridge replacement) as active items to monitor/escalate; operator-gated decisions surfaced. - §8: removed Wa-2 risk (closed); added explicit "first concrete CiUpsertStep replacement under #4091" framing for the structural bridge. - §10 renamed "Forecast (pre-bed read)" → "Watchlist (receipt-state, not time-based)" per operator's no-timeline discipline; consolidated PM-actionables into "decisions blocking forward progress" so it reads as receipt-state rather than time-adjacent. No code paths changed; docs-only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
3 tasks
briansrls
added a commit
that referenced
this pull request
May 31, 2026
… graph refresh (#4096) Reflects two state changes from 2026-05-31T06:28-07:04Z: 1. **#4091 elastic CI redesign MERGED** (squash c05a5a8, 07:04Z) — flips from "draft / exploration" to ratified substrate authority across §3.5 (P5 Layer 2 framing), §6 (blockers table), §6.5 (dispatch board priority 7), §7 (operator-gated decisions list), §8 (risk section), §9 (cross-refs), §10 (watchlist). 2. **Wave 2 TR-lane COMPLETE** — SG-6 #4085 MERGED 06:28Z (squash 2ac52f2); lively-eagle archived. Updates §6.5 priority 5 from "cursor RC" to "LANDED ✓ + Wave 2 TR-lane COMPLETE". 3. **proud-pike scope clarification (msg_6287177d)**: #4095 elastic compute fabric DFS worksheet is scoped to `dsl/std` + `v4.std` substrate vocabulary only (per #4091 §4.0f/§4.0g); explicitly out-of-scope for `ci.dag`/`ci.yml`/shell bridge deletion. Separate downstream §10.0 worksheet still required for structural-bridge replacement — the "first concrete CiUpsertStep replacement under #4091" per operator framing. §3.5 + §6 + §6.5 updated to reflect. 4. **Operator decisions resolved per #4094 review**: D3 (SG-RC vs SG-1b prioritization) = PARALLEL (proud-pike confirms separate sibling worksheet, not amendment); D5 (CI risk posture) = operational nuisance NOT merge-blocker until ci_v4 fail-rate climbs above ~1-in-3. 5. **sharp-otter M2 probe dispatched** as P2-B safety-net receipt (full corpus compile w/o bridge step) so operator has artifact when authorizing P2-B deletion. No code paths changed; docs-only. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
4 tasks done
4 of 5 tasks
briansrls
added a commit
that referenced
this pull request
May 31, 2026
…isions resolved Addresses cursor REQUEST_CHANGES on #4096: 1. §3.3 SG-6 row: in-review → merged (#4085 LANDED 06:28Z squash 2ac52f2; lively-eagle archived). 2. §5 Parallelizable workers: lively-eagle row updated to "LANDED + archived"; proud-pike row updated to reflect 3-parallel-front authoring (bridge + 2 child workers); deep-ferret marked ARCHIVED with respawn work-item adhoc-6b440f7f-6de noted. 3. §6 blockers table: removed stale "#4085 SG-6 cursor RC" row; SG-RC-LAYERING row updated to note PM-override-spawned child workers. 4. §3.3 state legend: added `merged` enum value. 5. §10 watchlist: split "decisions blocking" into "blocking" + "resolved this snapshot"; moved #4095-scope clarification, SG-RC-vs-SG-1b prioritization, CI risk posture, and #4091 ratification into resolved. Only P2-B authorization remains as blocking (with M2 probe receipt path in flight). INVARIANTS.md P2 single-authority discipline restored: SG-6 receipt has one authority statement (LANDED) across all sections. Per PM override (msg_9640253b) + proud-pike ack (msg_cdbe1925): 2 child §10.0 workers spawned (sharp-lark-878 SG-RC, sleek-badger-874 SG-1b) parallel with proud-pike's own bridge authoring + sharp-wolf #4095 review. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 31, 2026
briansrls
added a commit
that referenced
this pull request
May 31, 2026
briansrls
added a commit
that referenced
this pull request
May 31, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
briansrls
added a commit
that referenced
this pull request
Jun 1, 2026
This was referenced Jun 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-opened by session-dashboard for session
deep-badger-883.Pushing to
session/deep-badger-883advances this PR.Worker attestation
Before flipping this PR to ready for review, confirm each item:
npm test,cargo test) and the result.Closes #Ndirective.Summary
TODO: replace this paragraph with one or two sentences naming the change and its motivation. Reviewers read this first.
Test plan