Skip to content

Model DGX Sparks as fabric supply: a generic owned-capacity projection and the spark join - #9534

Closed
gunbai-bot[bot] wants to merge 7 commits into
mainfrom
fabric/spark-supply
Closed

gunbai-bot[bot] wants to merge 7 commits into
mainfrom
fabric/spark-supply

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

The fabric could not choose to run work on hardware we own. dag/gunbc/spark/ converges serving on srv5 and srv6, dag/product/fabric/ ranks offers, and nothing joined them: product/fabric had zero references to spark or dgx, and product/supplier/ubicloud.dag was the only modelled SupplierOffer in the tree — a supplier we rent from. This lands the join for a supplier we are.

Where the concepts landed, and why not all of it is in gunbc/spark

Three quarters of what a Spark offer needs is true of any capacity we own, and a CI-runner onboarding of exactly this shape was starting in a peer lane while this was being written. So the projection is generic and the Spark part is a specialization:

  • product/supplier/owned_host.dag (new) — owned capacity as a SupplierOffer. Imports product.fabric.* and std only, so onboarding a runner never imports spark.
  • dag/gunbc/spark/fabric_supply.dag (new) — the GB10 specifics: shape from the cited catalog row, capability admission, readiness from the serving verdict. It reads our fleet's observation carriers, which is why it is a gunbc module and not a product/ one.
  • product/fabric/supply.dag — UnexpressedSupplyFact and SupplierOfferProjection hoisted out of product.supplier.ubicloud, which now consumes them. Neither was ever a Ubicloud fact; the second binding to need IsolationNotObserved would otherwise have imported a vendor module for a generic concept or re-minted the arm.
  • std/measure.dag — one derived unit conversion, money_per_second_for_continuous_draw, beside energy_from_power_and_time. Both operands are declared there and it joins two cited magnitudes without changing either.

The three modelling decisions worth review

The quote is an explicit zero asking price, per grant. product.fabric.selection already owns this: "The quote is a supplier-side ASKING PRICE. For owned capacity that price is an explicit zero — we do not invoice ourselves — and it would be a serious error to read that zero as the cost of running the machine." The arm is QuotedFlatPerGrant because no elapsed time is invoiced, not because the machine is already bought — supply.dag rules that reasoning out by name (owned metal is never zero at the margin; power and cooling are incurred by the act of running). Reaching the right arm through the sunk-cost argument would have put energy in the quote and charged it twice, because energy already has a home: selection axis 5, OperatingCostObservation.

A datasheet TDP does not price anything. DrawObservation has two arms carrying the same Watt, and only MeteredDraw reaches OperatingCostMeasured. A thermal design power is a cooling ceiling, so pricing it would publish a fabricated measurement wearing a real citation. extdeps.energy.nj_electricity (three cited retail rates, previously zero consumers) is the tariff side; the conversion rounds up, because understating our own running cost is the direction that lets owned capacity win a comparison it should have lost.

Offerable memory is measured or the binding refuses — and today it refuses. Two independent instances of the same trap were found while writing this. A converged Spark is serving, so a model is resident: offering the catalogued 128 GiB would offer memory occupied by the very service whose convergence is the offer's evidence. And a peer lane measured 512 GiB nominal against ~502 GiB usable on the runners. Nothing observes a Spark's free pool yet, so spark_offerable_memory_today is the unmeasured arm and no Spark offer exists — landing the measurement is what turns supply on, rather than someone substituting the catalogue to make a green appear.

Readiness is closed by construction, not by a check

SupplierOffer.ready_at is an input to a delay-valuing policy, so a guessed readiness corrupts the comparison silently and in the flattering direction. SparkServingServedProof is sole_constructor and its only producer is the converged arm of a terminal-health verdict, so bind_spark_offer cannot be handed a readiness — it is handed a health verdict, and an unconverged one has none to extract.

The generic OwnedReadiness cannot enforce that, and says so in its own annotation: it carries a Timestamp and a prose basis, and nothing generic separates a derived timestamp from an invented one. The wall belongs to each specialization. Stated out loud because a carrier that looks like it enforces provenance and does not is worse than one that plainly does not.

What this deliberately does not do

unmet_shape_axes compares envelope axes independently, so a unified-memory SoC stating its pool on both the memory and gpu axes claims it twice, and work needing most of each is declared fungible and cannot run. That is a fail-open in the fabric core for any shared-pool hardware, not a spark quirk. This PR refuses at its own seam and makes the sharing visible (UnifiedMemoryPoolNotExpressibleInEnvelope carries the pool once); it does not teach the core how to compare axes, which would be the layer inversion and would leave the fabric holding two answers. The general repair is a follow-up in product/fabric.

Also not touched: the capacity class. gunbc.fabric_capacity_class_gap's ruled remedy is ClassOnWorkShape — ClassOnSupplierOffer is explicitly refused, because it asks the party with the least knowledge and the most incentive to say yes to make the safety assertion. An Offer carries no class under that ruling, so a supplier binding is untouched by it.

One fork recorded rather than fixed: gunbc/spark has two capability carriers. gunbc.spark.capability_observation is (host × principal) grained with probe-independence admission; gunbc.spark.serving_realization carries a host-grained three-armed one that the former was written to correct. This consumes the corrected carrier and does not reconcile them.

Evidence

dag/test/claim/spark_fabric_supply_witness_test.dag, 22 rows. Every refusal is produced by changing exactly one input away from a positive control that binds, because a fold refusing everything passes any test that only checks refusals — and the subject here is a binding whose job is to refuse. The discriminating rows:

  • a self-establishing probe reporting Present still refuses — a binding reading observation.outcome directly would bind, and every other capability row would still pass;
  • the measured remainder is asserted ≠ the catalogued capacity, so the row cannot pass by coincidence;
  • the running cost is pinned at exactly 7 micros/second (6.27 rounded up) with 6 — the truncated answer — excluded, plus a second draw to rule out a constant;
  • the same 140 W refuses to price when it arrives as a datasheet ceiling: only the arm distinguishes them;
  • with three refusals available, the upstream one wins, and all four unmet capabilities are reported together.

That last row went red on the first run and found a real defect, which is the only reason it is worth having. The first cut of owned_host took a finished Shape, so the Spark binding had to destructure the capacity in order to build one before delegating — and thereby answered "the pool is unmeasured" for a host whose service account could not reach the accelerator: a specialization quietly deciding an order the generic owns. The repair removes the duplication rather than reordering it. OwnedCapacityInputs now carries shape_for: fn(ByteSize) -> Shape and unexpressed_for, so the measured bytes are read only once capability and readiness have cleared, and a specialization has no capacity to destructure.

Run: claim_batch --source-root dag --source-root src/v2 --entry dag/test/claim/spark_fabric_supply_witness_test.dag, 22/22 pass, 0 fail.

Against the carrier-only rule, stated rather than argued around

The fabric direction's rule is that no carrier-only fabric PR merges unless the same cut introduces its first production consumer, or the following cut is executable and blocked only on landing order. Where each piece stands, honestly:

  • money_per_second_for_continuous_draw — consumed by owned_host in this cut.
  • the supply.dag hoist — consumed by ubicloud and by the spark binding in this cut.
  • product.supplier.owned_host — consumed by gunbc.spark.fabric_supply in this cut. That is a production module, not a witness.
  • bind_spark_offer itself has no production caller, and I have not manufactured one. The thing that would call it is an assembly point that gathers offers for select_supply, which does not exist yet on either side — the peer lane is holding its own floor-demand/selection join for this same rule. Writing a caller here whose only purpose was to satisfy the clause would be the scaffold the doctrine forbids, and it would be a caller that today can only ever receive OwnedOfferCapacityUnmeasured, since nothing measures a Spark's free pool.

So this cut is honestly one short of the rule on its last link, and I am flagging that rather than dressing it. If the reviewer's call is that it waits for the offer-roster cut, it waits.

Reclassified as a disjoint sidecar by the fabric direction while this was being built: sparks sit outside the srv3-06 → CI → widen ordering, so nothing here is on anyone's blocking path.

gunbc-ci-auto-heal and others added 5 commits August 28, 2026 00:09
…n and the spark join

The fabric could not choose to run work on hardware we own: gunbc/spark converges
serving on srv5 and srv6, product/fabric ranks offers, and nothing joined them.

Three quarters of what a Spark offer needs is true of any capacity we own, so the
projection is generic (product.supplier.owned_host) and the GB10 part is a
specialization (gunbc.spark.fabric_supply). UnexpressedSupplyFact and
SupplierOfferProjection hoist out of product.supplier.ubicloud, which now consumes
them; neither was ever a Ubicloud fact.

The quote is an explicit zero asking price per grant, because we do not invoice
ourselves -- not because the machine is already bought, which supply.dag rules out
by name. The running cost is selection axis 5, and only a metered draw prices it:
a thermal design power is a cooling ceiling, so pricing it would publish a
fabricated measurement wearing a real citation. Offerable memory is measured or
the binding refuses, and today it refuses -- a converged Spark has a model resident
in the pool it would sell.

Readiness cannot be asserted: SparkServingServedProof is sole_constructor with one
producer on the converged arm of terminal health, so an unconverged host has none
to extract. The generic OwnedReadiness cannot enforce that and says so.

Not done here: the core compares envelope axes independently, so a unified-memory
pool stated on both the memory and gpu axes is claimed twice. This refuses at its
own seam and declares the sharing; teaching the core to compare axes is the layer
inversion.

22 witness rows, each refusal produced by changing one input away from a positive
control that binds. The refusal-order row went red first and found a real defect:
the specialization was destructuring capacity to build a Shape and thereby
answering the capacity question ahead of capability. Repaired by passing the shape
RULE rather than a Shape, so the order lives in one place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he hoist's three unadjudicated deltas

FLOOR. `product.supplier.owned_host` declared `OwnedReadiness = ReadyAt | NotReady`,
and `gunbc.pr_digests` already declares `NotReady`. Two declarations of one spelling
resolve per-entry and REFUSE over the whole corpus, reported at an innocent third
module -- the floor named `code_change_workflow_witness_test` as unresolved, which
is a consumer of the ORIGINAL. An entry-scoped witness batch cannot see this class
at all: it passed 22/22 both before and after.

Swept all 37 declarations this branch adds against the corpus; `NotReady` was the
only collision. Both arms are renamed rather than only the colliding one, so the
pair reads as one type's vocabulary and the next generic readiness word does not
lose the same race.

NAMESPACE-WAVE-ADMISSION. The hoist changes which declarations three spellings in
`product.supplier.ubicloud` admit -- `SupplierOfferProjection`,
`DiskNotExpressibleInShape`, `IsolationNotObserved` now resolve to
`product.fabric.supply`. That is `TargetChanged`, which refuses unless an exact
admission names it, and it is the intended effect of the cut. Three rows added,
named by exact subject, with the wall's own measured blast radius of zero modules.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	src/v1/stage0/src/namespace_wave_admission.rs
…clarations

A .dag change is unmeasurable until its mirror is regenerated, and the required
regen phase compares a fresh emit against what is committed. `dag/std/measure.dag`
gained `watt_seconds_per_kilowatt_hour`, `seconds_per_hour_count` and
`money_per_second_for_continuous_draw`; this is that emit, installed from the
candidate tree rather than hand-written.

NOT INSTALLED, AND DELIBERATELY: the same regen run reports drift in
v1_compiler_emit_{core_support,go,python,rust}.rs. Those are NOT this branch's --
all four are byte-identical to main here, their drift is owner-qualification of
call targets (`crate::v1_compiler_emit_core_support::` replacing a bare name),
and main's own required run fails the regen phase on exactly those four with
std_measure.rs absent from its list. Regenerating them here would absorb another
cut's artifact into this PR and duplicate work already on
session/clever-ibex-894-regen. This branch's regen phase therefore stays red until
that lands, which is the honest state rather than a green bought by doing someone
else's cut.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gunbc-ci-auto-heal added 2 commits August 28, 2026 18:30
# Conflicts:
#	src/v1/stage0/src/namespace_wave_admission.rs
# Conflicts:
#	src/v1/stage0/src/namespace_wave_admission.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant