Repository navigation
The prefill budget is 2048, an exception handler chose it, and the counter that refuted it was never a step counter - #10813
Merged
Merged
Conversation
…unter that refuted it was never a step counter
Investigation receipts for the evidence/numerical lane, reconstructed on current
main rather than on the serving-prefill-scaling branch, whose commit descriptions
still assert the conclusions this work overturns.
WHAT WAS MEASURED. Four launches on the group A head, same harness, idle engine,
unique nonce per prompt. Group A was restored to its authored unit afterwards and
group B was never touched.
budget steps 16.4k prefill co-tenant stall KV pool
2048 9 17759 ms 2168 ms meas 11,767,856
8192 3 16652 ms 8225 ms meas 8,704,293
16384 2 16327 ms 15930 ms meas 5,836,755
Three-term fit over 20 clean probes, wall = 105.8 + 189.9*steps + 0.9661*tokens
(rms 82 ms). The stall law S + b*budget predicts every measured arm within 1%.
THE PER-TOKEN COST IS INVARIANT over a 32x range of batch size, so the ~1 ms/token
is not a batching, scheduling or MoE-granularity artifact. Going from 48 to 384
tokens per routed expert per step moved it ~2%.
WHY THE BUDGET IS 2048. nvmlDeviceGetMemoryInfo returns NotSupported on a GB10's
unified memory; vLLM catches it with a bare `except Exception: device_memory = 0`,
so 0 >= 70 GiB fails and a 121 GB machine takes the small-device default. Torch
reads the memory correctly; only the NVML path fails. DESIGN §5 absorbing fallback.
WHY THE REFUTATION WAS WRONG. iteration_tokens_total_count advances once per
EngineCoreOutputs batch carrying an output, not once per engine step, so a
prefill-only request advances it exactly once regardless of chunk count. Measured
directly: iter_delta == 1 on requests the per-step counter measured at 10 steps.
The honest step counter is estimated_flops_per_gpu_total under --enable-mfu-metrics.
AND THE 83-ITERATION NUMBER WAS NEVER FABRICATED. A co-tenant decoder beside a
chunked prefill receives one token per chunk, so the quotient approximates the chunk
size by construction. 1,913 ~ 2048 was not a coincidence; the original inference was
right by a route it misdescribed, and was then withdrawn as wrong. The specimen is
that a wrong-grain quotient can land near the true answer, which is what makes it
unfalsifiable by inspection -- first as apparent confirmation, then as refutation.
No carrier is changed here. This commit preserves the findings and the raw sweep
receipts so the corrected carriers can be built from them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
… an exception handler picked the budget
The corrected evidence carriers for the prefill work, authored on current main from
the receipts in the previous commit rather than on serving-prefill-scaling, whose
commit descriptions still assert what this overturns.
WHAT WAS WRONG. gunbc.spark.serving_critical_path carried
BehaviouralPerIterationCapacityAtLeast{20082}, established by polling
vllm:iteration_tokens_total_count on an idle engine and seeing every window from
512 to 20,082 computed tokens complete in "one iteration". That counter is not a
step counter on any engine: async_llm.py builds its stats object as
`IterationStats() if (log_stats and num_outputs) else None` and the recorder
returns early on None, so it advances once per output batch. A chunked-prefill
step emits no output for the request it is advancing, so a prefill-only request
advances it EXACTLY ONCE however many chunks it ran -- measured at 1 for requests
a true per-step counter measured at 3, 4, 7, 9 and 10 steps. The reading was
invariant across the whole sweep and carried no information about the budget.
Exclusivity is what made the withdrawal look safe, and exclusivity was never the
defect. It answers WHOSE events are counted; it says nothing about WHICH.
WHAT IS TRUE. The effective budget is 2048, established by three instruments that
do not share the defect: estimated_flops_per_gpu_total under --enable-mfu-metrics
advances once per engine step and its counts came out as exactly ceil(tokens/2048)
at seven sizes; the engine's own banner logs compile_ranges_endpoints [2048], which
_set_compile_ranges sets FROM scheduler_config.max_num_batched_tokens; and
inter_token_latency_seconds held 1,454,342 samples over 45 hours with none above
7.5s, which alone refutes a 20,082-token step.
WHY IT IS 2048. Not because anyone chose it. nvmlDeviceGetMemoryInfo returns
NotSupported on a GB10's unified memory, vLLM catches it with a bare
`except Exception: device_memory = 0`, and a 121.7 GiB machine takes the
small-device branch. An absorbing fallback, filed as a receipt on that class.
AND 1,913 WAS NEVER FABRICATED. A co-tenant decoder beside a chunked prefill gets
one token per chunk, each token one output batch, so the engine-wide quotient
approximates the chunk size by construction. The original inference was right by a
mechanism nobody identified, then discarded for a worse reason. The specimen is
that a wrong-grain quotient can land NEAR the true answer, which is what makes it
unfalsifiable by inspection -- first as confirmation, then as refutation.
CARRIERS. extdeps.nvidia.management_library models the device-memory query so that
its refusal arm carries NO byte count, making `device_memory = 0` unwritable rather
than discouraged. extdeps.vllm.batch_defaults models upstream's resolution and
keeps the SELECTION CAUSE beside the value, because 2048-because-small and
2048-because-absorbed behave identically and have different fixes.
extdeps.vllm.metrics pairs each metric with a QUESTION and refuses the pairings the
installed source does not support, including this one.
gunbc.spark.prefill_batch_sweep carries the four-launch sweep with MeasuredBatchArm
and PredictedBatchArm as INCOMPATIBLE types -- the 1024 and 512 arms were never run
and must not be readable as observations.
The extdeps do-not-set row is withdrawn as a layer inversion, not only as a wrong
claim: whether to set the flag trades interactive latency against throughput
against KV pool, which is product policy. Its "reduces the KV pool" half was true
and measured (11.77M -> 5.84M tokens); its ground for the other half was inferred
from average link utilisation. The optional prefill_scheduling field lands WITH its
consumer in the argv renderer rather than dangling.
Group A was restored to its authored unit; group B was never touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…numerically interchangeable
The batch axis could not move the per-token term at all. Collective topology moves
it, which makes this the first mechanism found that reaches the ~1 microsecond per
computed token, and the largest single lever measured on this fleet.
HELD CONSTANT: same head, image, container, model snapshot, tokenizer, executor,
backend selections and cache standing; batch budget PINNED EXPLICITLY at 2048 in
both arms; MFU metrics on in both; one context ceiling of 131,072 for both, lowered
from the production 1,048,576 because a TP=2 rank holds roughly half of a ~168.7 GB
model against a ~99.8 GiB budget. Only the tensor-parallel degree differs.
quantity TP=4 TP=2 TP=2 against TP=4
marginal per token 1.053 us 0.786 us 25.4% cheaper
prefill throughput 949 tok/s 1272 tok/s 34% higher
decode alone 30.2 ms 40.5 ms 34% WORSE
co-tenant stall 2146 ms 1629 ms 24% shorter
KV pool at 131,072 6,175,555 2,310,685 63% LESS
The gain holds at every probe above 2,048 tokens across a 40x range and is 7% at
510, where the fixed per-request term still dominates. Step counts from the per-step
FLOPs counter matched ceil(tokens/2048) in both arms, so the budget really was
pinned and the comparison really is topology.
THE TWO WORKLOADS WANT OPPOSITE TOPOLOGIES. Decode is 34% worse at TP=2 -- decode is
memory-bound per token and fewer ranks means less bandwidth to stream weights, while
prefill pays for every extra collective participant. An adoption is therefore a
decision about what a replica is FOR, not a tuning change.
AND THEY ARE NOT NUMERICALLY INTERCHANGEABLE. A greedy text comparison would have
been worthless: the arms diverge at the first token from a near-tie. The measurement
is prompt logprobs over one fixed 22-token input -- one deterministic forward pass,
no sampling. Same token sequence, but per-token logprobs differ by up to 1.517 nats
(a 4.56x ratio on one token's probability) with a mean of 0.330. That is orders of
magnitude above reduction-order noise: the sharding changes the computation, not its
rounding. Causes are UNSEPARATED -- per-shard fp8 accumulation, routed-expert
selection under a different partition of 256 experts, or a shape-dependent kernel.
So the carrier gates on it. `topology_substitutable` REFUSES on divergence rather
than warning, because a 25% speed win may not silently carry a numerical change into
production. The speed result and the equivalence result are different subjects and
only the second decides substitutability.
MISSING CONTROL, NAMED RATHER THAN GLOSSED: no arm was repeated, so within-topology
run-to-run variation is unmeasured and the 25.4% carries no interval. Whether TP=2
fits the production ceiling at all is likewise untested.
Group A was restored to its authored unit and verified: no drop-in, tensor_parallel
4, max_seq_len 1048576, enable_mfu_metrics False, KV 12,739,744 tokens. Group B was
never touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…pology caused the numerical divergence Two defects in the carriers landed by the previous two commits, both found in review. THE UNITS ERROR IS THE SERIOUS ONE, and it is the same class the branch had already repaired once. FittedExecutionLaw.micros_per_thousand_tokens carried 966 for a rate of 0.9661 MILLIseconds per token. 966 micros per thousand tokens is 0.966 microseconds per token, which is a million tokens per second. The field was wrong by exactly 1000, and both predicted_wall_micros and predicted_step_micros returned values three orders of magnitude too small. It survived authoring because 966 and 966100 are both plausible small integers beside a field name that does not say which. It is caught now by a derivation rather than by care: at the corrected 966100, predicted_step_micros(budget 2048) is 2,168,472 micros against an INDEPENDENTLY MEASURED co-tenant stall of 2,168,200 -- agreement to 0.013%, where the wrong value misses by 1000x. The topology carrier had the same error (1053 and 786 for 1.053 and 0.786 ms/token) AND a second representation of the same rate stored beside it as prefill_tokens_per_second. That is the §3 fork that let the error hide: two stored spellings of one quantity, neither checking the other. Throughput is now DERIVED from the rate, so a wrong-by-1000 value announces itself immediately. THE SECOND DEFECT IS AN OVERCLAIM. The module said the logprob divergence showed "the sharding changes the computation, not merely its rounding". That is a causal claim the population cannot carry: WITHIN-topology variation was never measured, so there is no envelope to compare the cross-topology distance against. The variant is renamed CrossTopologyDifferenceObservedWithinTopologyUnmeasured and carries the minimum design that would earn attribution -- three same-launch repeats plus one relaunch per degree, over identical token ids, comparing within and across before across-topology. The refusal arm now also fires on the unmeasured control, because an unexplained difference and an unmeasured baseline are both reasons not to substitute. ALSO CORRECTED, all from the same review: - "measured, not adopted" read as a verdict on TP=2's fitness. What the evidence refuses is FUNGIBILITY -- serving one alias from either topology as though a request could not tell them apart. A separately named profile is a different proposition and nothing here rules it out. The standing is renamed accordingly. - The per-replica view is not the fleet decision. At a fixed four-Spark footprint the alternative to one TP=4 replica is TWO TP=2 replicas, and the capacity story changes magnitude: 63% less KV per replica becomes about 25% less across the same four Sparks. Both views are carried; each is misleading alone. The aggregate decode figure is throughput, not latency -- a user on one stream still feels 40.5 ms. - The topologies do not force a global choice. TP=2 saves 267 micros per computed prompt token and pays 10,300 per generated token, so it wins whole-request time above about 38 prompt tokens per generated token. A live 60-second window on this head measured about 280 to 1, an order of magnitude above the crossover. - A printed pool size is not an admission pool, and summing two of them does not make one. Carried as a caveat on the aggregate. - replicas_in refuses on a zero rank population rather than answering 0, which would have flowed into the aggregates as a real zero. No new measurement was taken; the fleet was not touched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…ence IS the topology, and prefill is quadratic in prompt length
Three results from the repeat design and the production-ceiling gate, one of which
withdraws this lane's own headline.
THE MAGNITUDE WAS WRONG. Four alternated launches, two per degree, three cold 16k
probes each:
TP=4 across launches: 0.9398 .. 0.9835 ms/token spread 4.6%
TP=2 across launches: 0.8344 .. 0.8414 ms/token spread 0.8%
Effective at 16k that is 938 against 1016 tok/s -- TP=2 is 7.7% cheaper per token,
NOT the 25.4% this lane reported. The first TP=2 launch measured 0.786 and was never
repeated; both repeats land at 0.98 on two DIFFERENT Spark pairs, so that reading is
an unexplained outlier. TP=4 was stable across every launch, which is why the error
was one-sided. 8% is outside the 4.6% envelope but not hugely so.
THE NUMERICAL ATTRIBUTION IS NOW EARNED, and it is the strong direction. Prefix-cache
hit delta was zero on every numerical probe, so each reading is a real recomputation:
within-launch, all four launches 0.0000 nats
across-launch, same topology 0.0000 nats
cross-topology, all four pairings 2.2387 nats
The within-topology envelope is BIT-IDENTICAL across a fresh relaunch and across
different hardware -- Ray placed the two TP=2 launches on different Spark pairs and
they agreed to the last bit. All four cross pairings give the same 2.2387. That is a
deterministic function of the degree, not noise. The refusal is now HARDER than an
unexplained one: the difference is reproducible and will not average out. Which
degree is closer to the intended semantics remains unanswered and needs a reference.
TP=2 CLEARS THE PRODUCTION CEILING. At max_model_len 1,048,576 it started, reported
4,823,795 KV tokens at 4.6x concurrency, correctly refused a 1,048,576-token prompt
because prompt plus output must fit, and admitted 1,048,575 tokens in 1598.1s at
1.5241 ms/token mean, in EXACTLY 512 steps with zero preemptions. 1,048,575 / 2048 =
512.0 -- the batch budget confirming itself at the largest scale the model admits.
AND THE AFFINE LAW ONLY DESCRIBES SHALLOW CONTEXT. Sampling KV allocation through
that 26-minute prefill gives a per-token cost AFFINE IN DEPTH, residuals <= 0.05 ms
over a 123x range:
ms/token = 0.9236 + 1.4159 microseconds per million tokens of depth
so total prefill is QUADRATIC in prompt length, and the fleet's ~149k average sits an
order of magnitude outside the range every earlier law was fitted on. The consequence
for the batch budget is structural: chunking does not change which token pairs are
scored, only how they are grouped, so the quadratic term is INDEPENDENT of the budget
and the budget only ever moves ceil(tokens/budget) * per_step. That is why the
measured gain from 2048 to 16384 was at most 7.5% and could not have been more. It
also means the co-tenant stall GROWS with the offending prefill's depth -- about 2.1s
at 16k against 5.1s near 1M at budget 2048.
Which subsystem carries the depth term is NOT established; a cost linear in prior
context is what a key-selection scan looks like, which is a hypothesis for the
operation ladder and not a finding.
WITHDRAWN WITHOUT REPLACEMENT: a per-step cost broken out by topology. All four
repeat launches held the budget at 2048, so steps ~ tokens/2048 and the per-step and
per-token terms are collinear; only the batch sweep, where the budget varied at fixed
token count, separates them. The decode and co-tenant-stall rows also remain single
measurements and carry no envelope.
Group A restored to its authored unit and verified: no drop-in, tensor_parallel 4,
max_seq_len 1048576, enable_mfu_metrics False, KV 12,761,995. Group B never touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…model swap carries them as law
Corrections from review. Each is a claim of mine that was true for the profile it
was measured on and would be wrong the moment it is carried past it.
THE CO-TENANT LAW WAS THE ONE-ACCEPTED-TOKEN SPECIAL CASE. This module derived
"a co-tenant decoder receives exactly one token per chunk, so its inter-token
latency IS the step duration" from a profile serving no speculative decoding. Under
speculative decoding a single step can accept a BURST, so the general relation is
victim progress per step = accepted decode tokens from that step
and a profile with a draft model can suffer the same step duration with a materially
better felt cadence. The mechanism survives a model change; the equation does not.
The batch-policy trade measured beside it inherits the same caveat and it is now
stated: draft tokens consume the same budget, so a budget measurement taken with
speculation OFF does not describe a product served with it ON.
THE NVML FINDING IS PLATFORM TIMES RUNTIME REVISION, NOT THE HARDWARE ALONE. What
was established is that THIS vLLM revision, on a GB10, resolves the batch default
through a device-memory probe it catches and zeroes. The path does not consult the
model, so it should reproduce for any checkpoint served by the same build -- and a
different image must REACQUIRE it rather than inherit it. That is the rule the
cumulative_metric_read_as_per_event row already states for metric semantics, and my
own summary broke it. The durable remedy was never knowing the default; it is
pinning the budget explicitly so no default is consulted.
THE PER-STEP FLOPS QUANTUM IS NOT A CONSTANT TO CARRY. The counter advances by a
fixed amount per step for a GIVEN model and topology because it estimates that
model's work per rank -- it was observed to DOUBLE when the rank count halved. A
divisor carried across a model or topology change silently reports the wrong step
population, which is the same wrong-grain reading this corpus already has a failure
mode for.
WHAT IS NOT HERE, AND WHY. I had also written a context-window decision carrier with
its own serving-profile identity and candidate selector. It is deleted rather than
landed. The seams it reached for are already owned: pair_serving_capacity_floor
declares maximum_sequence_tokens 500000 beside an imported seat count, which IS a
context and concurrency policy expressed as a literal, so a second selector would
have been a second answer to one question -- the §3 fork, authored by me, one turn
after I wrote a scope note in that very module warning about it. The context
decision belongs downstream of a reconstructed observation substrate and consuming
its profile type, not beside it.
No measurement changed and the fleet was not touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…ese numbers expire with the profile Two corrections found while checking this branch against the disposition on the other lane's PR. PREFILL-FINDINGS.md SHOULD NOT BE IN THIS REPOSITORY. It is an internal RCA worksheet at the repository root, and the standing operator ruling after #4192 and #4200 is that planning and RCA documents do not land here -- worksheets are delivered as receipts. The findings themselves are not lost: they are in the typed carriers, which is where DESIGN section 4c says they belong, and the readable summary is the pull request body. The raw measurement receipts stay under docs/receipts, because the carriers CITE them as the producer of their fitted coefficients and a citation that does not resolve is worse than none. AND THE MEASUREMENT ROWS NOW SAY WHOSE PROFILE THEY DESCRIBE. Every number in prefill_batch_sweep and tensor_parallel_comparison was taken against one artifact -- DeepSeek-V4-Flash at snapshot 60d8d707 on one vLLM revision -- and that artifact is being replaced. The rows carry an explicit statement that every coefficient expires with it: per-token and per-step terms, the depth law, the KV pools, the decode and stall figures, the per-step FLOPs divisor. What survives is the MECHANISM each number established, which is already stated in prose beside it. THE STRUCTURAL VERSION OF THAT IS NAMED AS AN OBLIGATION AND DELIBERATELY NOT BUILT. A measurement should carry its model artifact, tokenizer, runtime, KV policy, topology and speculative policy in a TYPE, so a row from one profile cannot be joined to a decision about another -- the defect gunbc.spark.vllm_serving_launch prevents for process incarnations, one level up. That type belongs with the serving-observation substrate. Coining it here would be a second profile authority, which is exactly the fork this lane already made once and reverted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…nstructed # Conflicts: # dag/gunbc/recurring_failure_mode/roster.dag # docs/design-failure-modes.md
Ledger-Repair-Judged: docs/design-failure-modes.md Ledger-Rows-Repaired: docs/design-failure-modes.md absorbing_fallback Ledger-Rows-Repaired: docs/design-failure-modes.md cumulative_metric_read_as_per_event Ledger-Repair-Judged: docs/design-rung-drops.md
Contributor
Author
Proposal for the fabric lane: warmth and co-tenancy as selection inputsPosting here because this PR carries the calibrations, and because the conclusion is don't build any of it yet — the axes exist, the consumer doesn't. The finding: the economics vocabulary is already on main
with So this is wiring, not invention. Three corrections to my first proposal1. Predicted warmth must NOT construct For the relief cut, keep it strict — exact warmth observed, or the transition axis stays unpriced. Incumbency may be a deterministic tie-break outside the measured-cost claim. Only introduce an estimated-transition arm if estimated transition cost has several real consumers. 2. One operating rate does not monetize both axes, and there is a double-count risk. If grant duration already spans the cold prefill and Co-tenancy is a different quantity entirely — displaced service value, not electricity. Do not monetize the 72x yet. It is strong enough as an admissibility rule: a fresh cold-risk prefill beside an active interactive decoder is inadmissible unless the serving policy explicitly permits that service-class degradation. Soften to opportunity pricing later, if a defensible SLO function makes the allocation better. 3. The Calibrations here are profile-bound, and they do not even share one profileWorth stating plainly, because it makes the point sharper than "these are DeepSeek numbers":
Different topologies. They must not be mixed with each other, let alone carried to GLM. So the cost models should be sealed, profile-bound objects — Order of workFirst, and it needs none of the economics: launch-scoped grant invalidation. Engine death invalidates the realization, not the Work, and must not poison the executor identity: Fenced by exact launch identity: With the streaming split, which cannot be one fact: Evidence this matters: the group A head log carries 25 raylet terminations and 17 engine restarts. Each restart is currently converted 1:1 into permanently failed roadmap nodes, with Second: the grant-consuming router. Until the inference path actually goes Demand -> offers + DispatchInputs -> selection -> grant -> endpoint-from-grant, none of the cost work is load-bearing. The harness talks to Third, and only then: attach exact-profile transition and opportunity observations to the existing axes. Acceptance criteria should be behaviour, not compilationKnown observability gapPer-request warmth is not readable on this build — What I am deliberately not buildingThe two cost models, until the router exists. A calibrated model with no consumer is dangling modeling, and this lane has already made that mistake once in this investigation. 🤖 Generated with Claude Code |
…nstructed # Conflicts: # dag/gunbc/recurring_failure_mode/roster.dag # docs/design-failure-modes.md
Ledger-Repair-Judged: docs/design-failure-modes.md Ledger-Rows-Repaired: docs/design-failure-modes.md absorbing_fallback Ledger-Rows-Repaired: docs/design-failure-modes.md decision_surface_truncation Ledger-Rows-Repaired: docs/design-failure-modes.md cumulative_metric_read_as_per_event Ledger-Repair-Judged: docs/design-rung-drops.md
…failed device probe
Operator decision, 2026-09-08. The value does not change; its PROVENANCE does.
WHAT WAS HAPPENING. On a GB10 the NVML device-memory probe returns NotSupported,
vLLM's `except Exception` writes device_memory = 0, and a 121.7 GiB machine takes the
small-device branch to max_num_batched_tokens 2048. Nobody chose that number, and it
moves the moment the probe, the driver or the image changes. Requesting 2048
explicitly was MEASURED to be a no-op -- an explicit-2048 launch rendered
compile_ranges_endpoints [2048], identical to the implicit one -- so this is
provenance, not behaviour.
WHY 2048 AND NOT THE 8192 UPSTREAM WOULD HAVE PICKED, measured on this fleet at a
16.4k cold prefill:
budget prefill wall vs 2048 co-tenant stall KV pool
2048 17,759 ms - 2,168 ms 11,767,856
8192 16,652 ms -6.5% 8,225 ms 8,704,293
16384 16,327 ms -7.5% 15,930 ms 5,836,755
The gain is pure per-step amortisation and cannot exceed ~7.5%, because per-token
cost is FLAT from 512 to 16,384 tokens per step -- bigger batches do not make the
routed-expert GEMMs more efficient here. The sharper cost is not the stall but the KV
pool: -26% at 8192 is roughly 85 -> 63 warm conversations per replica at this fleet's
average context, and a conversation that falls out of the warm set recomputes its
whole prompt.
CHUNKED PREFILL IS PINNED ALONGSIDE IT because the budget is only meaningful under
chunking. Pinning the budget while inheriting the chunking mode pins half a policy,
and a future default change would silently make the pinned number mean something
else. Also a no-op today: the engine already resolves enable_chunked_prefill=True.
THE OPTIONAL FIELD HAD NO PRODUCER UNTIL NOW, which was a gap in the change that
introduced it. Both construction sites passed `none`, so the renderer's Present arm
was unreachable: it typechecked, it was imported, and nothing could ever emit the
flag. Consumption is not "a consumer exists" but "a consumer is reached".
THE WITNESS IS DISCRIMINATING, and its RED was executed rather than assumed. The
positive asserts the rendered argv carries --max-num-batched-tokens 2048; the
negative asserts an unrequested policy emits NO batch flag at all, so a renderer that
hard-coded it would fail. Planting the desired row back to `none` and re-running the
rendered-argv check exits 1 and prints the argv without the flag; restoring it exits
0. Rendered tail as pinned:
... --gpu-memory-utilization 0.82 --enable-prefix-caching
--max-num-batched-tokens 2048 --enable-chunked-prefill
SCOPE. These measurements are DeepSeek-V4-Flash on one vLLM revision. A different
model or image must re-measure rather than inherit the number, and must re-establish
that the NVML path still resolves the same way -- and with speculative decoding
enabled, since draft tokens consume this same budget.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…into serving-evidence-reconstructed
# Conflicts: # dag/gunbc/recurring_failure_mode/roster.dag # docs/design-failure-modes.md
…opulation producing two confident coefficients Closing the transcription gap this lane declared and then widened. The coefficients were literals copied out of a Python script, beside an annotation admitting that "editing them here would not be caught by anything" -- and that is exactly what happened. Asked how the reported ratios were calculated, the answer could not be read off the carrier: it needed reopening the script, where the percentages sitting beside measured wall times turned out to be MODEL PREDICTIONS at a normalised token count. THE SAMPLES ARE NOW THE AUTHORITY. Twenty clean probes are carried and every coefficient, prediction and ratio is a fold over them. Adding a probe moves the law; editing a coefficient is impossible because there is no coefficient to edit. The one contaminated probe -- 81,697 computed tokens booked against a 20,262-token request -- is excluded and named in the carrier rather than in the tool. THE INTEGER SOLVE REPRODUCES AN INDEPENDENT LEAST-SQUARES: 189 ms/step and 966 microseconds/token, against 189.9 and 966.1 from an ordinary floating-point fit over the same receipts. Scaling is declared rather than hidden -- wall in milliseconds, the per-token rate emitted in microseconds because a ~1 ms/token rate rounds to the useless integer 1 -- and the cross-products use the no-division form n*Sxy - Sx*Sy so no mean is rounded before the solve. Largest intermediate is ~3.1e16 against a 9.2e18 ceiling. AND THE REFUSAL I FIRST WROTE WAS WRONG, WHICH ONLY EXECUTING IT SHOWED. It tested `determinant == 0`, on the belief that a single-budget population makes step count exactly collinear with tokens. Running it against a planted single-budget population refuted that: ceil is NOT linear, the determinant came out at 236,236,900 rather than zero -- against 32,424,810,042,480 for the real population -- and the fit cheerfully returned S = 305 ms/step and b = 906 us/token from noise. A det == 0 guard catches nothing. So the wall now tests the structural property that makes the split possible at all: the population must span at least two budgets, because that is what gives step count information the token count does not already carry. Decidable, no tolerance, and not a smuggled heuristic about how small a determinant is too small. It is not hypothetical either -- the tensor-parallel repeat launches all held the budget at 2048, and their per-step/per-token split had to be withdrawn. EVERY READING NOW SAYS WHICH KIND IT IS. DirectMeasurement, DerivedAtCommonWorkload and PredictedOutsideMeasuredArm are incompatible by construction, because the defect they exist to prevent was a table rendering a measured wall and a model-normalised percentage in adjacent columns with nothing between them. THE CO-TENANT AXIS IS REPAIRED FOR ONE ARM AND HONESTLY INCOMPLETE FOR TWO. The 2048 arm was re-run under an attribution guard that polls the running count and books the engine-wide prompt-token delta against the window's own two requests: 16,451 booked against 16,449 own, max_running 2.0, giving 30.0 ms alone against 2,171.6 ms beside the prefill. That supersedes the itl_* fields in arm-2048.json, which came from the first co-tenant test whose victim finished before the prefill was submitted. The guard also refused three attempts at 8192 -- 311,543 foreign prompt tokens against 16,448 own -- and revealed that NEITHER sibling arm ever had an attribution boundary, because the instrument did not exist when they ran. One attributed reading of three. The refused attempt is committed beside the clean one, because a refusal that leaves no receipt is indistinguishable from never having tried. Notable: one refused window reported 2,172.6 ms, agreeing with the figure this lane had been quoting, and the guard rejected it anyway. Agreement with a prior belief is not attribution. Group A restored to its authored unit and verified. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW
…into serving-evidence-reconstructed
…the two rows it came due on The required run refused with `1 unadjudicated delta`. The delta is the roster's existing `recurring_failure_mode_roster` declaration resolving the spelling `cumulative_metric_read_as_per_event` into the class module this change adds -- NewPoolCoincidenceResolution, which does not auto-admit. That is the shape EVERY new failure-mode class produces: the ledger is a directory of one file per class precisely so two lanes never rewrite one file, so a class arrives as a name newly resolving in the roster's pool rather than as an authored rebind. Needing a row for it is the honest cost of the pool being adjudicated instead of assumed. The same run reported the two gunbc#10688 call-target rows as `already satisfied at the base -- consumed by its own merge`, and their entry said they came due on this roster's next touch. This is that touch, so they are deleted, together with the label constant they shared. Adjudicated by the join they demanded rather than by their own trigger sentence: the run's CONSUMED line IS that tuple join executed against main.
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 8, 2026
#10813 landed the identical motion independently -- it filed a recurring-failure-mode class, hit the same NewPoolCoincidenceResolution binding delta in the generated roster, and deleted the two consumed gunbc#10688 rows on the same reasoning I did. Its side is taken whole and my row is added beside it rather than merged into a rule covering "new class files": this roster's population is an enumeration and never a predicate, which is precisely why two rows of one shape stand side by side. That the shape arrived twice from two lanes in one day is the evidence it is a standing cost of the directory-as-membership design rather than an incident, and the annotation now says so. One thing main did NOT carry: #10813's RECURRING_FAILURE_MODE_CLASS_ADDED_LABEL is absent from the .dag declaration enumeration. That is the decay this very justification row warns about -- "each lane adds honestly and the claim decays silently" -- firing again. Added here on the next touch, with the omission recorded rather than quietly repaired. cargo check -p v1-compiler --lib green; the .dag entry compiles at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 8, 2026
…eview 62662 Both findings are real and both were authored by my merge, not inherited. (1) Two blocks read SEVENTH TRANSITION and the ordinal named two different transitions, with #10783's narrative sitting on #10813's RECURRING_FAILURE_MODE_CLASS_ADDED_LABEL. The duplicate is deleted: #10813's paragraph stays on its constant, mine stays on RECURRING_FAILURE_MODE_CLASS_FILING_LABEL as the EIGHTH, and each doc block now resolves to the symbol it documents (DESIGN section 3: cite the symbol). (2) The gunbc#10688 TRIGGER paragraph survived the deletion of the rows it governed -- a standing trigger sentence over a population of zero, on the wall's own authority surface. It went with the duplicate block; the dissolution paragraph that records why those rows are gone is the one that stays. cargo check -p v1-compiler --lib green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 8, 2026
…s own trigger The floor's second refusal was not my delta -- that one is adjudicated, 0 unadjudicated. It was "1 consumed admission due for deletion on this roster-touching change": #10813's RECURRING_FAILURE_MODE_CLASS_ADDED_LABEL row. Its class file dag/gunbc/recurring_failure_mode/cumulative_metric_read_as_per_event.dag is now on main, so base and head both resolve the spelling into that module and the row is a permission standing over a motion that is no longer happening. Adjudicated by the join the row demanded rather than by its trigger sentence: the run reported it CONSUMED, and the class module is present on main under the exact name the row named as its target. The row, its label constant, and its declaration enumeration entry go together. Recorded beside it, because it is the reusable part: the row is owed for exactly as long as the class is unlanded, and the next roster touch after it lands collects it. Two lanes filed a class on one day and both rows passed through this slot in sequence -- that is the expected shape of a standing cost, not a collision. cargo check -p v1-compiler --lib green; the .dag entry compiles at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 8, 2026
The EIGHTH TRANSITION annotation said "two recurring-failure-mode classes are filed" and, two lines down, "the change files TWO CLASS MODULES AND ONE STALL MODULE". Both are wrong and they disagree with each other. git diff --name-status against main reports three added files: one class module, one guarantee_stall module, one probe-test module. Corrected to that, and the sentence now says WHY only one produced a binding delta rather than only that it did: the stall module is named by a hand-authored roster, so its reference is an authored one, and the probe module is named by nothing. A third clause was stale in the same block -- it said #10813's row and this one stand side by side, which stopped being true one commit ago when #10813's class landed and its trigger collected it. Rewritten to record that they were never generalised into one predicate, which is exactly what let each retire on its own class. No wall behaviour changes: the admission row was already exact and is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey
briansrls
pushed a commit
that referenced
this pull request
Sep 9, 2026
…le (#10783) * A measurement discharges one ceiling sentence and reveals a second, sharper hole DESIGN section 4b requires a ceiling to be derived rather than aspirational, and generic_application_admits_distinct_unit_variant_arguments said its own was unestablished in as many words: 'The diagnostic name says generic-formal undecidable, but no measurement establishes undecidability; do not use that name to declare a permanent ceiling.' A four-arm fixture is that measurement. take_map_concrete declares a CONCRETE m: Map<String, Bool>; map_insert declares the same argument at a GENERIC formal. A String at the concrete formal REFUSES with a located, typed mismatch. The same String at the generic formal is SILENT at that argument. One argument, one shape, two positions, two answers -- so refusal is reachable and genericity is not the reason it does not fire, and a String inhabits Map<K, V> for no instantiation of K and V. That receipt lands on the existing row and discharges its ceiling sentence and NOTHING ELSE. That row's population is variant identity; the fixture's is shape; the row itself forbids borrowing evidence across populations, and the boundary is stated in the receipt so a later reader cannot credit a repair for one to the other. A second arm found a hole that is not that row's subject at all: an unapplied function -- empty_map rather than empty_map() -- admitted at the CONCRETE position, in the same compile where a String at that identical position refuses. No instantiation makes fn() -> T inhabit T, so it is decidable from the argument alone. That gets its own identity rather than a receipt, because two mechanisms sharing one consequence would let a repair for either be credited to both. The new row is rostered at the END of the list: roster order is source order and the projection renders it, so inserting alphabetically would have reordered docs/design-failure-modes.md and destroyed the empty-diff oracle. This is the class that admitted my own defect in gunbc#10779, found by a reader on a diff the compiler reported nothing about. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The probe exists and runs; the enrolment and the stall land with it review 62002 refused two things and both were right: the fixture was described in prose with its output copied into the rows, and the row said arm 4 IS ENROLLED in the present tense while no fixture existed in the tree at all. THE PROBE IS NOW REAL AND IT DISCRIMINATES. Four sources through compile_dag_diagnostic_census, the same harness the duplicate-definition probe uses, folded to a blocking-diagnostic count: a_string_at_a_concrete_position_is_refused PASS positive control an_unapplied_function_at_a_concrete_position... FAIL the defect a_string_at_a_generic_position_is_refused FAIL the deferral the_well_typed_call_is_clean PASS fixture not broken Two greens bracketing two reds is what makes this a discrimination matrix rather than a test that happens to be red. Without the concrete-position control, a red on the function arm would be indistinguishable from a probe whose source broke some other way; without the well-typed control, an import typo would red everything and read as walls landing. The fixture is source handed to the compiler rather than corpus modules because two of the four programs are invalid and one is correctly refused -- as corpus members they would break every build that resolved them. DESIGN section 4b rules exactly that case. THE ASSERTIONS ARE THE CORRECT BEHAVIOUR, NOT THE OBSERVED ONE. Written the other way the rows would pass today and become defenders of the defect: the day someone lands the wall, the test would red and read as a regression caused by the fix. A check that must be deleted to make progress was recording the bug as the specification. ENROLMENT AND THE PROBE LAND TOGETHER, never separately -- two unenrolled reds would block every lane in the program, and there is no acceptable window for that. Both identities go into floor_expected_red, and the stall row is what makes that admissible rather than green-by-suppression: an expected red with a declared way to stop being expected is tracked debt, one without is the untracked stall section 4b(2) forbids. The trigger states the exit explicitly -- on the day both rows refuse, they LEAVE the roster and stay enrolled as permanent regression controls per 4b(4). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The stall went into restored_stalls, which is not the list anything folds review 62023 caught the new stall appended to restored_stalls instead of all_guarantee_stalls. My edit took the LAST list in the file, and the last list is not the roster. That is the section 3c defect this very file already names: a stall declared, typed, valid, and absent from every projection the roster feeds. Nothing folds restored_stalls for rendering -- gunbc.rostered_row_join joins against all_guarantee_stalls -- so the failure-mode row and the expected-red enrolment were both pointing at a tracking consumer that no fold would ever reach. The enrolment's whole admissibility rests on that stall being visible, so this was not cosmetic: it would have left two expected reds tracked by a row nothing renders. It also violates that list's own stated rule -- a new stall row must be admitted WITHOUT editing restored_stalls, which pins four historically restored subjects so their loss refuses -- and it would have broken every_restored_stall_is_still_rostered, since a restored member missing from all_guarantee_stalls is exactly what that fold refuses on. Removed from restored_stalls, appended at the END of all_guarantee_stalls because order there is source order and load-bearing. Projection regenerated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The probe is DORMANT, so the expected-red enrolment asserted something false review 62034 found that test.claim.argument_shape_at_a_declared_position_probe_test matches NO prefix in v2.workflow.required_floor required_gate_prefixes. I checked all thirty mechanically rather than by eye: none matches. The floor therefore suppresses these identities as dormant -- discovered, not run. SO THE ENROLMENT ASSERTED THE ONE THING THAT ROSTER EXISTS TO PREVENT. floor_expected_red's own header: enrolling an identity asserts that it REACHES ITS SUBJECT AND ANSWERS, and it records 101 rows that left it for exactly this reason -- never-executed identities reading as debt someone was working. A dormant red carried as expected debt is the more dangerous of the two mistakes, because a genuine red is loud once someone looks. Enrolment removed. Removing it is safe for the reason that made it look necessary: two unenrolled reds would block every lane, and these do not, precisely BECAUSE they are dormant. A SECOND DEFECT IN THE SAME ROW, AND IT IS THE ONE I SHOULD HAVE CAUGHT: the row said ReadsLiveTree sites have routed to DeclinedLiveTree. That arm IS DELETED -- required_floor.dag says so at its own root. I lifted the sentence from the duplicate-definition probe's historical comment without checking it still held, which is a stale citation standing where a current fact was required. A THIRD: the row called the string-at-generic arm 'the sibling row's subject', contradicting the sibling receipt this same PR adds. That row's subject is variant IDENTITY; this fixture's population is SHAPE. The arm discharges that row's CEILING SENTENCE and nothing else, which is what the receipt already says and what this row now says too. The row and the stall trigger now state the standing plainly: the executing consumer is the named claim_batch invocation and nothing else, and admitting the module to required_gate_prefixes is the prerequisite that would convert re-derivable evidence into enrolled evidence. That admission is a second gate and a cost on every floor run, so it is asked for rather than taken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * Name the gate admission as a trigger so 'not today' cannot decay into never Section 4b(2) requires a class below its ceiling to name its next-rung trigger, separating cannot-climb from can-climb-after-one-grounding from can-climb-now-but-unbuilt. This class is below its ceiling for TWO independent reasons and only one of them was written down. The class trigger was there: the declared-type judgment decides argument shape from the argument's own constructor before consulting the declared position. The EVIDENCE trigger was not, and it is the one actually blocking. The probe matches no prefix in required_gate_prefixes, so a required lane never runs it. That leaves the class PROVEN BUT UNROUTED -- it executes, it discriminates, and no required consumer repeats it, so the rung rests on a human-run command. Admission to that gate is now named as the trigger rather than left as an observation, because an honest sentence about the present state quietly becomes the permanent state if nothing names its end. The condition on asking is in the row too: request the admission when the runner pool is not saturated, and as its OWN change rather than as a rider. It adds four rows to every required floor run, and it was deferred on a day when the pool sat at roughly twenty concurrent runs and main had gone red on memory pressure from that load. Both triggers are ClimbableButUnbuilt: neither waits on a grounding that does not exist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The arm table promised a measurement this module does not contain review 62052 found the annotation laying out a 2x2 including generic+function observed accept, while the module declares three shape arms and a well-typed control. No such row exists. A reader re-deriving the names in the stall's trigger would have found a control where the table promised a fourth measurement -- in the file whose own closing note says WHAT THIS DOES NOT MEASURE, stated so no receipt reads coverage it does not have. Corrected the table rather than adding the arm, and the reason is in the file. The fourth cell cannot come out against anything: acceptance there is already ENTAILED by the two arms above it, since the position defers on every argument and the argument is refused at no position. An arm that cannot produce a result contradicting what its neighbours already establish is not a measurement, it is a fifth witness to author, review, maintain and eventually run, which section 2 counts as redundant work on all three of section 1's quantities. Its provenance is recorded rather than dropped: it was compiled once during the investigation and behaved as entailed. That is not evidence this module carries and it is not cited as any. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * Regenerate the projection for the corrected arm table * Positional arm citations broke the moment I reordered the arm table review 62067 found three defects and all three are one shape: an edit changed the thing and the prose describing it stayed standing and well-formed. A MEANING FORK ON ENROLMENT. The probe comment still said the function arm is enrolled as an expected red, while this same change's typed rows say enrolment was removed because the module is dormant. One change asserting both. The comment now says what is true: the reds are REAL AND UNROUTED rather than held -- nothing suppresses them because nothing runs them -- and it points at the stall that names gate admission as its own trigger. A ROSTER EXIT THAT CANNOT HAPPEN. The row attributed to the stall an exit of LEAVE floor_expected_red. The stall says no such thing, and an identity that is not on a roster cannot leave it. Replaced with what the trigger carries: on the day both rows refuse they flip to permanent regression controls. POSITIONAL CITATIONS, IN THE RULE'S OWN WORDS. arm 3 and arm 4 stopped matching the module the moment the previous commit reordered the arm table -- section 3's stated failure mode exactly: any edit ABOVE the cited line silently invalidates it. The references were positional in three rows and are now the test function names, which the namespace tree can decide. That last one is worth more than its fix. It is not a slip: a review fix broke the citations, and nothing caught it but another review. A position is unreachable from the tree that owns the thing it names, so no gate can see it rot -- which is why the rule says name the symbol, and why I should not have been writing ordinals into rows while citing that rule at other people. Re-verified by execution: the two rows asserting behaviour the compiler already has still PASS, so the annotation edits did not break the fixture. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * wip-receipt * The floor DOES run this probe, and the artifact said so while the row claimed otherwise HELD UNPUSHED: #10783 is parked DIRTY. This is the fix for its CI red, ready for the window. The red is not the host and not a flake. Its typed cause is standing=measurement_completed with four blockers, two of them my own expecting -red arms returning Bool(false) with cause=claim_failed -- a reached answer, not a refusal to compute. I had claimed this module was dormant: it matches no prefix in required_gate_prefixes, so I concluded the floor discovers and does not run it, and REMOVED the expected-red enrolment on that basis. The prefix roster is not the whole admission rule. The floor also plans a witness whose file the change touches, and the landing run's own required-floor-disposition artifact carries all four identities at planned_as_changed_witness -- the only four rows at that disposition in a 16836-row census -- with the two expecting-red arms failed. So the module is dormant on ORDINARY floors and RUN on every change that touches it, which necessarily includes the change that lands it. Un-enrolled, these two block their own landing change while documenting a defect the row admits is real. The enrolment is restored, and it is true by the roster's own rule: an enrolled identity must reach its subject and answer, and both arms evaluate and answer. The 101 rows that roster's header records as removed were removed for the opposite condition -- identities that never ran at all. Precedent for this exact shape is already in the same chunk: guarantee_floor_class_probe_witness.floor_generic_instantiation_is_still_silently_accepted. The row and the stall are corrected to say dormant-on-ordinary-floors rather than not-run, since the artifact answers directly what I had inferred from the prefix list. The gap they report is unchanged: between changes, no required lane repeats this measurement. Verified by compiling floor_expected_red rather than counting brackets: 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * Two statements of the enrolment fact, one of them stale: review 62582 The last commit restored the floor_expected_red enrolment of both expecting-red identities without touching the two places that assert the opposite, so the change carried both sides of one fact -- a DESIGN section 3 meaning fork, and on the row a section 4b(1) rung-honesty defect, since a reader could not tell from the row whether its reds execute. The probe module's annotation said "THEY ARE NOT ENROLLED ... AND MUST NOT BE DESCRIBED AS IF THEY WERE" and read the prefix roster as the whole admission rule. It now states what the landing run's required-floor-disposition artifact actually answers: the module matches no required_gate_prefixes prefix, so no ORDINARY floor repeats it, but the floor also plans a witness whose file the change touches, so every change to this module runs these identities -- which is why the enrolment is true by that roster's own rule. The row's stall field claimed "no roster exit is claimed here, because there is nothing to exit". The premise was wrong and the conclusion is still right for a different reason: section 4b(4) dissolves production handling and never the evidence, so these rows flip to permanent regression controls and STAY enrolled. Both edited files compile at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * One concept answering from two authorities: consume total_blocking_count_for: review 62594 census_of and blocking_count were a byte-for-byte re-implementation of gunbc.compile_census_probe total_blocking_count_for, whose own header declares that module THE SINGLE HOME for "compile this source string and count what it refused" -- written because a second copy would make one concept answer from two authorities (DESIGN section 3), and because the two copies can drift on the failure-arm convention, which is the vacancy-green trap both headers exist to prevent (section 2: net concepts must not grow by re-invention). Both local functions are deleted and the four claims call the canonical helper, the way the sibling algebra_receiver_alias_witness_test already does. The -1-on-CensusNotRunnable rationale is kept as an annotation at the import site and now names it as the helper's arm rather than restating it as a local convention. Compiles at 0 blocking errors. The four identities execute on this change by planned_as_changed_witness, so the required floor run on this head is their consumer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * Filing a class is a binding motion now that the roster is generated: admit it, and retire the two consumed rows The floor refused on namespace-wave-admission with 1 unadjudicated delta. Since membership of the failure-mode ledger became THE DIRECTORY, the roster is generated before compile, so filing a class file makes the generated roster gain a reference to it and the spelling unapplied_function_value_admitted_at_a_value_position goes from binding nothing at the base to binding the new module at the head. The wall reports that as NewPoolCoincidenceResolution, correctly. One row, naming the exact tuple the wall reported rather than a rule about "new class files", because the roster's population is an enumeration and never a predicate. That standing cost is recorded beside the row rather than absorbed: every future class file will owe the same shape. Adding a row makes this change roster-touching, which is what the two gunbc#10688 call_semantics_target rows named as their own due date. Both reported CONSUMED in this run. #10688 is on main -- v1.std.core declares call_semantics_target and v1.compiler.emit_rust imports it -- so the deletion was adjudicated by the per-tuple join against main their trigger demands, not by the trigger sentence. Both rows and CALL_SEMANTICS_TARGET_REHOME_LABEL are deleted, so the roster gains one declaration and loses one. The seed-growth justification's label enumeration is updated in the same change, because that row already records the failure mode where a label lives unenumerated and the "every declaration is enumerated here" sentence decays silently. cargo check -p v1-compiler --lib is green, proven by a control that must fail: appending a deliberate type error to this file reds it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The merge left one lane's paragraph attached to another lane's row: review 62662 Both findings are real and both were authored by my merge, not inherited. (1) Two blocks read SEVENTH TRANSITION and the ordinal named two different transitions, with #10783's narrative sitting on #10813's RECURRING_FAILURE_MODE_CLASS_ADDED_LABEL. The duplicate is deleted: #10813's paragraph stays on its constant, mine stays on RECURRING_FAILURE_MODE_CLASS_FILING_LABEL as the EIGHTH, and each doc block now resolves to the symbol it documents (DESIGN section 3: cite the symbol). (2) The gunbc#10688 TRIGGER paragraph survived the deletion of the rows it governed -- a standing trigger sentence over a population of zero, on the wall's own authority surface. It went with the duplicate block; the dissolution paragraph that records why those rows are gone is the one that stays. cargo check -p v1-compiler --lib green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * #10813's admission row came due on this roster touch: retire it by its own trigger The floor's second refusal was not my delta -- that one is adjudicated, 0 unadjudicated. It was "1 consumed admission due for deletion on this roster-touching change": #10813's RECURRING_FAILURE_MODE_CLASS_ADDED_LABEL row. Its class file dag/gunbc/recurring_failure_mode/cumulative_metric_read_as_per_event.dag is now on main, so base and head both resolve the spelling into that module and the row is a permission standing over a motion that is no longer happening. Adjudicated by the join the row demanded rather than by its trigger sentence: the run reported it CONSUMED, and the class module is present on main under the exact name the row named as its target. The row, its label constant, and its declaration enumeration entry go together. Recorded beside it, because it is the reusable part: the row is owed for exactly as long as the class is unlanded, and the next roster touch after it lands collects it. Two lanes filed a class on one day and both rows passed through this slot in sequence -- that is the expected shape of a standing cost, not a collision. cargo check -p v1-compiler --lib green; the .dag entry compiles at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The receipt counted modules it did not add: review 62682 The EIGHTH TRANSITION annotation said "two recurring-failure-mode classes are filed" and, two lines down, "the change files TWO CLASS MODULES AND ONE STALL MODULE". Both are wrong and they disagree with each other. git diff --name-status against main reports three added files: one class module, one guarantee_stall module, one probe-test module. Corrected to that, and the sentence now says WHY only one produced a binding delta rather than only that it did: the stall module is named by a hand-authored roster, so its reference is an authored one, and the probe module is named by nothing. A third clause was stale in the same block -- it said #10813's row and this one stand side by side, which stopped being true one commit ago when #10813's class landed and its trigger collected it. Rewritten to record that they were never generalised into one predicate, which is exactly what let each retire on its own class. No wall behaviour changes: the admission row was already exact and is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * A count of any blocking diagnostic is not an oracle for a class-specific claim: review 62715 The positive control and both expecting-reds folded through total_blocking_count_for >= 1, which establishes THAT something was refused and never WHAT. Every conclusion these rows carry is class-specific -- "refused, located and typed" on the class row, and the ceiling discharge on its sibling -- so a non-discriminating oracle under a rung claim is the inflation DESIGN 4b(1) names. Two arms are now pinned with blocking_subject_count_for to TypeMismatch / Primitive(String): the class half is the compiler's own histogram key, and for TypeMismatch the census subject is the `got` type, which is the same string the rendered diagnostic prints. Measured on the concrete-string fixture rather than assumed. The function-at-a-concrete-position arm STAYS on the total count, and the annotation now says why: an unapplied function value is accepted at every position today, so no refusal exists whose subject render could be read, and a guessed subject would author an assertion that either never greens when the wall lands or greens for the wrong reason. A wider oracle that is honest about its width beats a narrow one that is invented. That is measured too -- the same value is accepted at a plain Bool position, so the defect is not specific to the generic carrier -- and the arm narrows when the lane that lands the wall can read the subject the compiler actually emits. Compiles at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * The row that says not to copy counts copied one: review 62732 Two receipts apart, this row says "the counts are not copied here, because a copied count is unreachable from the producer that owns it" and then transcribes "the only four rows at that disposition in a 16836-row census". The denominator is a run-local number of the landing run, nothing re-derives it, and it is silently wrong next week -- DESIGN section 6, name the instrument, never transcribe its output. Both rows now name the producer and how to read it: download required-floor-disposition from any completed witnesses run on a change that touches the module, and read the identities BY HEADER NAME. No count survives in either, including the denominator: the load-bearing claim is the disposition those four identities carry, which needs none. The sibling row admission_roster_read_as_the_whole_membership_rule carried the identical sentence and is corrected in the same change. It is the row filed to catch reading errors, so leaving a transcription in it would be the second instance of its own class. Its correction says so rather than quietly rewriting the number away. One wording fix while there: the arms are recorded known-red-held, not failed. That is the disposition the artifact actually carries for an enrolled expecting-red. Both files compile at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey * One identity, two standing claims about its enrolment: review 62747 Two receipts apart this row said the two reds ARE correctly enrolled -- with the disposition artifact as the evidence, and an earlier removal recorded as the error -- and then that prefix admission "is what would convert the probe into ENROLLED evidence, and only then is an expected-red enrolment both true and necessary". Those cannot both be the standing claim for one identity, and the second reopens exactly the removal this row exists to record. The residual gap is real and is kept, stated without denying present enrolment: on every change that touches this module the floor runs these identities, they reach their subject and answer, and un-enrolled they would block their own landing change -- so the enrolment is true and necessary today. What it does not buy is REPETITION on ordinary floors, since a change touching nothing here never re-runs them. required_gate_prefixes admission closes that gap and nothing else; it does not make the enrolment true, because the artifact already does. The superseded reading is recorded in the row rather than deleted, because a row whose whole subject is a misread roster should carry its own. Compiles at 0 blocking errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQuBs4mLu9ArojSWFiRcey --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reconstructed from current main, not from
serving-prefill-scaling. It carries the corrected evidence substrate and deliberately touches no placement, fabric or context-selection authority.Intended as the "remove or replace" input for the #10786 reconstruction, not as a competing PR.
The headline: the budget is 2048, and an exception handler chose it
nvmlDeviceGetMemoryInforeturnsNVML_ERROR_NOT_SUPPORTEDon a GB10 — the memory is unified and there is no discrete partition to report. vLLM'sget_batch_defaultswraps that probe in a bareexcept Exception: device_memory = 0, so0 >= 70 GiBfails and a 121.7 GiB machine takes the small-device branch:max_num_batched_tokens = 2048. Torch reads the pool correctly; only the NVML path refuses.The path never consults the model, so any checkpoint served by this build inherits it — GLM included.
Why the previous refutation was wrong
vllm:iteration_tokens_total_countis not a step counter.async_llm.pybuilds its stats object asIterationStats() if (log_stats and num_outputs) else Noneand the recorder returns early onNone, so it advances once per output batch. A chunked-prefill step emits no output for the request it is advancing, so a prefill-only request advances it exactly once regardless of chunk count — measured at 1 for requests a true per-step counter measured at 3, 4, 7, 9 and 10 steps.Exclusivity does not repair this. It answers whose events are counted; it says nothing about which.
The honest step counter is
vllm:estimated_flops_per_gpu_totalunder--enable-mfu-metrics. Its counts came out as exactlyceil(tokens/2048)at seven sizes, and at the production ceiling as exactly 512 for 1,048,575 tokens (= 1048575/2048).And 1,913 was never fabricated. A co-tenant decoder beside a chunked prefill receives one token per chunk, each token one output batch, so the engine-wide quotient approximates the chunk size by construction. The original inference was right by a mechanism nobody identified, then discarded for a worse reason. That near-miss is the specimen: a wrong-grain quotient can land near the true answer, which is what makes it unfalsifiable by inspection — first as confirmation, then as refutation.
Blockers this implies for #10786 @
ac95284Verified directly against that head:
serving_critical_pathBehaviouralPerIterationCapacityAtLeast { computed_tokens: 20082 }— refuted, still live authoritypair_serving_desiredmax_model_len: 1048576,kv_cache_dtype: KvFp8DsMla,prefill_scheduling: nonepair_serving_capacity_floormaximum_sequence_tokens: 500000,sessions: harness_seat_operator_policy_seatsThe third is why this PR contains no context-window selector. A context/concurrency policy already exists there as a literal; a second selector would have been a second answer to one question.
What was measured
Batch sweep, four launches, group A, restored afterwards:
Stall law
S + b·budgetpredicts every arm within 1%. Per-token cost is flat from 512 to 16,384 — the budget buys only per-step amortisation, which refutes the MoE-granularity hypothesis.Externality: decode alone 30.1 ms/token; decode during a 16,381-token prefill 2168.2 ms/token — 72×, and 2168 ms is one step. That is the ~0.4 tok/s users felt.
TP=2 vs TP=4, two launches per degree, alternated: TP=2 is 7.7% cheaper per token (938 → 1016 tok/s). An earlier single-launch reading of 25.4% is withdrawn — both repeats land at 0.98 on two different Spark pairs.
Numerical: within-launch and across-launch bit-identical (including across different Spark pairs); cross-topology 2.2387 nats in all four pairings. Attribution earned; substitution refused.
Depth:
ms/token = 0.9236 + 1.4159 µs per million tokens of depth, residuals ≤0.05 ms over 123×. Total prefill is quadratic in prompt length, and chunking cannot touch that term — it only movesceil(T/B)·S.Carriers
extdeps.nvidia.management_library— the device-memory query whose refusal arm carries no byte count, sodevice_memory = 0has no constructorextdeps.vllm.batch_defaults— upstream's resolution with the selection cause beside the valueextdeps.vllm.metrics— metric × question pairing;IterationTokensTotalnow refuses the step questiongunbc.spark.prefill_batch_sweep— measured arms and predicted arms as incompatible typesgunbc.spark.tensor_parallel_comparison— topology arms, fleet-normalised view, substitution gatecumulative_metric_read_as_per_event(new),absorbing_fallback(upstream receipt)Scoping (from review)
Three findings are bounded to what established them: the co-tenant law is the accepted=1 special case (speculative decoding changes it, and draft tokens consume the same budget); the NVML finding is platform × runtime revision — a different image must reacquire it; the per-step FLOPs quantum is not a constant (it doubled when ranks halved).
Not established
No interval on decode or stall (single measurements) · which topology is closer to intended semantics · which subsystem carries the depth term · whether the NVML path holds in a GLM image · every coefficient here expires with the DeepSeek profile.
Verification
All carriers 0 blocking errors.
claim_executor --required-ci --required-lane build:phases_failed=0,fixed_point_equal=true, both generated-artifact populations OK. Receipts and harness underdocs/receipts/prefill-batch-sweep/.Group A restored to its authored unit and verified; group B never touched.
🤖 Generated with Claude Code
https://claude.ai/code/session_01TLj9vB2KVcTUA9wvuLa8fW